RTX Neural Shaders Bring AI to Programmable Shaders
NVIDIA today unveiled next-generation hardware for gamers, creators, and developers—the GeForce RTX 50 Series desktop and laptop GPUs. Alongside these GPUs, NVIDIA introduced NVIDIA RTX Kit, a suite of neural rendering technologies to ray trace games with AI, render scenes with immense geometry, and create game characters with lifelike visuals.
RTX Neural Shaders
25 years ago NVIDIA introduced GeForce and programmable shaders, unlocking two decades of innovation in graphics technology, from pixel shading to compute shading to real-time ray tracing. Alongside the new GeForce RTX 50 Series GPUs powered by NVIDIA Blackwell architecture, NVIDIA is introducing RTX Neural Shaders, which bring small neural networks into programmable shaders, enabling a new era of graphics innovation.
The RTX Neural Shaders SDK enables developers to train their game data and shader code on an RTX AI PC and accelerate their neural representations and model weights with NVIDIA Tensor Cores at runtime. During training, neural game data is compared to the output of the traditional data and is refined over multiple cycles. Developers can simplify the training process with Slang, a shading language that splits large, complex functions into smaller pieces that are easier to handle.
Applications of Neural Shaders
This breakthrough technology is used for three applications: RTX Neural Texture Compression, RTX Neural Materials, and Neural Radiance Cache (NRC).
- RTX Neural Texture Compression: uses AI to compress thousands of textures in less than a minute. Their neural representations are stored or accessed in real time or loaded directly into memory without further modification. The neurally compressed textures save up to 7x more VRAM or system memory than traditional block compressed textures at the same visual quality.
- RTX Neural Materials: uses AI to compress complex shader code typically reserved for offline materials and built with multiple layers such as porcelain and silk. The material processing is up to 5x faster, making it possible to render film-quality assets at game-ready frame rates.
- RTX Neural Radiance Cache: uses AI to learn multi-bounce indirect lighting to infer an infinite amount of bounces after the initial one to two bounces from path traced rays. This offers better path traced indirect lighting and performance versus path traced lighting without a radiance cache. NRC is now available through the RTX Global Illumination SDK, and will be available soon through RTX Remix and Portal with RTX.
Get Started
GeForce RTX 50 Series introduces the next era of rendering and AI. Train and deploy AI directly within shaders for better compression and approximation techniques. Build more complex open worlds with breakthrough BVH building techniques. Leverage new digital human rendering technologies with real-time performance. Use the latest ray tracing algorithms powered by AI to simulate accurate light transport. Transform traditional game characters into AI-powered autonomous versions for new gameplay experiences. Access the RTX Kit, DLSS 4, Reflex 2, and NVIDIA ACE technologies through the Game Development resource hub.
Models for Perception, Cognition, and Action
NVIDIA ACE models for perception, cognition, and action are coming soon in early access. These models will enable developers to create more realistic and engaging game characters with lifelike decision making and actions.
- Models for Perception: NemoAudio 4B Instruct, Cosmos Nemotron 4B 128K Instruct, Parakeet CTC XXL 1.1B Multilingual, and NV-OCR.
- Models for Cognition: Mistral Nemo Minitron 8B 128K Instruct, Mistral Nemo Minitron 4B 128K Instruct, and Mistral Nemo Minitron 2B 128K Instruct.
- Models for Action: A2 Flow.
Conclusion
The RTX Neural Shaders and NVIDIA RTX Kit bring a new era of graphics innovation, enabling developers to create more realistic and engaging game characters with lifelike decision making and actions. With the RTX Neural Shaders SDK, developers can train and deploy AI directly within shaders for better compression and approximation techniques. The RTX Kit and NVIDIA ACE technologies will be available through the Game Development resource hub.
FAQs
Q: What is RTX Neural Shaders?
A: RTX Neural Shaders is a breakthrough technology that brings small neural networks into programmable shaders, enabling a new era of graphics innovation.
Q: What are the applications of RTX Neural Shaders?
A: RTX Neural Shaders is used for RTX Neural Texture Compression, RTX Neural Materials, and Neural Radiance Cache (NRC).
Q: How do I get started with RTX Neural Shaders?
A: Developers can get started with the RTX Neural Shaders and RTX Neural Texture Compression SDKs at the end of the month through NVIDIA RTX Kit. Sign up to be notified of availability.
Q: What is NVIDIA ACE?
A: NVIDIA ACE is a suite of neural rendering technologies that enables developers to create more realistic and engaging game characters with lifelike decision making and actions.
