Nvidia Shows Off AI Model That Turns A Few Dozen Snapshots Into A 3D-Rendered Scene

Nvidia’s latest AI demo is pretty impressive: a tool that quickly turns a “few dozen” 2D snapshots into a 3D-rendered scene. The tool is called Instant NeRF, referring to “neural radiance fields” a technique developed by researchers from UC Berkeley, Google Research.

The method maps the color and light intensity of different 2D shots, then generates data to connect these images from different vantage points and render a finished 3D scene. In addition to images, the system requires data about the position of the camera.

Nvidia says its new Instant NeRF model is one of the fastest yet developed and reduces rendering time from a few minutes to a process that is finished almost instantly. As the technique becomes quicker and easier to implement, it could be used for all sorts of tasks, says Nvidia in a blog post describing the work.

Instant NeRF could be used to create avatars or scenes for virtual worlds, to capture video conference participants and their environments in 3D, or to reconstruct scenes for 3D digital maps writes Nvidia’s Isha Salian. 

The technology could be used to train robots and self-driving cars to understand the size and shape of real-world objects by capturing 2D images or video footage of them.  It could also be used in architecture and entertainment to rapidly generate digital representations of real environments that creators can modify and build on.