NVIDIA on Friday launched Magic3D, a new AI tool that automatically generates 3D models based on text descriptions. For example, after users type in “a blue poison dart frog sitting on a water lily,” Magic3D generates a 3D mesh model with colored textures in about 40 minutes.
Nvidia is taking Magic3D up against Google’s DreamFusion. DreamFusion is a text-to-3D model AI tool announced by Google researchers in September. Similar to how DreamFusion uses a text-to-image model to generate 2D images that are then optimized to volumetric NeRF (Neural Radiation Field) data, Magic3D’s rendering process is also a two-step process that optimizes a low-resolution generated rough model to high-resolution .
Magic3D also enables hint-based editing of 3D models. Given a low-resolution 3D model and a basic hint, the text can be altered to alter the resulting model. Also, the authors of Magic3D show a theme that has been preserved in the past few editions (this concept is often called consistency) and apply the style of two-dimensional images (such as cubist paintings) to three-dimensional models.
Nvidia did not release any Magic3D code with its academic papers.
