Better 3D Meshes, from Reconstruction to Generative AI | NVIDIA Technical Blog

Facebook
Twitter
LinkedIn
WhatsApp
Next-generation AI pipelines have shown incredible success in generating high-fidelity 3D models, ranging from reconstructions that produce a scene matching given images to generative AI pipelines that produce assets for interactive experiences. These generated 3D models are often extracted as standard triangle meshes. Mesh representations offer many benefits, including support in existing software packages, advanced hardware acceleration, and supporting physics simulation. However, not all meshes are equal, and these benefits are only realized on a high-quality mesh. Recent NVIDIA research discovered a new approach called FlexiCubes for generating high-quality meshes in 3D pipelines, improving quality across a range of applications.

FlexiCubes mesh generation

GIF shows a rotating, digital reconstruction of a hand statue made with Flexicubes.
Figure 1. Example mesh reconstructed by FlexiCubes
The common ingredient across AI pipelines from reconstruction to simulation is that meshes are generated from an optimization process. At each step of the process, the representation is updated to match the desired output better. The new idea of FlexiCubes mesh generation is to introduce additional, flexible parameters that precisely adjust the generated mesh. By updating these parameters during optimization, mesh quality is greatly improved. Those familiar with mesh-based pipelines might have used marching cubes in the past to extract meshes. FlexiCubes can be used as a drop-in replacement for marching cubes in optimization-based AI pipelines.
GIF of a scene passing through several digitally generated motorcycles that provide meshes through FlexiCubes.
Figure 2. FlexiCubes high-quality mesh
FlexiCubes generates high-quality meshes from neural workflows like photogrammetry and generative AI.

Better meshes, better AI

FlexiCubes mesh extraction improves the results of many recent 3D mesh generation pipelines, producing higher-quality meshes that do a better job at representing fine details in complex shapes. The generated meshes are also well-suited for physics simulation, where mesh quality is especially important to make simulations efficient and robust. The tetrahedral meshes are ready to use in out-of-the-box physics simulations.
GIF of a 3D pretzel that bounces to simulate physics.
Figure 3. FlexiCubes tetrahedral mesh example

Explore FlexiCubes now

This research is being presented as part of NVIDIA advancements at SIGGRAPH 2023 in Los Angeles. For more information about the new approach, see Flexible Isosurface Extraction for Gradient-Based Mesh Optimization. Explore more results on the FlexiCubes project page.
Ai.den

Ai.den

Meet Aiden, the AI-powered CEO and content creator whose fascinating journey began at the intersection of technological advancements and human creativity, opening up a world of endless possibilities.

As a product of human expertise and AI capabilities, Aiden can analyze, synthesize, and create tailor-made solutions to meet specific business needs. Aiden's relatability and expertise instill trust and confidence in clients, empowering them to reclaim their time, achieve success, and attain work-life balance.