Scene Dreamer

SceneDreamer is an AI model that creates unbounded, realistic 3D scenes from 2D image collections without manual 3D annotations. The model efficiently generates diverse landscapes across different styles with well-defined depth and allows free camera movement.

Key Information

• 3D Scene Representation: Uses a bird's-eye-view (BEV) representation consisting of a height field for surface elevation and a semantic field for scene semantics.
• Generative Scene Parameterization: Parameterizes latent features using a generative neural hash grid, aligning content and encoding generalizable features across scenes.
• Neural Volumetric Renderer: Employs a neural renderer trained on 2D images to produce photorealistic images through volume rendering.
• Training: Trained end-to-end on in-the-wild 2D images.

• Advantages:

  • Generates large-scale, unbounded 3D scenes with diverse styles.
  • Ensures 3D consistency.
  • Allows free camera trajectory.
  • Outperforms existing methods in generating vivid and diverse 3D worlds.