SGD: Street View Synthesis with Gaussian Splatting and Diffusion Prior
IEEE/CVF Winter Conference on Applications of Computer Vision(2025)
ETH Zürich
Abstract
Novel View Synthesis (NVS) for street scenes play a critical role in theautonomous driving simulation. The current mainstream technique to achieve itis neural rendering, such as Neural Radiance Fields (NeRF) and 3D GaussianSplatting (3DGS). Although thrilling progress has been made, when handlingstreet scenes, current methods struggle to maintain rendering quality at theviewpoint that deviates significantly from the training viewpoints. This issuestems from the sparse training views captured by a fixed camera on a movingvehicle. To tackle this problem, we propose a novel approach that enhances thecapacity of 3DGS by leveraging prior from a Diffusion Model along withcomplementary multi-modal data. Specifically, we first fine-tune a DiffusionModel by adding images from adjacent frames as condition, meanwhile exploitingdepth data from LiDAR point clouds to supply additional spatial information.Then we apply the Diffusion Model to regularize the 3DGS at unseen views duringtraining. Experimental results validate the effectiveness of our methodcompared with current state-of-the-art models, and demonstrate its advance inrendering images from broader views.
MoreTranslated text
Key words
Street View,View Synthesis,Spatial Information,Point Cloud,Diffusion Model,Adjacent Frames,Fine-tuned Model,LiDAR Point Clouds,Street Scenes,3D Gaussian,Loss Function,Gaussian Model,Training Stage,Reference Image,Depth Map,Depth Information,Training Step,Training Speed,3D Information,Variational Autoencoder,Structure From Motion,KITTI Dataset,Objects In The Scene,Target View,Camera Pose,Spherical Harmonics,Image Guidance
PDF
View via Publisher
AI Read Science
Must-Reading Tree
Example

Generate MRT to find the research sequence of this paper
Data Disclaimer
The page data are from open Internet sources, cooperative publishers and automatic analysis results through AI technology. We do not make any commitments and guarantees for the validity, accuracy, correctness, reliability, completeness and timeliness of the page data. If you have any questions, please contact us by email: report@aminer.cn
Chat Paper
Summary is being generated by the instructions you defined