Yuzhou Tang · Dejun Xu · Yongjie Hou · Zhenzhong Wang · Min Jiang*
Xiamen University
[2024.11.25] 🎈 We have released the codebase for NexusSplats.
- Codebase release
- Pretrained models
- Merge with the latest version of NerfBaselines
we propose a nexus kernel-driven approach, called NexusSplats, for efficient and finer 3D scene reconstruction under complex lighting and occlusion conditions.
Experimental results demonstrate that NexusSplats achieves state-of-the-art rendering quality and reduces reconstruction time in different scenes by up to 70.4% compared to the current best method in quality.
Clone the repository and create a python == 3.11 Anaconda environment with CUDA toolkit 11.8.
Our code is implemented based on NerfBaselines.
Install the dependencies and the codebase:
git clone [email protected]:juliantang324/NexusSplats.git
cd NexusSplats
conda create -y -n ns python=3.11
conda activate ns
conda env config vars set NERFBASELINES_BACKEND=python
pip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 --index-url https://download.pytorch.org/whl/cu118
pip install --upgrade pip
pip install -r requirements.txt
pip install nerfbaselines==1.2.5
pip install -e ./submodules/diff-gaussian-rasterization ./submodules/simple-knn
pip install -e .To start the training on the Photo Tourism dataset, run one of following commands:
ns train --data external://phototourism/trevi-fountain
ns train --data external://phototourism/sacre-coeur
ns train --data external://phototourism/brandenburg-gateTo evaluate the trained model on the Photo Tourism dataset, run the following commands:
# render predictions
ns render --checkpoint {checkpoint} --data external://phototourism/trevi-fountain --output {output_path}
ns render --checkpoint {checkpoint} --data external://phototourism/sacre-coeur --output {output_path}
ns render --checkpoint {checkpoint} --data external://phototourism/brandenburg-gate --output {output_path}
# evaluate predictions
ns evaluate {path/to/predictions} --output results.jsonThere are several concurrent works that also aim to extend 3DGS to handle in-the-wild scenarios:
- WildGaussians: 3D Gaussian Splatting in the Wild
- Gaussian in the Wild: 3D Gaussian Splatting for Unconstrained Image Collections
We sincerely appreciate the authors of 3DGS and NerfBaselines for their great work and released code. Please follow their licenses when using our code.
If you find our code or paper useful, please star this repository and cite:
@article{tang2024nexussplats,
title={NexusSplats: Efficient 3D Gaussian Splatting in the Wild},
author={Tang, Yuzhou and Xu, Dejun and Hou, Yongjie and Wang, Zhenzhong and Jiang, Min},
journal={arXiv preprint arXiv:2411.14514},
year={2024}
}