DTU118
Texture-GS fetches the color from the reconstructed 2D texture image during real-time rendering, thereby enabling various interactive editing applications such as texture swapping and texture painting.
3D Gaussian splatting, emerging as a groundbreaking approach, has drawn increasing attention for its capabilities of high-fidelity reconstruction and real-time rendering. However, it couples the appearance and geometry of the scene within the Gaussian attributes, which hinders the flexibility of editing operations, such as texture swapping. To address this issue, we propose a novel approach, namely Texture-GS, to disentangle the appearance from the geometry by representing it as a 2D texture mapped onto the 3D surface, thereby facilitating appearance editing. Technically, the disentanglement is achieved by our proposed texture mapping module, which consists of a UV mapping MLP to learn the UV coordinates for the 3D Gaussian centers, a local Taylor expansion of the MLP to efficiently approximate the UV coordinates for the ray-Gaussian intersections, and a learnable texture to capture the fine-grained appearance. Extensive experiments on the DTU dataset demonstrate that our method not only facilitates high-fidelity appearance editing but also achieves real-time rendering on consumer-level devices, e.g. a single RTX 2080 Ti GPU.
We propose to allow pixels covered by a single Gaussian to have different UV coordinates, which encourages a smooth texture for the convenience of appearance editing.
Texture-GS fetches the color from the reconstructed 2D texture image during real-time rendering, thereby enabling various interactive editing applications such as texture swapping and texture painting.
Our method is not only capable of reconstructing high-fidelity textures from multi-view images, but also enables real-time rendering speed on consumer-level devices, e.g. a single RTX 2080 Ti GPU.
@misc{xu2024texturegs,
title={Texture-GS: Disentangling the Geometry and Texture for 3D Gaussian Splatting Editing},
author={Tian-Xing Xu and Wenbo Hu and Yu-Kun Lai and Ying Shan and Song-Hai Zhang},
year={2024},
eprint={2403.10050},
archivePrefix={arXiv},
primaryClass={cs.CV}
}