Rebuttal of ICLR2025-Paper1469

DiffGS: Repurposing Image Diffusion Models for Scalable Gaussian Splatting Generation

• For Reviewer Xhe6, please refer to Section A, E.
• For Reviewer i4mY, please refer to Section B, C.
• For Reviewer 2Rvk, please refer to Section D, B.
• For Reviewer L5NX, please refer to our offical comments in OpenReview for quantitative results.

Table of Contents

• Section A. Compared with CLAY (Rodin Gen-1)
• Section B. Compared with Amortized SDS-based method, LATTE3D
• Section C. Compared with SDS-based method, GaussianDreamer
• Section D. Generation of Very Thin Objects
• Section E. More Visualizatin Results

We evaluated a commercial product named Rodin via membership subscription on its website, which is based on the CLAY technique, since CLAY itself is not open-sourced.

It is worth noting that Rodin:

1. is trained on proprietary internal datasets
2. originates from CLAY, which was initially trained on 256 A800 GPUs for 15 days
3. has undergone months of updates incorporating sophisticated techniques such as RLHF
4. employs a complex generation pipeline composed of multiple distinct models (text-to-image, image-to-raw 3D, 3D object captioning and attribute prediction, 3D geometry refinement, PBR material generation and refinement, etc.)
5. take about 1 minute to generate a 3D object though its pipeline

Despite these extensive resources and refinement steps, Rodin (Gen-1 RLHF V0.9) still:

• struggles to generate intricate and precise structures such as leaves and fur
• often fails to faithfully adhere to the provided image and text conditions

In contrast, our method is trained on open-source datasets using only 8 A100 GPUs over 2~5 days, demonstrating greater efficiency and accessibility. Moreover, we believe incorporating refinement stages in the proposed method is a promising direction for future exploration.

Results of LATTE3D are taken from its project page.

Our results demonstrate competitive visual quality with less over-saturation compared to the SDS-based MVDream, while achieving significantly faster inference speed. Although our method is slightly slower than the feed-forward (non-generative) approach LATTE3D, it offers better alignment with text prompts.

Results of GaussianDreamer are taken from its project page.

Our results demonstrate competitive visual quality with less over-saturation compared to the SDS-based GaussianDreamer, while achieving significantly faster inference speed. Similar to GaussianDreamer that use Shap-E to generate initial point clouds, our generated 3D Gaussians can also be used as input to the SDS-based methods for further refinement, which could be a promising direction for future research.

To evaluate the 3D consistency of our generated multi-view Gaussian latents, we take prompts from T3Bench or generated by ChatGPT that describe very thin objects. The results demonstrate that our method can generate 3D coherent multi-view splatter images for thin objects without obvious artifacts or distortions.

Text prompts are taken from T3Bench, Instant3D and LATTE3D.
Single-view images are taken from InstantMesh, Era3D, GPTEval3D and the Internet.