| Presentation | 2023-05-19 Incorporating Signed Distance Fields to Improve Text-to-3D Generation Zhuofan Sun, Daichi Horita, Satoshi Ikehata, Kiyoharu Aizawa, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | Frameworks for generating 3D objects from text description have been proposed in recent years. These frameworks utilize pre-trained image language models like CLIP or diffusion models to optimize the 3D scene, so that rendered images from any angle of the 3D scene matches the text provided. In these frameworks, 3D scenes are represented by Neural Radiance Fields (NeRF), which has been shown to be suitable for novel-view synthesis tasks. However, NeRF-based methods tend to produce meshes with noisy surfaces. This research intends to solve this problem by incorporating Signed Distance Fields (SDFs) into existing text-to-3D frameworks since SDFs are shown to be suitable for surface reconstruction. We compared the results of NeRF-based and SDF-based approaches and demonstrated that SDF-based methods are able to generate more refined surfaces by suppressing jaggy and floating artifacts that were often observed in results using NeRF-based methods. We also conducted user studies and showed that results SDF-based approaches are not only more refined but also more favorable to users. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | Text-to-3D / Signed Distance Fields / Neural Radiance Fields / CLIP / Diffusion models |
| Paper # | PRMU2023-9 |
| Date of Issue | 2023-05-11 (PRMU) |
| Conference Information | |
| Committee | PRMU / IPSJ-CVIM |
|---|---|
| Conference Date | 2023/5/18(2days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | Seiichi Uchida(Kyushu Univ.) |
| Vice Chair | Takuya Funatomi(NAIST) / Mitsuru Anpai(Denso IT Lab.) |
| Secretary | Takuya Funatomi(CyberAgent) / Mitsuru Anpai(Univ. of Tokyo) |
| Assistant | Nakamasa Inoue(Tokyo Inst. of Tech.) / Yasutomo Kawanishi(Riken) |
| Paper Information | |
| Registration To | Technical Committee on Pattern Recognition and Media Understanding / Special Interest Group on Computer Vision and Image Media |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Incorporating Signed Distance Fields to Improve Text-to-3D Generation |
| Sub Title (in English) | |
| Keyword(1) | Text-to-3D |
| Keyword(2) | Signed Distance Fields |
| Keyword(3) | Neural Radiance Fields |
| Keyword(4) | CLIP |
| Keyword(5) | Diffusion models |
| 1st Author's Name | Zhuofan Sun |
| 1st Author's Affiliation | The University of Tokyo(Univ. of Tokyo) |
| 2nd Author's Name | Daichi Horita |
| 2nd Author's Affiliation | The University of Tokyo(Univ. of Tokyo) |
| 3rd Author's Name | Satoshi Ikehata |
| 3rd Author's Affiliation | National Institute of Informatics/Tokyo Institute of Technology/The University of Tokyo(NII) |
| 4th Author's Name | Kiyoharu Aizawa |
| 4th Author's Affiliation | The University of Tokyo(Univ. of Tokyo) |
| Date | 2023-05-19 |
| Paper # | PRMU2023-9 |
| Volume (vol) | vol.123 |
| Number (no) | PRMU-30 |
| Page | pp.pp.45-50(PRMU), |
| #Pages | 6 |
| Date of Issue | 2023-05-11 (PRMU) |