| Presentation | 2023-05-18 Image Harmonization Using Diffusion Model for Perceptual Quality Improvement Taito Naruki, Norimichi Ukita, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | Image harmonization is the task of eliminating the discomfort of color tones that occurs when images are composited. However, conventional image harmonization datasets were not able to take into account the perceptual aspect of discomfort, and models were learned and evaluated only by the difference from the GT. For this reason, even if the generated image was not uncomfortable, if the difference from the GT was large, it was treated as an insufficiently trained image. Recently, the diffusion model has been attracting attention. The diffusion model generates images by gradually removing noise from a noisy image over multiple steps, and is known for its higher quality and diversity of generated images compared to other generative models due to this gradually restoring generative process and stable objective function. In addition, the method called DPS applies an inverse problem to the diffusion model to restore an image in zero shots such that it satisfies a given condition. In the proposed method, the image is generated by applying constraints based on image gradients and VGG features in DPS, which results in image harmonization with high quality while preserving foreground textures. Furthermore, by gradually increasing the constraints at each step of the reverse process, the restoration is performed with higher perceptual quality. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | Image harmonization / Diffusion model / Inverse problem |
| Paper # | PRMU2023-1 |
| Date of Issue | 2023-05-11 (PRMU) |
| Conference Information | |
| Committee | PRMU / IPSJ-CVIM |
|---|---|
| Conference Date | 2023/5/18(2days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | Seiichi Uchida(Kyushu Univ.) |
| Vice Chair | Takuya Funatomi(NAIST) / Mitsuru Anpai(Denso IT Lab.) |
| Secretary | Takuya Funatomi(CyberAgent) / Mitsuru Anpai(Univ. of Tokyo) |
| Assistant | Nakamasa Inoue(Tokyo Inst. of Tech.) / Yasutomo Kawanishi(Riken) |
| Paper Information | |
| Registration To | Technical Committee on Pattern Recognition and Media Understanding / Special Interest Group on Computer Vision and Image Media |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Image Harmonization Using Diffusion Model for Perceptual Quality Improvement |
| Sub Title (in English) | |
| Keyword(1) | Image harmonization |
| Keyword(2) | Diffusion model |
| Keyword(3) | Inverse problem |
| 1st Author's Name | Taito Naruki |
| 1st Author's Affiliation | Toyota Technological Institute(TTI) |
| 2nd Author's Name | Norimichi Ukita |
| 2nd Author's Affiliation | Toyota Technological Institute(TTI) |
| Date | 2023-05-18 |
| Paper # | PRMU2023-1 |
| Volume (vol) | vol.123 |
| Number (no) | PRMU-30 |
| Page | pp.pp.1-5(PRMU), |
| #Pages | 5 |
| Date of Issue | 2023-05-11 (PRMU) |