| Presentation | 2024-02-19 CLIP-based Zero-shot In-Distribution Detection Atsuyuki Miyai, Qing Yu, Go Irie, Kiyoharu Aizawa, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | Extracting in-distribution (ID) images from noisy images scraped from the Internet is an important preprocessing for constructing datasets, which can be partially addressed by zero-shot out-of-distribution (OOD) detection. However, the existing setting does not consider the realistic case where an image has both ID objects and OOD objects. As we can see why MS-COCO was created, it is crucial to identify images containing not only ID objects but also both ID and OOD objects as ID images to create robust recognizers. In this paper, we propose a novel problem setting called zero-shot in-distribution (ID) detection, where we identify images containing ID objects as ID images (even if they contain OOD objects), and images lacking ID objects as OOD images without any training. To solve this problem, we present a simple and effective approach, Global-Local Maximum Concept Matching (GL-MCM), based on both global and local visual-text alignments of CLIP features. Extensive experiments demonstrate that GL-MCM outperforms comparison methods on both multi-object datasets and single-object ImageNet benchmarks. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | CLIP |
| Paper # | ITS2023-53,IE2023-42 |
| Date of Issue | 2024-02-12 (ITS, IE) |
| Conference Information | |
| Committee | ITS / IE / ITE-MMS / ITE-ME / ITE-AIT |
|---|---|
| Conference Date | 2024/2/19(2days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | Hokkaido Univ. |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | Image Processing, etc. |
| Chair | Yusuke Takatori(Kanagawa Inst. of Tech.) / Hiroyuki Bandoh(NTT) / Kenji Machida(NHK) / Shogo Muramatsu(Niigata Univ.) / Hisaki Nate(Tokyo Polytechnic Univ.) |
| Vice Chair | Tetsuya Manabe(Saitama Univ.) / Shintaro Ono(Fukuoka Univ.) / Yuichi Tanaka(Osaka Univ.) / Toshihiko Yamazaki(Univ. of Tokyo) / / Shogo Tokai(Univ. of Fukui) |
| Secretary | Tetsuya Manabe(Toyama Prefectural Univ.) / Shintaro Ono(Gunma Univ.) / Yuichi Tanaka(NHK) / Toshihiko Yamazaki(Tottori Univ.) / (Yamanashi Univ.) / Shogo Tokai(NHK) / (Hokkaido Univ.) |
| Assistant | Taishi Swabe(NAIST) / Kazunori Uruma(Kogakuin Univ.) / Yoshitaka Kitani(KDDI Research) |
| Paper Information | |
| Registration To | Technical Committee on Intelligent Transport Systems Technology / Technical Committee on Image Engineering / Technical Group on Multi-media Storage / Technical Group on Media Engineering / Technical Group on Artistic Image Technology |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | CLIP-based Zero-shot In-Distribution Detection |
| Sub Title (in English) | |
| Keyword(1) | CLIP |
| 1st Author's Name | Atsuyuki Miyai |
| 1st Author's Affiliation | The Univerisity of Tokyo(UTokyo) |
| 2nd Author's Name | Qing Yu |
| 2nd Author's Affiliation | The Univerisity of Tokyo(UTokyo) |
| 3rd Author's Name | Go Irie |
| 3rd Author's Affiliation | Tokyo University of Science(TUS) |
| 4th Author's Name | Kiyoharu Aizawa |
| 4th Author's Affiliation | The Univerisity of Tokyo(UTokyo) |
| Date | 2024-02-19 |
| Paper # | ITS2023-53,IE2023-42 |
| Volume (vol) | vol.123 |
| Number (no) | ITS-380,IE-381 |
| Page | pp.pp.40-45(ITS), pp.40-45(IE), |
| #Pages | 6 |
| Date of Issue | 2024-02-12 (ITS, IE) |