| Presentation | 2023-09-30 Shared Neural Representations of Semantic Categories for Images and Words Kai Nakajima, Jion Tominaga, Dmitry Patashov, Keita Tanaka, Akihiko Tsukahara, Hiroki Miyanaga, Shoji Tsunematsu, Rieko Osu, Hiromu Sakai, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | Even when objects are presented as words or images, humans can identify their semantic categories. The extent to which these neural representations of semantic categories are modality-independent remains unclear and has not fully understood yet. Previous research using Magnetencephalography data and cross-decoding methods demonstrated that numerical concepts represented in symbolic formats, such as digits and dot patterns, share a common neural representation (Teichmann et al, 2018) and that there are shared representations of superordinate categories (e.g., “animal” for dogs and cats) between images and words (Dirani and Pylkkanen, 2023). The goal of this study is to examine further details of shared representations of eight distinct categories (animal, human, body part, vehicle, food, inanimate object, artificial place, and tool/artifact) by employing cross-decoding techniques on MEG signals. Participants performed two tasks while MEG data were recorded: orally naming images and rating word familiarity. We trained an SVM model based on image and word data, examining classification accuracy for categories, and computed the cross-decoding accuracy in scenarios where the model was trained on image data and tested on word data, and vice versa. Our results indicate that picture data yielded higher accuracy in early time windows than word data, and that cross-decoding accuracies peaked in early time windows for the image dataset trained on word data. The overall results suggest that there are early shared neural representations between the modalities. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | semantic categoryneural decodingmagnetencephalographymachine learning |
| Paper # | TL2023-16 |
| Date of Issue | 2023-09-23 (TL) |
| Conference Information | |
| Committee | TL |
|---|---|
| Conference Date | 2023/9/30(2days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | University of Tokyo |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | Language Processing and Language Learning |
| Chair | Miwa Morishita(Kobe Gakuin Univ.) |
| Vice Chair | Yasushi Tsubota(Kyoto Inst. of Tech.) / Akinori Takada(Ferris Univ.) |
| Secretary | Yasushi Tsubota(Osaka Electro-Comm. Univ.) / Akinori Takada(Miidas) |
| Assistant | Hiroaki Yamada(Tokyo Inst. of Tech) / Akio Shimogori(Hakodate-ct) |
| Paper Information | |
| Registration To | Technical Committee on Thought and Language |
|---|---|
| Language | ENG |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Shared Neural Representations of Semantic Categories for Images and Words |
| Sub Title (in English) | A Study Using Decoding Analysis of MEG Data |
| Keyword(1) | semantic categoryneural decodingmagnetencephalographymachine learning |
| 1st Author's Name | Kai Nakajima |
| 1st Author's Affiliation | Waseda University(Waseda Univ.) |
| 2nd Author's Name | Jion Tominaga |
| 2nd Author's Affiliation | Waseda University(Waseda Univ.) |
| 3rd Author's Name | Dmitry Patashov |
| 3rd Author's Affiliation | Waseda University(Waseda Univ.) |
| 4th Author's Name | Keita Tanaka |
| 4th Author's Affiliation | Tokyo Denki University(TDU) |
| 5th Author's Name | Akihiko Tsukahara |
| 5th Author's Affiliation | Tokyo Denki University(TDU) |
| 6th Author's Name | Hiroki Miyanaga |
| 6th Author's Affiliation | Sumitomo Heavy Industries, Ltd.(SHI) |
| 7th Author's Name | Shoji Tsunematsu |
| 7th Author's Affiliation | Sumitomo Heavy Industries, Ltd.(SHI) |
| 8th Author's Name | Rieko Osu |
| 8th Author's Affiliation | Waseda University(Waseda Univ.) |
| 9th Author's Name | Hiromu Sakai |
| 9th Author's Affiliation | Waseda University(Waseda Univ.) |
| Date | 2023-09-30 |
| Paper # | TL2023-16 |
| Volume (vol) | vol.123 |
| Number (no) | TL-197 |
| Page | pp.pp.3-8(TL), |
| #Pages | 6 |
| Date of Issue | 2023-09-23 (TL) |