| Presentation | 2016-11-17 [Poster Presentation] Analysis of Multimodal Deep Neural Networks Yoh-ichi Mototake, Takashi Ikegami, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | With the rapid development of information technology in recent years, several machine learning algorithms that integrate some information which have different modalitiessuch as images, texts, and sounds for generating sentences and any other have been put to practical use. On the other hand, little is known about the mechanisms by which our modalities are integrated. The purpose of this study is to explore the mechanisms through the analysis of the internal stateof Multimodal Deep Neural Networks(DNNs) whose learning algorithms are rapidly developed in recent years. First, we assumed that datasets have structures based on the manifold hypothesis, and then computed the manifolds' tangent spaces from the mapping function of DNNs. Finally, we calculated how the geometric structures were converted inside the DNNs. The result of the analysis suggested that Multimodal DNNs have functionsto map manifold structures of the distribution of datasets to a global coordinate system, andthat is processed after the integration of the modalities. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | Multimodal Deep Learning / Manifold Hypothesis |
| Paper # | IBISML2016-97 |
| Date of Issue | 2016-11-09 (IBISML) |
| Conference Information | |
| Committee | IBISML |
|---|---|
| Conference Date | 2016/11/16(3days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | Kyoto Univ. |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | Information-Based Induction Science Workshop (IBIS2016) |
| Chair | Kenji Fukumizu(ISM) |
| Vice Chair | Masashi Sugiyama(Univ. of Tokyo) / Hisashi Kashima(Kyoto Univ.) |
| Secretary | Masashi Sugiyama(Univ. of Tokyo) / Hisashi Kashima(Nagoya Inst. of Tech.) |
| Assistant | Toshihiro Kamishima(AIST) / Tomoharu Iwata(NTT) |
| Paper Information | |
| Registration To | Technical Committee on Infomation-Based Induction Sciences and Machine Learning |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | [Poster Presentation] Analysis of Multimodal Deep Neural Networks |
| Sub Title (in English) | Towards the elucidation of the modality integration mechanism |
| Keyword(1) | Multimodal Deep Learning |
| Keyword(2) | Manifold Hypothesis |
| 1st Author's Name | Yoh-ichi Mototake |
| 1st Author's Affiliation | University of Tokyo(unit of Tokyo) |
| 2nd Author's Name | Takashi Ikegami |
| 2nd Author's Affiliation | University of Tokyo(unit of Tokyo) |
| Date | 2016-11-17 |
| Paper # | IBISML2016-97 |
| Volume (vol) | vol.116 |
| Number (no) | IBISML-300 |
| Page | pp.pp.369-373(IBISML), |
| #Pages | 5 |
| Date of Issue | 2016-11-09 (IBISML) |