| Presentation | 2018-06-28 Study of improving speech intelligibility for glossectomy patients via voice conversion with sound and lip movement. Seiya Ogino, Hiroki Murakami, Sunao Hara, Masanobu Abe, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | In this paper, we propose the multimodal voice conversion based on Deep Neural Network using audio and lip movement information for improving speech intelligibility uttered by glossectomy patients. The glossectomy patients remove more than half of their tongue, sound uttered by them contain less intelligibility compared to healthy persons. In beseline using audio information, intelligibility isn't imporved enough. Hence, we improve more by the multimodal voice conversion. The lip movement information is face feature points obtained by Microsoft Kinect v2. From the result of evaluation, proposed approach cannot improve precision of voice conversion, but can improve intelligibility compared to baseline. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | multimodal voice conversion / Deep Neural Network / glossectomy patients / speech intelligibility / Microsoft Kinect v2 |
| Paper # | PRMU2018-23,SP2018-3 |
| Date of Issue | 2018-06-21 (PRMU, SP) |
| Conference Information | |
| Committee | PRMU / SP |
|---|---|
| Conference Date | 2018/6/28(2days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | Shinichi Sato(NII) / Yoichi Yamashita(Ritsumeikan Univ.) |
| Vice Chair | Yoshihisa Ijiri(Omron) / Toru Tamaki(Hiroshima Univ.) / Akinobu Ri(Nagoya Inst. of Tech.) |
| Secretary | Yoshihisa Ijiri(NEC) / Toru Tamaki(Osaka Univ.) / Akinobu Ri(Kyoto Univ.) |
| Assistant | Go Irie(NTT) / Yoshitaka Ushiku(Univ. of Tokyo) / Tomoki Koriyama(Tokyo Inst. of Tech.) / Satoshi Kobashikawa(NTT) |
| Paper Information | |
| Registration To | Technical Committee on Pattern Recognition and Media Understanding / Technical Committee on Speech |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Study of improving speech intelligibility for glossectomy patients via voice conversion with sound and lip movement. |
| Sub Title (in English) | |
| Keyword(1) | multimodal voice conversion |
| Keyword(2) | Deep Neural Network |
| Keyword(3) | glossectomy patients |
| Keyword(4) | speech intelligibility |
| Keyword(5) | Microsoft Kinect v2 |
| 1st Author's Name | Seiya Ogino |
| 1st Author's Affiliation | Okayama University(Okayama Univ.) |
| 2nd Author's Name | Hiroki Murakami |
| 2nd Author's Affiliation | Okayama University(Okayama Univ.) |
| 3rd Author's Name | Sunao Hara |
| 3rd Author's Affiliation | Okayama University(Okayama Univ.) |
| 4th Author's Name | Masanobu Abe |
| 4th Author's Affiliation | Okayama University(Okayama Univ.) |
| Date | 2018-06-28 |
| Paper # | PRMU2018-23,SP2018-3 |
| Volume (vol) | vol.118 |
| Number (no) | PRMU-111,SP-112 |
| Page | pp.pp.7-12(PRMU), pp.7-12(SP), |
| #Pages | 6 |
| Date of Issue | 2018-06-21 (PRMU, SP) |