| Presentation | 1998/6/1 LIP MOVEMENT SYNTHESIS FROM SPEECH AND TEXT Masatsune Tamura, Takashi Masuko, Takao Kobayashi, |
|---|---|
| PDF Download Page | PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | This paper presents a technique for synthesizing lip movements that synchronize with given utterances based on HMM. In the training stage of the technique, speech unit HMMs are trained with audio and visual parameter vector sequences that represent speech and mouth shapes. Then speech unit HMMs are splitted into speech and visual parameter parts. In the recognition stage, input speech is converted into a transcription and a state sequence using the speech part of the HMMs. In the synthesis stage, a sentence HMM is constructed by concatenating visual parameter part of the HMMs corresponding to the transcription for the given speech. Then an optimum parameter vector sequence in an ML sense is obtained from the sentence HMM. The generated parameter sequence reflects statistical information of both static and dynamic features, and synthetic lip animation becomes quite smooth and natural. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | lip movement systhesis / hidden Markov model / lip synchronization / multimodal interface |
| Paper # | MVE98-38 |
| Date of Issue |
| Conference Information | |
| Committee | MVE |
|---|---|
| Conference Date | 1998/6/1(1days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | |
| Vice Chair | |
| Secretary | |
| Assistant | |
| Paper Information | |
| Registration To | Media Experience and Virtual Environment (MVE) |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | LIP MOVEMENT SYNTHESIS FROM SPEECH AND TEXT |
| Sub Title (in English) | |
| Keyword(1) | lip movement systhesis |
| Keyword(2) | hidden Markov model |
| Keyword(3) | lip synchronization |
| Keyword(4) | multimodal interface |
| 1st Author's Name | Masatsune Tamura |
| 1st Author's Affiliation | Interdisciplinary Graduate School of Science and Engineering, Tokyo Institute of Technology() |
| 2nd Author's Name | Takashi Masuko |
| 2nd Author's Affiliation | Interdisciplinary Graduate School of Science and Engineering, Tokyo Institute of Technology |
| 3rd Author's Name | Takao Kobayashi |
| 3rd Author's Affiliation | Interdisciplinary Graduate School of Science and Engineering, Tokyo Institute of Technology |
| Date | 1998/6/1 |
| Paper # | MVE98-38 |
| Volume (vol) | vol.98 |
| Number (no) | 97 |
| Page | pp.pp.- |
| #Pages | 6 |
| Date of Issue | |