| Presentation | 2006-10-19 Evaluation method of Non-task-oriented dialogue system by HMM Naoki ISOMURA, Fujio TORIUMI, Kenichiro ISHII, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | Recently, computerized dialogue systems are researched and used in various fields. In order to realize a practical system, the performance should be evaluated quantitatively. The objective and quantitative evaluation method for dialogue systems for reservation services has already been established ,non-task-oriented dialogue systems, however, have been evaluated only by subjective method, like a questionnaire. In this paper, we propose a new criterion which can evaluate non-task-oriented dialogue systems objectively and quantitatively. We assume a human-human dialogue is an ideal dialogue. We build a HMM by learning a sequence of human-human dialogue utterance tag which is automatically assigned. We apply n-gram for auto-tagging and evaluate naturality of a dialogue using HMM. In this simulation, a correct rate apply auto-tagging is 54%. If we relax semantic conditions, a correct rate is 82%. And it was clarified that a HMM method can evaluate the naturality of a dialogue. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | Dialogue System / HMM / Evaluation Method / DAMSL Tag / non-task-oriented |
| Paper # | TL2006-24,NLC2006-20,PRMU2006-101 |
| Date of Issue |
| Conference Information | |
| Committee | TL |
|---|---|
| Conference Date | 2006/10/12(1days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | |
| Vice Chair | |
| Secretary | |
| Assistant | |
| Paper Information | |
| Registration To | Thought and Language (TL) |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Evaluation method of Non-task-oriented dialogue system by HMM |
| Sub Title (in English) | |
| Keyword(1) | Dialogue System |
| Keyword(2) | HMM |
| Keyword(3) | Evaluation Method |
| Keyword(4) | DAMSL Tag |
| Keyword(5) | non-task-oriented |
| 1st Author's Name | Naoki ISOMURA |
| 1st Author's Affiliation | Graduate School of Information Science, Nagoya University() |
| 2nd Author's Name | Fujio TORIUMI |
| 2nd Author's Affiliation | Graduate School of Information Science, Nagoya University |
| 3rd Author's Name | Kenichiro ISHII |
| 3rd Author's Affiliation | Graduate School of Information Science, Nagoya University |
| Date | 2006-10-19 |
| Paper # | TL2006-24,NLC2006-20,PRMU2006-101 |
| Volume (vol) | vol.106 |
| Number (no) | 296 |
| Page | pp.pp.- |
| #Pages | 6 |
| Date of Issue | |