Presentation | 2007/9/28 Comparison of non-task-oriented dialogue systems by HMM Naoki ISOMURA, Fujio TORIUMI, Kenichiro ISHII, |
---|---|
PDF Download Page | PDF download Page Link |
Abstract(in Japanese) | (See Japanese page) |
Abstract(in English) | Recently, computerized dialogue systems are researched and used in various fields. In order to realize a practical system, the performance should be evaluated quantitatively. The objective and quantitative evaluation method for dialogue systems for reservation services has already been established, non-task-oriented dialogue systems, however, have been evaluated only by subjective method, like a questionnaire. In this paper, we propose a new criterion which can evaluate non-task-oriented dialogue systems objectively and quantitatively. We assume a human-human dialogue is an ideal dialogue. We build a HMM by learning a sequence of human-human dialogue utterance tag which is automatically assigned. We apply 2-gram for auto-tagging and evaluate naturality of a dialogue using HMM. In this simulation, a correct rate applying auto-tagging is 71%. If we relax semantic conditions, a correct rate is 87%. It was clarified that a HMM method can evaluate the naturality of dialogue systems. |
Keyword(in Japanese) | (See Japanese page) |
Keyword(in English) | Dialogue System / HMM / Evaluation Method / DAMSL Tag / non-task-oriented |
Paper # | NLC2007-29 |
Date of Issue |
Conference Information | |
Committee | NLC |
---|---|
Conference Date | 2007/9/28(1days) |
Place (in Japanese) | (See Japanese page) |
Place (in English) | |
Topics (in Japanese) | (See Japanese page) |
Topics (in English) | |
Chair | |
Vice Chair | |
Secretary | |
Assistant |
Paper Information | |
Registration To | Natural Language Understanding and Models of Communication (NLC) |
---|---|
Language | JPN |
Title (in Japanese) | (See Japanese page) |
Sub Title (in Japanese) | (See Japanese page) |
Title (in English) | Comparison of non-task-oriented dialogue systems by HMM |
Sub Title (in English) | |
Keyword(1) | Dialogue System |
Keyword(2) | HMM |
Keyword(3) | Evaluation Method |
Keyword(4) | DAMSL Tag |
Keyword(5) | non-task-oriented |
1st Author's Name | Naoki ISOMURA |
1st Author's Affiliation | Graduate School of Information Science, Nagoya University() |
2nd Author's Name | Fujio TORIUMI |
2nd Author's Affiliation | Graduate School of Information Science, Nagoya University |
3rd Author's Name | Kenichiro ISHII |
3rd Author's Affiliation | Graduate School of Information Science, Nagoya University |
Date | 2007/9/28 |
Paper # | NLC2007-29 |
Volume (vol) | vol.107 |
Number (no) | 246 |
Page | pp.pp.- |
#Pages | 6 |
Date of Issue |