Presentation | 2008-12-09 User modeling for a satisfaction evaluation of a speech recognition system Sunao HARA, Norihide KITAOKA, Kazuya TAKEDA, |
---|---|
PDF Download Page | PDF download Page Link |
Abstract(in Japanese) | (See Japanese page) |
Abstract(in English) | A mathematical model for predicting the user satisfaction of a speech dialogue systems is studied based on a field trial of a voice-navigated music retrieval system. The Subjective Word Accuracy (subjective-WA), of the user is introduced as a background psychometrics for the satisfaction. In the field test, subjective-WA is collected through questionnaires together with satisfactory indexes and various user profiles. First we show that the subjective-WA is more significant to the user satisfactory than (Objective) Word Accuracy (objective-WA), which is calculated using the manually given transcriptions for the recorded dialogue. Then through top-down clustering of the joint distribution of subjective- and objective-WAs, we show that the user population can be grouped into several sub-groups in terms of sensitivity to recognition accuracy. The lower bound of the objective-WA for the given subjective-WA is also calculated from the joint distribution. Finally, a graphical model is build that predicts the user satisfactory index from user profiles and reduces the distribution uncertainty of user satisfaction by 13% of its variance. |
Keyword(in Japanese) | (See Japanese page) |
Keyword(in English) | System Evaluation / Performance Measure / User Profile / Field Test |
Paper # | NLC2008-34,SP2008-89 |
Date of Issue |
Conference Information | |
Committee | NLC |
---|---|
Conference Date | 2008/12/2(1days) |
Place (in Japanese) | (See Japanese page) |
Place (in English) | |
Topics (in Japanese) | (See Japanese page) |
Topics (in English) | |
Chair | |
Vice Chair | |
Secretary | |
Assistant |
Paper Information | |
Registration To | Natural Language Understanding and Models of Communication (NLC) |
---|---|
Language | JPN |
Title (in Japanese) | (See Japanese page) |
Sub Title (in Japanese) | (See Japanese page) |
Title (in English) | User modeling for a satisfaction evaluation of a speech recognition system |
Sub Title (in English) | |
Keyword(1) | System Evaluation |
Keyword(2) | Performance Measure |
Keyword(3) | User Profile |
Keyword(4) | Field Test |
1st Author's Name | Sunao HARA |
1st Author's Affiliation | Graduate School of Information Science, Nagoya University() |
2nd Author's Name | Norihide KITAOKA |
2nd Author's Affiliation | Graduate School of Information Science, Nagoya University |
3rd Author's Name | Kazuya TAKEDA |
3rd Author's Affiliation | Graduate School of Information Science, Nagoya University |
Date | 2008-12-09 |
Paper # | NLC2008-34,SP2008-89 |
Volume (vol) | vol.108 |
Number (no) | 337 |
Page | pp.pp.- |
#Pages | 6 |
Date of Issue |