| Presentation | 2008-12-09 User modeling for a satisfaction evaluation of a speech recognition system Sunao HARA, Norihide KITAOKA, Kazuya TAKEDA, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | A mathematical model for predicting the user satisfaction of a speech dialogue systems is studied based on a field trial of a voice-navigated music retrieval system. The Subjective Word Accuracy (subjective-WA), of the user is introduced as a background psychometrics for the satisfaction. In the field test, subjective-WA is collected through questionnaires together with satisfactory indexes and various user profiles. First we show that the subjective-WA is more significant to the user satisfactory than (Objective) Word Accuracy (objective-WA), which is calculated using the manually given transcriptions for the recorded dialogue. Then through top-down clustering of the joint distribution of subjective- and objective-WAs, we show that the user population can be grouped into several sub-groups in terms of sensitivity to recognition accuracy. The lower bound of the objective-WA for the given subjective-WA is also calculated from the joint distribution. Finally, a graphical model is build that predicts the user satisfactory index from user profiles and reduces the distribution uncertainty of user satisfaction by 13% of its variance. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | System Evaluation / Performance Measure / User Profile / Field Test |
| Paper # | NLC2008-34,SP2008-89 |
| Date of Issue |
| Conference Information | |
| Committee | NLC |
|---|---|
| Conference Date | 2008/12/2(1days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | |
| Vice Chair | |
| Secretary | |
| Assistant | |
| Paper Information | |
| Registration To | Natural Language Understanding and Models of Communication (NLC) |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | User modeling for a satisfaction evaluation of a speech recognition system |
| Sub Title (in English) | |
| Keyword(1) | System Evaluation |
| Keyword(2) | Performance Measure |
| Keyword(3) | User Profile |
| Keyword(4) | Field Test |
| 1st Author's Name | Sunao HARA |
| 1st Author's Affiliation | Graduate School of Information Science, Nagoya University() |
| 2nd Author's Name | Norihide KITAOKA |
| 2nd Author's Affiliation | Graduate School of Information Science, Nagoya University |
| 3rd Author's Name | Kazuya TAKEDA |
| 3rd Author's Affiliation | Graduate School of Information Science, Nagoya University |
| Date | 2008-12-09 |
| Paper # | NLC2008-34,SP2008-89 |
| Volume (vol) | vol.108 |
| Number (no) | 337 |
| Page | pp.pp.- |
| #Pages | 6 |
| Date of Issue | |