Presentation | 2005/7/15 The Relationship between Answer Ranking and User Satisfaction in a Question Answering System Tomoharu KOKUBU, Tetsuya SAKAI, Yoshimi SAITO, Hideki TSUTSUI, Toshihiko MANABE, Hiroko FUJII, Makoto KOYAMA, |
---|---|
PDF Download Page | PDF download Page Link |
Abstract(in Japanese) | (See Japanese page) |
Abstract(in English) | Although research in effective Question Answering (QA) has become active in recent years, it was not clear how system accuracy affects user satisfaction in practice. We therefore devised two practical scenarios in which QA may be useful (namely, desktop and mobile) and conducted a questionnaire survey for each scenario. The objective was to clarify the relationship between the rank of a correct answer and the proportion of satisfied users. Results show that, while the graph of the proportion of satisfied users resembles that of Reciprocal Rank for the desktop case, it is almost proportional to the rank for the mobile case. Thus, from the viewpoint of user satisfaction, one should set a goal in terms of the distribution of the correct answers instead of a single Mean Reciprocal Rank value. |
Keyword(in Japanese) | (See Japanese page) |
Keyword(in English) | Question Answering / questionnaire / user satisfaction |
Paper # | NLC2005-8 |
Date of Issue |
Conference Information | |
Committee | NLC |
---|---|
Conference Date | 2005/7/15(1days) |
Place (in Japanese) | (See Japanese page) |
Place (in English) | |
Topics (in Japanese) | (See Japanese page) |
Topics (in English) | |
Chair | |
Vice Chair | |
Secretary | |
Assistant |
Paper Information | |
Registration To | Natural Language Understanding and Models of Communication (NLC) |
---|---|
Language | JPN |
Title (in Japanese) | (See Japanese page) |
Sub Title (in Japanese) | (See Japanese page) |
Title (in English) | The Relationship between Answer Ranking and User Satisfaction in a Question Answering System |
Sub Title (in English) | |
Keyword(1) | Question Answering |
Keyword(2) | questionnaire |
Keyword(3) | user satisfaction |
1st Author's Name | Tomoharu KOKUBU |
1st Author's Affiliation | R&D Center, Toshiba Corporation() |
2nd Author's Name | Tetsuya SAKAI |
2nd Author's Affiliation | R&D Center, Toshiba Corporation |
3rd Author's Name | Yoshimi SAITO |
3rd Author's Affiliation | R&D Center, Toshiba Corporation |
4th Author's Name | Hideki TSUTSUI |
4th Author's Affiliation | R&D Center, Toshiba Corporation |
5th Author's Name | Toshihiko MANABE |
5th Author's Affiliation | R&D Center, Toshiba Corporation |
6th Author's Name | Hiroko FUJII |
6th Author's Affiliation | R&D Center, Toshiba Corporation |
7th Author's Name | Makoto KOYAMA |
7th Author's Affiliation | R&D Center, Toshiba Corporation |
Date | 2005/7/15 |
Paper # | NLC2005-8 |
Volume (vol) | vol.105 |
Number (no) | 203 |
Page | pp.pp.- |
#Pages | 8 |
Date of Issue |