Presentation 2006/12/14
Method for Evaluating Task-Oriented Spoken Dialogs Based on Communication Efficiency
Toshiyuki TAKEZAWA, Masahide MIZUSHIMA, Tohru SHIMIZU, Genichiro KIKUI,
PDF Download Page PDF download Page Link
Abstract(in Japanese) (See Japanese page)
Abstract(in English) We propose a method for measuring communication efficiency from the viewpoint of conveying essential information in a task-oriented dialog. We show the results of a dialog experiment using speech-to-speech translation systems and one using the Wizard of Oz method, which was carried out using hidden interpreters instead of a speech-to-speech translation system. We also present a relative score for the performance of the speech-to-speech translation system which was obtained by measuring the performance of the machine against that of human, i.e., hidden interpreters. Finally, we discuss the relationship between users' linguistic behavior and system performance. We found that users of the system tended to make shorter utterances without decreasing the number of essential items needed to achieve a task and also to improve the transmission efficiency by controlling the strategy of dialogs.
Keyword(in Japanese) (See Japanese page)
Keyword(in English) Speech communication / natural language interfaces / speech recognition / interactive systems / human factors
Paper # NLC2006-35,SP2006-91
Date of Issue

Conference Information
Committee NLC
Conference Date 2006/12/14(1days)
Place (in Japanese) (See Japanese page)
Place (in English)
Topics (in Japanese) (See Japanese page)
Topics (in English)
Chair
Vice Chair
Secretary
Assistant

Paper Information
Registration To Natural Language Understanding and Models of Communication (NLC)
Language JPN
Title (in Japanese) (See Japanese page)
Sub Title (in Japanese) (See Japanese page)
Title (in English) Method for Evaluating Task-Oriented Spoken Dialogs Based on Communication Efficiency
Sub Title (in English)
Keyword(1) Speech communication
Keyword(2) natural language interfaces
Keyword(3) speech recognition
Keyword(4) interactive systems
Keyword(5) human factors
1st Author's Name Toshiyuki TAKEZAWA
1st Author's Affiliation Knowledge Creating Communication Research Center, National Institute of Information and Communication Technology:ATR Spoken Language Communication Research Labs.()
2nd Author's Name Masahide MIZUSHIMA
2nd Author's Affiliation NTT Cyberspace Laboratories
3rd Author's Name Tohru SHIMIZU
3rd Author's Affiliation Knowledge Creating Communication Research Center, National Institute of Information and Communication Technology:ATR Spoken Language Communication Research Labs.
4th Author's Name Genichiro KIKUI
4th Author's Affiliation NTT Cyberspace Laboratories
Date 2006/12/14
Paper # NLC2006-35,SP2006-91
Volume (vol) vol.106
Number (no) 441
Page pp.pp.-
#Pages 6
Date of Issue