Interactive Hesitation Synthesis and its Evaluation

Betz S, Carlmeyer B, Wagner P, Wrede B (Submitted) .

Download
OA 649.73 KB
Preprint | Submitted | English
Abstract
Conversational spoken dialogue systems that interact with the user rather than merely reading text can be equipped with hesitations to manage the dialogue flow and the users' attention. Based on a series of empirical studies, we built an elaborated hesitation synthesis strategy for dialogue systems that inserts hesitations of scalable extent wherever needed in the ongoing utterance. So far, evaluations of hesitating systems have shown that synthesis quality is affected negatively by hesitations, but that there is improvement in interaction quality. We argue that due to its conversational nature, hesitation synthesis needs interactive evaluation rather than traditional MOS-based questionnaires. To prove this point, we dually evaluate our system’s speech synthesis component: on the one hand, linked to the dialogue system evaluation, on the other hand, in the traditional MOS way. This way we are able to analyze and discuss differences that arise due to the evaluation methodology. Our results suggest that MOS scales are not sufficient to assess speech synthesis quality, which has implications for future research that are discussed in this paper. Furthermore, our results indicate that hesitations work well to increase task performance and that an elaborated strategy is necessary to avoid likability issues.
Publishing Year
PUB-ID

Cite this

Betz S, Carlmeyer B, Wagner P, Wrede B. Interactive Hesitation Synthesis and its Evaluation. Submitted.
Betz, S., Carlmeyer, B., Wagner, P., & Wrede, B. (Submitted). Interactive Hesitation Synthesis and its Evaluation
Betz, S., Carlmeyer, B., Wagner, P., and Wrede, B. (Submitted). Interactive Hesitation Synthesis and its Evaluation.
Betz, S., et al., Submitted. Interactive Hesitation Synthesis and its Evaluation.
S. Betz, et al., “Interactive Hesitation Synthesis and its Evaluation”, Submitted.
Betz, S., Carlmeyer, B., Wagner, P., Wrede, B.: Interactive Hesitation Synthesis and its Evaluation. (Submitted).
Betz, Simon, Carlmeyer, Birte, Wagner, Petra, and Wrede, Britta. “Interactive Hesitation Synthesis and its Evaluation”. (Submitted).
All files available under the following license(s):
Creative Commons License:
CC-BY
Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)
Main File(s)
Access Level
OA Open Access
Last Uploaded
2017-12-21T13:49:27Z

This data publication is cited in the following publications:
This publication cites the following data publications:

Export

0 Marked Publications

Open Data PUB

Search this title in

Google Scholar