PUB - Publikationen an der Universität Bielefeld

Communicative feedback is an expression of addressees’ listening-related mental states that parallels and influences their dialogue partners’ speech production (Clark 1996) by expressing ‘basic communication functions’ (e.g., perception, understanding, acceptance; Allwood et al. 1992). When occurring in the form of pragmatic interjections (e.g., ‘mm’, ‘huh?’) feedback occurs in a large number of forms. Applying phonologic, morphologic, or syntactic operations results in a combinatorially growing space of feedback expressions. These can be further varied using nonverbal markers (prosody, gesture; Freigang et al. 2017), which add continuous dimensions to the feedback form-space. Humans exploit this richness in form to enrich feedback meaning with attitudinal or epistemic components and to express subtle differences on various dimensions (e.g., certainty, degree of understanding, ongoing cognitive processing). Although the mapping between the form of feedback and its meaning has some aspects that are conventionalised, feedback meaning is idiosyncratic and relies heavily on iconic properties and – as a purely interactional phenomenon – on its dialogue context. Because of this, we see communicative feedback as a ‘model phenomenon’ of language processing that allows for modelling the cognitive processes underlying pragmatic reasoning in language use without the need to model all of language. We present a computational model of feedback interpretation (Buschmeier 2018), which embodies a probabilistic approach to pragmatic inference (Goodman and Frank 2016) and conceptualises speakers’ feedback interpretation as attribution of listening-related mental states to their feedback-providing interlocutors. Given an addressee’s feedback and its dialogue context, the model attributes a second order belief-state to the addressee (a probability distribution over their listening-related mental states, such as perception, understanding, acceptance, etc.). The model is thus able to (1) represent and reason about a speaker’s degree of belief in the dimensions and grades of their listener’s listening-related mental states (e.g., there is a high probability that the listener's understanding is estimated to be low). And (2) model the traditional semantic and pragmatic processes assumed to underly the hierarchical relationship of feedback functions (Allwood et al. 1992, Bunt 2011), namely ‘upward completion’ (Clark 1996) and ‘upper-bound implicata’ generated by the cooperative principle (Horn 2004). We combined this model of feedback interpretation with an incrementally adaptive natural language generation model in an artificial conversational agent and evaluated it in a semi-autonomous Wizard-of-Oz study (Buschmeier 2018). Autonomously interpreting its human interlocutors’ multimodal feedback and adapting to their needs, this ‘attentive speaker agent’ communicated more efficiently than an agent that explicitly ensured participants’ understanding. Participants rated the agent more helpful and cooperative and found it to be able to understand their mental state of listening. * Allwood et al. (1992). On the semantics and pragmatics of linguistic feedback. https://doi.org/10.1093/jos/9.1.1 * Bunt (2011). Multifunctionality in dialogue. https://doi.org/​10.1016/j.csl.2010.04.006 * Buschmeier (2018). _Attentive Speaking. From Listener Feedback to Interactive Adaptation_. PhD thesis, Bielefeld University. https://doi.org/​10.4119/unibi/2918295 * Clark (1996). _Using Language_. https://doi.org/​10.1017/CBO9780511620539 * Freigang et al. (2017). Pragmatic multimodality: Effects of nonverbal cues of focus and certainty in a virtual human. https://doi.org/​10.1007/978-3-319-67401-8_16 * Horn (2004). Implicature. https://doi.org/​10.1002/9780470756959.ch1 * Goodman & Frank (2016). Pragmatic language interpretation as probabilistic inference. https://doi.org/​10.1016/j.tics.2016.08.005