Header

UZH-Logo

Maintenance Infos

Intrapersonal dependencies in multimodal behavior


Blomsma, Pieter A; Linders, Guido M; Vaitonyté, Julija; Louwerse, Max M (2020). Intrapersonal dependencies in multimodal behavior. In: 20th ACM International Conference on Intelligent Virtual Agents, Virtual/Scotland UK, 20 October 2020 - 22 October 2020. ACM Digital library, 1-8.

Abstract

Human interlocutors automatically adapt verbal and non-verbal signals so that different behaviors become synchronized over time. Multimodal communication comes naturally to humans, while this is not the case for Embodied Conversational Agents (ECAs). Knowing which behavioral channels synchronize within and across speakers and how they align seems critical in the development of ECAs. Yet, there exists little data-driven research that provides guidelines for the synchronization of different channels within an interlocutor. This study focuses on intrapersonal dependencies of multimodal behavior by using cross-recurrence analysis on a multimodal communication dataset to better understand the temporal relationships between language and gestural behavior channels. By shedding light on the intrapersonal synchronization of communicative channels in humans, we provide an initial manual for modality synchronisation in ECAs.

Abstract

Human interlocutors automatically adapt verbal and non-verbal signals so that different behaviors become synchronized over time. Multimodal communication comes naturally to humans, while this is not the case for Embodied Conversational Agents (ECAs). Knowing which behavioral channels synchronize within and across speakers and how they align seems critical in the development of ECAs. Yet, there exists little data-driven research that provides guidelines for the synchronization of different channels within an interlocutor. This study focuses on intrapersonal dependencies of multimodal behavior by using cross-recurrence analysis on a multimodal communication dataset to better understand the temporal relationships between language and gestural behavior channels. By shedding light on the intrapersonal synchronization of communicative channels in humans, we provide an initial manual for modality synchronisation in ECAs.

Statistics

Citations

Dimensions.ai Metrics
2 citations in Web of Science®
3 citations in Scopus®
Google Scholar™

Altmetrics

Downloads

13 downloads since deposited on 17 Aug 2023
13 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Conference or Workshop Item (Paper), not_refereed, original work
Communities & Collections:06 Faculty of Arts > Department of Comparative Language Science
Dewey Decimal Classification:000 Computer science, knowledge & systems
410 Linguistics
Scopus Subject Areas:Physical Sciences > Artificial Intelligence
Physical Sciences > Human-Computer Interaction
Language:English
Event End Date:22 October 2020
Deposited On:17 Aug 2023 09:08
Last Modified:30 Sep 2023 02:09
Publisher:ACM Digital library
Series Name:Proceedings of the ACM International Conference on Intelligent Virtual Agents
ISBN:9781450375863
OA Status:Green
Publisher DOI:https://doi.org/10.1145/3383652.3423872
  • Content: Published Version
  • Language: English