Header

UZH-Logo

Maintenance Infos

Multimodal Recognition of Valence, Arousal and Dominance via Late-Fusion of Text, Audio and Facial Expressions


Rios, Annette; Reichel, Uwe; Bhuvaneshwara, Chirag; Filntisis, Panagiotis; Maragos, Petros; Burkhardt, Felix; Eyben, Florian; Schuller, Björn; Nunnari, Fabrizio; Ebling, Sarah (2023). Multimodal Recognition of Valence, Arousal and Dominance via Late-Fusion of Text, Audio and Facial Expressions. In: European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Bruges, Belgium, 4 October 2023 - 6 October 2023. i6doc.com, 571-576.

Abstract

We present an approach for the prediction of valence, arousal, and dominance of people communicating via text/audio/video streams for a translation from and to sign languages. The approach consists of the fusion of the output of three CNN-based models dedicated to the analysis of text, audio, and facial expressions. Our experiments show that any combination of two or three modalities increases prediction performance for valence and arousal.

Abstract

We present an approach for the prediction of valence, arousal, and dominance of people communicating via text/audio/video streams for a translation from and to sign languages. The approach consists of the fusion of the output of three CNN-based models dedicated to the analysis of text, audio, and facial expressions. Our experiments show that any combination of two or three modalities increases prediction performance for valence and arousal.

Statistics

Citations

Dimensions.ai Metrics

Altmetrics

Downloads

93 downloads since deposited on 10 Oct 2023
100 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Conference or Workshop Item (Paper), refereed, original work
Communities & Collections:06 Faculty of Arts > Institute of Computational Linguistics
06 Faculty of Arts > Zurich Center for Linguistics
Dewey Decimal Classification:000 Computer science, knowledge & systems
410 Linguistics
Language:English
Event End Date:6 October 2023
Deposited On:10 Oct 2023 09:13
Last Modified:03 Jan 2024 15:00
Publisher:i6doc.com
ISBN:978-2-87587-088-9
OA Status:Green
Free access at:Publisher DOI. An embargo period may apply.
Publisher DOI:https://doi.org/10.14428/esann/2023.ES2023-128
  • Language: English