MEXCA

Multimodal Emotion Expression Capture Amsterdam

The aim of MEXCA is to capture the emotional appeals of politicians. In election debates, tv interviews and parliamentary sessions politicians make appeals to our emotions: they want us to be angry about an issue, proud of our country, or fearful about the future. In making these appeals they use their face to make expressions, they choose particular words and also use the tone of their voice. Not all of this is deliberate and controllable. Yet what the face, the voice, and the text communicate impacts the listeners. The face, the voice and text are different modalities that communicate emotions. MEXCA is software that takes a video of an election debate, a tv interview or discussion, or a parliamentary session and it identifies the multimodal emotion expression of the people in the video. It automatically assigns video and audio segments to the same person, and uses external tools to analyze changes in facial expressions, tone in voice and sentiment in text.

MEXCA is currently open-source software downloadable from Github (https://github.com/mexca/mexca). It is currently deployed in two projects: (1) a large-scale analysis of 78 German election debates in the period 1997-2021; and (2) a series of focus-groups conversations around climate change. At this stage the data analysis is still ongoing.

The software is open for other people as well. Following three workshops the intended audience primarily consists of social scientists.

Participating organisations

Social Sciences & Humanities
Social Sciences & Humanities
Netherlands eScience Center
University of Amsterdam

Output

Team

GS
Gijs Schumacher
Principal investigator
University of Amsterdam
CP
Christian Pipal
Malte Lüken
Malte Lüken
Research Software Engineer
Netherlands eScience Center
Eva Viviani
Research Software Engineer
Netherlands eScience Center
Jisk Attema
Programme Manager
Netherlands eScience Center
Kody Moodley
Research Software Engineer
Netherlands eScience Center
Niels  Drost
Programme Manager
Netherlands eScience Center
Elena Ranguelova
Elena Ranguelova
Tech Lead
Netherlands eScience Center

Related projects

Vocalisations of Disgust

Vocalisations of Disgust across Behavioural Contexts: A Computational Analysis

Updated 11 months ago
Finished

REL 2.0

Multilingual and Multipurpose Entity Linking Toolkit

Updated 7 months ago
Finished

Different kinds of laughter

A machine learning approach

Updated 2 weeks ago
Finished

Emotion Recognition in Dementia

Advancing technology for multimodal analysis of emotion expression in everyday life

Updated 25 months ago
Finished

Related software

emvoice

EM

A Python package for computing emotion expression-related features from speech signals.

Updated 6 months ago
1

mexca

ME

Capture emotion expressions from video, audio, and text with a single pipeline.

Updated 10 months ago
4 5