# Researchers achieve 77.95% F1 on ambivalence detection using audio‑text cross‑attention

Digest AI · Research · published 2026-10-21T04:00:00Z

Canonical: https://digestai.news/story/researchers-achieve-77-95-f1-on-ambivalence-detection-using-audiotext

## Summary

Ambivalence and hesitancy are subtle behavioural cues that appear both in spoken words and how they are delivered.  A new framework encodes full‑video transcripts with a pre‑trained multilingual BERT model and the accompanying speech waveforms with wav2vec 2.0.  Two stacked bidirectional cross‑attention blocks let linguistic tokens attend to acoustic frames and vice‑versa, removing the need for explicit temporal alignment.  The resulting representations are pooled with a masked mean operation and merged through concatenation, element‑wise products, and absolute differences before a binary classifier decides the presence of ambivalence or hesitancy.

The system was tested on the subject‑independent split of the Behavioural Ambivalence/Hesitancy (BAH) dataset.  Reported results include 72.19% accuracy, 75.00% precision, 81.13% recall, a 77.95% A/H F1‑score, and a 70.16% macro‑F1 on the test set.  These numbers suggest that jointly modelling linguistic and acoustic information can substantially improve automatic recognition of these nuanced states.

## Key points

- Audio and text are encoded with multilingual BERT and wav2vec 2.0, then fused via bidirectional cross‑attention.
- Evaluated on the Behavioural Ambivalence/Hesitancy dataset, the model reaches 77.95% A/H F1‑score.
- Sequences are aggregated with masked mean pooling and combined using concatenation, element‑wise products, and absolute differences.

## Why it matters

Joint audio‑text modeling boosts detection of subtle behavioural signals, aiding applications in mental‑health assessment, human‑computer interaction, and affective computing.

## Sources

1. [Multimodal Ambivalence and Hesitancy Recognition with Bidirectional Audio--Text Cross-Attention](https://arxiv.org/abs/2610.26868) (arXiv cs.CL, 2026-10-21, primary source)

## Cite

Digest AI, "Researchers achieve 77.95% F1 on ambivalence detection using audio‑text cross‑attention", 21 October 2026, https://digestai.news/story/researchers-achieve-77-95-f1-on-ambivalence-detection-using-audiotext

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/researchers-achieve-77-95-f1-on-ambivalence-detection-using-audiotext.json