{"version":1,"type":"story","url":"https://digestai.news/story/modality-discrepancy-transformer-reaches-0-7408-macro-f1-on-bah-datase","json":"https://digestai.news/story/modality-discrepancy-transformer-reaches-0-7408-macro-f1-on-bah-datase.json","markdown":"https://digestai.news/story/modality-discrepancy-transformer-reaches-0-7408-macro-f1-on-bah-datase.md","slug":"modality-discrepancy-transformer-reaches-0-7408-macro-f1-on-bah-datase","headline":"Modality Discrepancy Transformer reaches 0.7408 Macro F1 on BAH dataset","summary":"A new paper on arXiv proposes the Modality Discrepancy Transformer (MDT) to detect ambivalence and hesitancy in clinical videos, where facial, vocal, and linguistic cues conflict. MDT expands a prior 6‑token design to nine tokens: three modality embeddings, three absolute‑difference features, and three Hadamard‑product discrepancy features, processed by Transformer self‑attention. The architecture adds FiLM‑based text‑conditioned modulation, LoRA fine‑tuning, and a text‑guided late‑fusion branch that merges a text‑only head with the multimodal output at inference.\n\nOn the BAH dataset from the 3rd ABAW Challenge, MDT attains a Macro F1 of 0.7408 on the labelled test split and 0.7368 on the private leaderboard, surpassing the strongest published baseline by more than ten points. Training completes in under 20 minutes on a single GPU, demonstrating both high performance and efficiency for cross‑modal affective state recognition.","keyPoints":["MDT uses a 9-token multimodal representation: three modality embeddings, three absolute-difference, three Hadamard-product features.","Achieves 0.7408 Macro F1 on BAH test split, 0.7368 on private leaderboard, over 10 points above prior best.","Trains in under 20 minutes on a single GPU using FiLM‑based text modulation and LoRA fine‑tuning."],"whyItMatters":"Accurate detection of ambivalence and hesitancy in clinical videos can improve mental‑health assessments; the fast‑training, high‑performing MDT makes such analysis more feasible for researchers and practitioners.","category":{"slug":"models","name":"Generative AI & Models","url":"https://digestai.news/category/models"},"entities":{"companies":[],"models":["Modality Discrepancy Transformer"],"people":["Bekhouche et al."]},"firstPublishedAt":"2026-09-18T04:00:00Z","updatedAt":"2026-09-18T04:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"arXiv cs.CL","title":"Modality Discrepancy Transformer for Ambivalence and Hesitancy Recognition","url":"https://arxiv.org/abs/2609.19148","publishedAt":"2026-09-18T04:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"Modality Discrepancy Transformer reaches 0.7408 Macro F1 on BAH dataset\", 18 September 2026, https://digestai.news/story/modality-discrepancy-transformer-reaches-0-7408-macro-f1-on-bah-datase","publisher":"Digest AI","title":"Modality Discrepancy Transformer reaches 0.7408 Macro F1 on BAH dataset","datePublished":"2026-09-18T04:00:00Z","url":"https://digestai.news/story/modality-discrepancy-transformer-reaches-0-7408-macro-f1-on-bah-datase"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}