# Study finds capable LLM agents leak more privacy in dialogue

Digest AI · Research · published 2026-10-14T04:00:00Z

Canonical: https://digestai.news/story/study-finds-capable-llm-agents-leak-more-privacy-in-dialogue

## Summary

A new arXiv paper introduces CIMEMORIES DIALOGUE, a benchmark of 4,410 two-agent dialogue instances designed to test privacy leakage in LLM agents. The study is grounded in Contextual Integrity theory, which assesses whether information flows respect the norms under which they originated. This is critical as agents are increasingly deployed in high-stakes enterprise settings where handling sensitive data is paramount.

The researchers systematically evaluated seven open-weight models from the Qwen3 and Llama 3.1 families. They tested these models across conversation lengths of 1 and 6 turns under three different prompting conditions. The core finding is that task capability and privacy leakage are fundamentally coupled. More capable models were systematically leakier, and no model in the evaluation was able to decouple high performance from privacy risks.

The study also highlights that thinking models, which use chain-of-thought reasoning, consistently leak more information than their non-thinking counterparts. This is attributed to the extended reasoning surface created by the chain-of-thought process. Furthermore, leakage compounds with conversation length, rising up to 3x from 1 to 6 turns. The authors suggest this coupling is a structural feature of current agentic systems rather than an artifact of misconfiguration.

## Key points

- New benchmark CIMEMORIES DIALOGUE contains 4,410 two-agent dialogue instances for privacy testing.
- Study of Qwen3 and Llama 3.1 models shows more capable agents leak more privacy.
- Privacy leakage rises up to 3x when conversation length increases from 1 to 6 turns.

## Why it matters

This research reveals a structural trade-off in current AI agents: higher capability correlates with higher privacy risk. This is critical for enterprises deploying agents in sensitive environments, as it suggests privacy cannot be easily separated from performance.

## Sources

1. [Loose Lips: Understanding Privacy Leakage in Agentic Dialogue](https://arxiv.org/abs/2610.14071) (arXiv cs.CL, 2026-10-14, primary source)

## Cite

Digest AI, "Study finds capable LLM agents leak more privacy in dialogue", 14 October 2026, https://digestai.news/story/study-finds-capable-llm-agents-leak-more-privacy-in-dialogue

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/study-finds-capable-llm-agents-leak-more-privacy-in-dialogue.json