# Synthetic personas underperform plain LLMs in predicting click-through, study finds

Digest AI · Research · published 2026-09-23T04:00:00Z

Canonical: https://digestai.news/story/synthetic-personas-underperform-plain-llms-in-predicting-click-through

## Summary

A new arXiv paper evaluates whether large language models (LLMs) used as synthetic personas can forecast real audience reactions to copy. The authors compare a ten‑persona panel, built from demographic data, against a zero‑shot baseline that simply asks the model how likely a typical reader is to click. Using the Upworthy Research Archive – thousands of headline A/B tests with measured click‑through rates – they restrict analysis to the reliable subset of 399 tests where a clear winner exists.

On this subset, the no‑persona baseline achieves a Kendall τ of 0.361 and top‑1 accuracy of 49.2%, while the persona‑conditioned panel records τ of 0.084 and top‑1 accuracy of 34.6%, with non‑overlapping confidence intervals. The advantage of the plain LLM holds across three Gemini tiers, OpenAI’s gpt‑4.1, and a separate news‑domain dataset, and is robust to seed, prompt phrasing, and model choice. The authors conclude that for aggregate engagement prediction, a simple LLM ranker outperforms persona simulation.

## Key points

- No‑persona baseline reached Kendall τ 0.361 and 49.2% top‑1 accuracy on 399 reliable A/B tests.
- Persona‑conditioned panel scored τ 0.084 and 34.6% top‑1 accuracy, worse than baseline.
- Findings replicated across three Gemini tiers, OpenAI gpt‑4.1, and a separate news dataset.

## Why it matters

Marketers using LLMs to simulate audience personas may achieve more reliable engagement forecasts by prompting a plain model directly, simplifying workflows and reducing bias.

## Sources

1. [Do Synthetic Personas Predict Real Audience Response? A Sim-to-Real Study Where a No-Persona Baseline Beats Persona-Based Copy Simulation](https://arxiv.org/abs/2609.25010) (arXiv cs.AI, 2026-09-23, primary source)

## Cite

Digest AI, "Synthetic personas underperform plain LLMs in predicting click-through, study finds", 23 September 2026, https://digestai.news/story/synthetic-personas-underperform-plain-llms-in-predicting-click-through

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/synthetic-personas-underperform-plain-llms-in-predicting-click-through.json