{"version":1,"type":"story","url":"https://digestai.news/story/synthetic-personas-underperform-plain-llms-in-predicting-click-through","json":"https://digestai.news/story/synthetic-personas-underperform-plain-llms-in-predicting-click-through.json","markdown":"https://digestai.news/story/synthetic-personas-underperform-plain-llms-in-predicting-click-through.md","slug":"synthetic-personas-underperform-plain-llms-in-predicting-click-through","headline":"Synthetic personas underperform plain LLMs in predicting click-through, study finds","summary":"A new arXiv paper evaluates whether large language models (LLMs) used as synthetic personas can forecast real audience reactions to copy. The authors compare a ten‑persona panel, built from demographic data, against a zero‑shot baseline that simply asks the model how likely a typical reader is to click. Using the Upworthy Research Archive – thousands of headline A/B tests with measured click‑through rates – they restrict analysis to the reliable subset of 399 tests where a clear winner exists.\n\nOn this subset, the no‑persona baseline achieves a Kendall τ of 0.361 and top‑1 accuracy of 49.2%, while the persona‑conditioned panel records τ of 0.084 and top‑1 accuracy of 34.6%, with non‑overlapping confidence intervals. The advantage of the plain LLM holds across three Gemini tiers, OpenAI’s gpt‑4.1, and a separate news‑domain dataset, and is robust to seed, prompt phrasing, and model choice. The authors conclude that for aggregate engagement prediction, a simple LLM ranker outperforms persona simulation.","keyPoints":["No‑persona baseline reached Kendall τ 0.361 and 49.2% top‑1 accuracy on 399 reliable A/B tests.","Persona‑conditioned panel scored τ 0.084 and 34.6% top‑1 accuracy, worse than baseline.","Findings replicated across three Gemini tiers, OpenAI gpt‑4.1, and a separate news dataset."],"whyItMatters":"Marketers using LLMs to simulate audience personas may achieve more reliable engagement forecasts by prompting a plain model directly, simplifying workflows and reducing bias.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":["Google","OpenAI","Upworthy"],"models":["Gemini","gpt-4.1"],"people":[]},"firstPublishedAt":"2026-09-23T04:00:00Z","updatedAt":"2026-09-23T04:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"arXiv cs.AI","title":"Do Synthetic Personas Predict Real Audience Response? A Sim-to-Real Study Where a No-Persona Baseline Beats Persona-Based Copy Simulation","url":"https://arxiv.org/abs/2609.25010","publishedAt":"2026-09-23T04:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"Synthetic personas underperform plain LLMs in predicting click-through, study finds\", 23 September 2026, https://digestai.news/story/synthetic-personas-underperform-plain-llms-in-predicting-click-through","publisher":"Digest AI","title":"Synthetic personas underperform plain LLMs in predicting click-through, study finds","datePublished":"2026-09-23T04:00:00Z","url":"https://digestai.news/story/synthetic-personas-underperform-plain-llms-in-predicting-click-through"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}