Behavioral history outperforms descriptions for LLM synthetic personas
The study examined how well synthetic personas, created by large language models, can predict individual survey responses. Researchers used a two‑wave panel of 845 U.S. adults who answered 14 behavioral bias questions. Five conditions were tested: no personal data, demographics, personality traits, cognitive scores, and behavioral history.
Key points
- Study used 845 U.S. adults across 14 behavioral biases.
- Description‑based personas captured only 7‑12% of informedness at individual level.
Results showed that while the average number of biases per respondent was similar across conditions (7.1‑8.1 versus 7.1 for humans), description‑based personas captured only 53‑67% of the between‑person variation. Adding behavioral history restored this variation to roughly human levels. At the individual level, description‑based personas achieved only 7‑12% of the informedness seen in human test‑retest responses, whereas the behavioral‑history condition reached 28%. The behavioral‑history condition also performed best across all 17 demographic groups and revealed stronger education‑ and income‑related differences than human responses.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- GitHub releases ReviewBench to evaluate AI code reviews across 219 pull requests · 2 src
- Anthropic study: task understanding beats job title for AI success · 1 src
- Researchers test fixed token codes for language models at 100B-token scale · 1 src
- Baibaichuchu at the NTCIR-19 FinArg-3 Task: When Is Maximum Possible Profit Predictable from Investor Text? · 1 src
- Researchers introduce JEVal benchmark to test General decision models · 1 src
Comments
via GitHub Discussions