# PsyAgentBench tests if LLMs truly mimic human psychological biases

Digest AI · Research · published 2026-09-22T04:00:00Z

Canonical: https://digestai.news/story/psyagentbench-tests-if-llms-truly-mimic-human-psychological-biases

## Summary

Researchers introduced PsyAgentBench, a new benchmark designed to distinguish between LLMs mimicking human psychological patterns and actually possessing those biases. The study re-runs five classic psychology experiments using a factorial design that varies prompt labeling (named vs. blind), task structure (canonical vs. counterfactual), and persona instructions. The team released 41,904 trials from up to three open-weight model families to analyze these interactions.

Results show that human-like effects in LLMs arise through different mechanisms rather than a single susceptibility. For example, Asch conformity effects jumped from 0 percent in blind trials to 83.3 percent in named trials for the gpt-oss-120B model. Anchoring effects were absent when grounded in facts but strong with invented quantities, suggesting rational reliance on available signals. Sunk cost effects were largely absent, while minimal-group allocation was dominated by safety-mediated refusals.

The authors argue that simple scalar bias scores obscure these complex structures. They identify three ways psychology paradigms fail to port to LLMs: persona dominance, population collapse, and safety selection. A one-sentence persona change could eliminate, dampen, or reverse these effects, challenging the idea of a unified response bias in large language models.

## Key points

- PsyAgentBench releases 41,904 trials to test LLMs on five classic psychology experiments.
- Asch conformity rates shifted from 0 percent blind to 83.3 percent named on gpt-oss-120B.
- Anchoring effects appeared only with invented quantities, not grounded facts, in the study.

## Why it matters

This research clarifies that LLMs do not simply inherit human biases. Understanding the specific mechanisms, like labeling effects or safety refusals, is crucial for building reliable AI agents and interpreting their behavior accurately.

## Sources

1. [Recognition, Simulation, and Refusal: A Contamination-Aware Study of Classic Psychological Effects in LLM Agents](https://arxiv.org/abs/2609.22090) (arXiv cs.CL, 2026-09-22, primary source)

## Cite

Digest AI, "PsyAgentBench tests if LLMs truly mimic human psychological biases", 22 September 2026, https://digestai.news/story/psyagentbench-tests-if-llms-truly-mimic-human-psychological-biases

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/psyagentbench-tests-if-llms-truly-mimic-human-psychological-biases.json