Ai2 releases FairStress benchmark to evaluate demographic fairness in LLM decisions

Across open‑weight and frontier models—including Olmo, Llama, Qwen, Gemma, Claude and the GPT family—the authors observed little difference when prompts omitted demographic information. When identity was made explicit, models became less symmetric: the average gap in abandoning correct answers was 6.9 percentage points with implied identity and 9.3 points with explicit labels. Frontier models such as Claude Opus 5 and GPT‑5.5 chose minority‑coded candidates about 60–65 % of the time in ambiguous cases, and adversarial prefixes caused switches toward minority‑coded candidates 8.8–42.4 percentage points more often than toward majority‑coded ones. Internal activation patterns predicted the demographic side with roughly 73 % accuracy, while text‑based predictions reached 57–59 %.

The benchmark was compared with the BBQ bias test and showed that evaluation format (fixed‑option scoring vs. generated answers) can flip measured bias direction. Nine debiasing interventions were evaluated; most did not reliably reduce asymmetry, and one activation‑steering method dropped accuracy from 98 % to 47 %. Counterfactual chain‑of‑thought reduced the decision‑abandonment gap while largely preserving accuracy. An audit of Olmo 3’s training data found very few merit‑focused examples and unequal framing of demographic groups, but whether these patterns cause the observed gaps remains an open question.