Researchers decode 26 legal factors in three open LLMs
A new study on arXiv examines how large language models (LLMs) organize legal knowledge internally. The paper, titled arXiv:2610.28799v1, explores whether LLMs retain structured representations of the 26 CATO factors—a framework used by legal experts to categorize legally significant elements in case law. The authors tested three open models: Qwen3-1.7B, Qwen3-8B, and Gemma-2 2B, using methods…
Key points
- Researchers decoded all 26 CATO legal factors in three open LLMs: Qwen3-1.7B, Qwen3-8B, and Gemma-2 2B
- Linear decodability and sparse autoencoders achieved AUC scores between 0.86 and 1.00 for factor detection
- Study mapped 78 layer-feature coordinates, linking each factor to specific model activations, but did not test real-world legal utility
The results show that all 26 CATO factors are detectable in each model’s internal activations, with test AUC scores ranging from 0.86 to 1.00. The study also identifies 78 model-specific layer-feature coordinates, mapping each factor to precise locations in the models. While the findings suggest LLMs encode legal abstractions similarly to human expertise, the paper does not assess whether these representations improve legal reasoning or are usable in practice. The work builds on prior research linking HYPO dimensions (another symbolic legal framework) to LLMs but focuses on open-source models rather than proprietary ones.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- PHP patches critical RCE flaws discovered with GPT-5.6 Sol, affecting up to 70% of websites · 9 src
- OpenAI and Google AI models discovered critical PHP flaw affecting millions of websites · 8 src
- Researchers release madartb, a parallel Multi-Dialect Arabic dependency treebank · 1 src
- Researchers test how reinforcement learning teaches AI agents to enforce market collusion · 1 src
- LLMs can assess psychotherapy fidelity, study shows · 1 src
Comments
via GitHub Discussions