# arXiv study finds open-weight models match closed models in CTI-REALM tasks

Digest AI · Research · published 2026-10-14T04:00:00Z

Canonical: https://digestai.news/story/arxiv-study-finds-open-weight-models-match-closed-models-in-cti-realm

## Summary

The authors tested nineteen open-weight and seven closed models under the same conditions, scoring their ability to convert reports into working detections by querying telemetry data.

The paper found that open-weight models can match the best closed models when given more interaction turns. Removing this limit revealed that open-weight models outperformed closed ones in later stages. The study also introduced stage-level metrics to evaluate each model’s specific strengths, suggesting workflows could route tasks to the best-performing model per step rather than relying on a single model’s overall score. Token efficiency varied widely, with no clear link between cost and performance.

## Key points

- Nineteen open-weight and seven closed models tested on **CTI-REALM** benchmark for threat-intelligence agents
- Open-weight models matched closed models’ performance when given more interaction turns, per paper
- Stage-level metrics show different models excel at distinct workflow stages, per authors

## Why it matters

Proves open-weight models can compete with closed frontier models in security workflows, cutting costs without sacrificing performance.

## Sources

1. [Unconstrained CTI-REALM with Open-Weights](https://arxiv.org/abs/2610.14180) (arXiv cs.AI, 2026-10-14, primary source)

## Cite

Digest AI, "arXiv study finds open-weight models match closed models in CTI-REALM tasks", 14 October 2026, https://digestai.news/story/arxiv-study-finds-open-weight-models-match-closed-models-in-cti-realm

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/arxiv-study-finds-open-weight-models-match-closed-models-in-cti-realm.json