{"version":1,"type":"story","url":"https://digestai.news/story/arxiv-study-finds-open-weight-models-match-closed-models-in-cti-realm","json":"https://digestai.news/story/arxiv-study-finds-open-weight-models-match-closed-models-in-cti-realm.json","markdown":"https://digestai.news/story/arxiv-study-finds-open-weight-models-match-closed-models-in-cti-realm.md","slug":"arxiv-study-finds-open-weight-models-match-closed-models-in-cti-realm","headline":"arXiv study finds open-weight models match closed models in CTI-REALM tasks","summary":"The authors tested nineteen open-weight and seven closed models under the same conditions, scoring their ability to convert reports into working detections by querying telemetry data.\n\nThe paper found that open-weight models can match the best closed models when given more interaction turns. Removing this limit revealed that open-weight models outperformed closed ones in later stages. The study also introduced stage-level metrics to evaluate each model’s specific strengths, suggesting workflows could route tasks to the best-performing model per step rather than relying on a single model’s overall score. Token efficiency varied widely, with no clear link between cost and performance.","keyPoints":["Nineteen open-weight and seven closed models tested on **CTI-REALM** benchmark for threat-intelligence agents","Open-weight models matched closed models’ performance when given more interaction turns, per paper","Stage-level metrics show different models excel at distinct workflow stages, per authors"],"whyItMatters":"Proves open-weight models can compete with closed frontier models in security workflows, cutting costs without sacrificing performance.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":[],"models":["CTI-REALM"],"people":[]},"firstPublishedAt":"2026-10-14T04:00:00Z","updatedAt":"2026-10-14T04:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"arXiv cs.AI","title":"Unconstrained CTI-REALM with Open-Weights","url":"https://arxiv.org/abs/2610.14180","publishedAt":"2026-10-14T04:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"arXiv study finds open-weight models match closed models in CTI-REALM tasks\", 14 October 2026, https://digestai.news/story/arxiv-study-finds-open-weight-models-match-closed-models-in-cti-realm","publisher":"Digest AI","title":"arXiv study finds open-weight models match closed models in CTI-REALM tasks","datePublished":"2026-10-14T04:00:00Z","url":"https://digestai.news/story/arxiv-study-finds-open-weight-models-match-closed-models-in-cti-realm"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}