Vals finds Chinese open AI model nearly matches OpenAI’s hacking model in tests
Vals, a company that benchmarks AI capabilities, tested how well different models could execute complex, multi-step network attacks on simulated corporate networks. The results suggest open-weight models—like DeepSeek—are closing the gap with proprietary ones, such as OpenAI’s Daybreak, which remains the most capable but costs significantly more to run.
Key points
- Vals tested two simulated scenarios: a public-facing network and a multi-step compromise task
- OpenAI’s Daybreak model scored 81% task completion, DeepSeek 73%, with open models cheaper to run
- Experts warn closed labs may have unreleased models exceeding open-weight capabilities, prolonging the cat-and-mouse game
In two scenarios, Daybreak completed 81% of tasks on average, while DeepSeek achieved 73%. Vals notes open models like DeepSeek cost $2–$3 in tokens per attack versus $17–$46 for Daybreak. The findings challenge assumptions that gated access to frontier models could delay the spread of advanced hacking tools. Experts debate whether AI fundamentally alters cybersecurity or accelerates existing threats, though attackers still favor older models for cost efficiency. Some warn unreleased closed models may surpass open-weight capabilities, creating an ongoing arms race.
The story so far
6 episodes →- Vals finds Chinese open AI model nearly matches OpenAI’s hacking model in teststhis story
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Policy & Regulation
All →- Microsoft warns hackers use agentic AI to find GitHub secrets and breach Azure · 4 src
- Democratic wins in Texas, Georgia, Ohio and Pennsylvania threaten 50% of proposed US AI data center projects · 17 src
- Anthropic’s AI development pace doubles, METR analysis finds · 11 src
- OpenAI pledges to avoid NDAs and cover electricity costs for data centers after election backlash · 9 src
- Meta delays commercial terms update that would require AI training on external sites · 2 src
Comments
via GitHub Discussions