A new benchmark from Vals AI suggests that, for offensive cyber work, freely downloadable models are now close behind the restricted frontier ones. That should change how security teams plan.
The test
Vals built its CyberOffense benchmark around simulated corporate networks. Each AI agent works alone from an isolated attack machine. It starts with a public web address and gets no human help.
The results
- GPT-6 Sol, part of OpenAI's restricted Daybreak cyber programme, completed 80.8% of objectives, the best score.
- DeepSeek V4.1 Flash, an open-weight model anyone can download, reached 72.8%. It cost about $3 a run against roughly $22 for GPT-6 Sol.
In a harder scenario called Lighthouse, the Daybreak model took admin control of a first company, stole passwords and used them to get into a second one. It hit 27 of 31 objectives in under three hours. DeepSeek reached 21 objectives at about a tenth of the cost.
The token numbers are worth noting. DeepSeek scored nothing in its first 25 million tokens and made most of its progress only after 100 million. So short evaluations understate what open models can do when they are allowed to keep going.
Why "open-weight" changes the risk
Vals' point is that once weights are public, no provider can restrict how a model is used. The Decoder notes the results line up with the UK AI Security Institute's July analysis, which put the open-versus-closed gap in cyber capability at four to seven months, down from six to ten months for most of 2025. AISI also estimated a cyber-range run at $1.19 with DeepSeek V4-Pro versus $85 with Anthropic's Opus 4.5, and found safety measures on open models largely ineffective.
The defender's side
Vals frames this as an opportunity too: the same models that find and exploit flaws can help companies find and fix them first. It also expects attacks on real networks to increase in the coming months.
My take
If I were running a security team, I would assume a capable, cheap, unrestricted attacker agent is already available, and set patch windows on that basis. I would also be running the same class of models against my own estate, under my own controls, as routine work rather than a research project.
Source: The gap between open and closed AI models in cyber capabilities keeps shrinking, The Decoder
