AI has been a big part of the cybersecurity industry for many years. Over the past three especially, it has begun to redefine the threat landscape and amplify the arms race between attackers and defenders.
However, it’s important to be clear about what it can’t yet do.
Unfortunately, two recent reports on AI-powered cyber-attacks may have given the impression that the technology is a more potent offensive weapon than it currently is. Security teams should approach them critically and focus their defensive efforts only on proven threats.
AI Cyber-Attacks Hype vs Reality
The first report was published earlier this year by MIT Sloan School of Management and Safe Security. Citing analysis of over 2800 incidents, it claimed that 80% of ransomware attacks are AI-enabled. It was subsequently dismissed by several prominent security researchers, temporarily forcing the report to be pulled.
In a second controversial study, AI developer Anthropic claimed to have discovered the first AI-orchestrated cyber-espionage campaign. It explained that state-sponsored threat actors had jailbroken its Claude Code tool to target around 30 global organizations, succeeding in a “small number” of cases.
Anthropic claimed that the AI worked autonomously to carry out most stages of the attack, from discovering and exploiting vulnerabilities in target organizations, to credential harvesting, lateral movement and data exfiltration.
Humans were involved in just 10-20% of the operational workload, it claimed.
Once again, experts questioned the claims, arguing that the absence of any indicators of compromise (IOCs) in the report means it can’t be trusted. Further doubts have arisen around the open source tooling allegedly used by the AI which many suggest would have been detected by most security controls.
The Truth Behind AI in Ransomware Development
Both reports, in their own way, may overstate what AI can actually do to help threat actors right now.
Yes, we have seen some examples of “AI-powered” ransomware. In August, researchers discovered PromptLock, which uses OpenAI’s locally hosted gpt-oss:20b model via the Ollama API to dynamically generate malicious Lua scripts.
These are reportedly then executed on infected systems for reconnaissance, data encryption and exfiltration. A few months earlier, it was assessed that the FunkSec group had probably used generative AI (GenAI) to help it develop ransomware.
However, these are both isolated examples which certainly don’t support the theory that 80% of ransomware is AI-enabled today.
It appears that the MIT Sloane report authors falsely assumed that any group known to have used AI in any capacity was also using it to create malware. The truth is that the top ransomware groups have their code developed in-house. They are already proficient at this, so if GenAI has a role to play, it is possibly in assessing and improving code rather than creating it from scratch. Or in helping with other stages of an attack, such as social engineering and/or reconnaissance.
There’s another, related reason why AI is unlikely to take the place of human threat actors any time soon. AI-powered ransomware comes straight out of the chatbot, so it isn’t tested in the field. This is an important part of the iterative development process that allows programmers to check what works best and what functionality and techniques could be used to enhance its efficacy. It could be argued that these “improved” features could be added to a chatbot's training set. But the latest versions of ransomware sources are not open, so it’s unlikely that they are available on those training sets.
As for Anthropic’s claims, even if we’re to take them at face value, there are two main hurdles for this attacker strategy to become mainstream.
First, the firm’s own write-up acknowledges that Claude “frequently overstated findings and occasionally fabricated data during autonomous operations, claiming to have obtained credentials that didn’t work or identifying critical discoveries that proved to be publicly available information.”
Such hallucinations mean less reliability and a greater need for human validation, which defeats the purpose of an agentic AI-orchestrated attack.
Second, such attacks are AI vendor dependent. Once developers like Anthropic or OpenAI realize their tech is being abused, they revoke API keys and the attack stops. As they become more aware that these attacks may happen, they will add extra guardrails to detect such abuses.
An attacker may opt to use an open source local model instead, but those can be less reliable and accurate than the latest commercial equivalent.
Preparing for AI-Driven Cyber Threats
However, we can’t afford to let our guard down. The UK’s own National Cyber Security Centre (NCSC) warns in its latest forward-looking threat assessment that AI will “almost certainly” make certain elements of intrusion activity “more effective and efficient.”
This, it adds, will lead to “an increase in frequency and intensity of cyber threats.” Although it cites “basic malware generation” as one of these elements, the biggest development will come from AI-assisted vulnerability research and exploit development, it warns.
Fortunately, AI innovation cuts both ways. While the bad guys are certainly harnessing the technology in attacks, so can security teams. The AI arms race has only just begun. To stay one step ahead, the security industry needs to be transparent about where risk actually is and not succumb to sowing fear, uncertainty and doubt.