The Chaos Phase: How AI is Transforming Cybersecurity Threats

We’re entering the most volatile period in modern cyber defense. AI is compounding attacker capability faster than most security programs can adapt. Over the next 24 months we’ll enter the chaos phase: a period where autonomous and semi‑autonomous adversaries move faster than most defenders, driving an unprecedented spike in successful intrusions and breach impact.

I think that the cybersecurity capability of these models, are quite significant. We continue to flash warning lights on this, I think the world is not taking us seriously.
– Sam Altman, CEO, OpenAI

When Offense Moves First

The AI-Fueled Offensive

In the last 12 months, adversarial use of AI has become standard operating procedure.

This shift is driven by a fundamental change in the economics of cybercrime. Historically, attackers were constrained by the same bottleneck as defenders: a scarcity of elite talent. This created a natural economic barrier, as sophisticated attacks required a high return on investment to be worthwhile.

AI is dismantling that barrier. The limiting factor becomes no longer human expertise but access to compute - and the cost of that compute is plummeting. The cost of AI inference is falling exponentially, making advanced capabilities accessible to a much broader range of actors (see a16z and Epoch AI). Reports from Google and OpenAI confirm that threat actors are already using advanced AI models in every phase of the attack lifecycle, from reconnaissance and vulnerability discovery to malware development and malicious scripting.

Some tangible examples from Google’s report, The AI Attacks Repository, and OpenAI reports include exploitation of public CVEs, development of Android malware, reverse engineering of EDR server for evasion, and endless phishing attacks.

This proliferation of offensive AI is not a US-centric phenomenon. Tech firms and research groups globally are developing powerful models with significant cyber capabilities, from China’s Kimi and Qwen to Russia’s YandexGPT, Sberbank's GigaChat, and T-Bank's models (T-Lite and T-Pro). This trend is compounded by the explosion of powerful open-source models. Anyone can download and run systems like Meta's Llama, DeepSeek, and OpenAI’s recently released GPT-OSS series without restriction, creating private, untraceable tools for malicious activity. The potential for misuse was demonstrated when the first known AI-ransomware, powered by OpenAI's gpt-oss:20b, was discovered just three weeks after the model’s release.

Beyond the models, the ecosystem of AI pentesting assistants like TARS and AI-OPS is maturing rapidly, democratizing advanced offensive tools. Combined with constant progress in "jailbreaking" techniques that dismantle safety guardrails, AI is removing barriers for everyone: Nation-states can scale their operations, while under-resourced cybercriminals can now deploy sophisticated malware that was previously beyond their reach.

‍
As the National Cyber Security Centre (UK) warns, "AI will almost certainly increase the volume and heighten the impact of cyber attacks over the next two years." Gartner further predicts AI agents will reduce the time to exploit account exposures by 50% by 2027, enabling attackers to move at machine speed. 

The Unprepared Defender

The irony of the moment is this: while everyone is still wondering whether the models are safe and debating risk, the functionality is already being used.

Enterprises—especially the largest—are slow to adapt. While attackers use AI agents to generate and rewrite malware in seconds, most defensive processes remain manual or rely on outdated automation that's no longer keeping up.

Across the board, defenders are under-resourced and underconfident. Per Darktrace report

• 45% of CISOs say they are not ready for the reality of AI-powered threats.
• 50% of security professionals don’t trust legacy tools to detect AI-driven attacks.
• The #1 inhibitor of organizations' abilities to defend against AI-powered threats is insufficient personnel

Defensive security often resists change, whether due to compliance constraints, budget cycles, or entrenched assumptions. The result is predictable: by the time a new control is deployed, the threat has already evolved.

Early Signs of Chaos

The consequences of defensive stagnation are measurable and escalating.

Since the release of ChatGPT in late 2022, we’ve seen a sharp, global rise in both attack volume and breach impact. Tactics like phishing, deepfakes, and document forgery have exploded in scale and sophistication:

• Phishing attacks have surged 4,151% (SOCRadar, 2024)
• Deepfake attacks now occur every five minutes (Entrust, 2024)
• Digital document forgeries have increased by 244% (Entrust, 2024)

These attempts turn into security breaches, and the Healthcare sector offers a great illustration. According to the HIPAA Journal, a record high of 136 million patient records were breached in 2023. In 2024, the number was significantly higher, with no signs of slowing.

Defense Will Catch Up

AI Will Power the Next Generation of Cyber Defense

While the immediate future portends increased cyber chaos, it's crucial to understand that this is a transitional period.

The same principles that give attackers their current edge —speed, scale, and automation—are the keys to a more defensible future. The long-term advantage will not go to the side that hires more people; it will go to the side that most effectively operationalizes AI for defense.

While 90% of CISOs expect a significant impact from AI-powered threats in the next two years, most also believe AI can improve the speed and efficiency of cyber defense. The goal is to move from a reactive, under-resourced posture to a proactive, continuous one.

The key to a more stable cyber defense is adoption. Defenders need to evolve now. Security tooling must match the capabilities and speed of both modern development and modern attackers.

Forging the Tools for Autonomous Defense

Fighting AI-powered offense with AI-powered defense is the only viable path forward. This raises two critical questions: First, where can AI make the biggest impact? And more importantly, can it truly achieve the expert-level creativity needed to win?

The answer to the first question is clear - AI-powered defense is essential across the entire security stack. A new set of companies is applying AI to everything from intelligent response to adaptive controls. A crucial segment of these, including firms like XBOW, is focused on offensive security: using autonomous AI to think like an attacker and find vulnerabilities before attackers do.

The answer to the second question—can AI be that good?—is being proven in the field. We validated this by pitting our autonomous agent, XBOW, against top human talent on HackerOne. It achieved the #1 rank, decisively outperforming elite researchers. But matching expert skill is just the baseline. The true advantage is superhuman speed and scale: XBOW operates 80x faster than manual teams and has already discovered over 1,400 zero-day vulnerabilities.

Crucially, these capabilities are not static. They are built on foundation models that improve at an exponential rate, meaning today’s performance is merely the baseline for tomorrow's. The growth in AI’s cyber-specific reasoning is staggering:

The Next 24 Months are Critical

Over the next two years, the gap between autonomous offense and traditional defense will widen. The organizations that adapt now will make it through. The ones that don’t will struggle to keep up.

Early adopters of AI-native security will gain real, durable advantage. They’ll move from reacting to attacks to continuously hardening their systems. They’ll test faster than they ship. And they’ll do it at a scale humans alone can’t match.

Closing Thoughts

The chaos is real, but it won’t last forever.

Right now, the offensive side has the advantage. Autonomous AI agents are lowering the barrier to entry, accelerating attack cycles, and outpacing legacy defenses. But this is a transitional phase, not the new normal.

As with every wave of disruptive technology, there is an initial, temporary, asymmetry. The defenders will catch up and get ahead, but only if they adopt the same mindset: automation, scale, and speed. Manual security practices cannot hold the line against machine-speed threats.

Autonomous offense demands autonomous defense. The only way forward is to build a security posture that lets you see the future of offense - before it sees you.