Few topics generate more hype and more genuine substance simultaneously than AI in cybersecurity. Cutting through the marketing requires separating what machine learning actually does well, where it’s overhyped, and how the same capabilities are already being weaponized by attackers.
Where AI Genuinely Helps Defenders
Anomaly Detection at Scale
Behavioral baselining, the foundation of modern NDR, UEBA (User and Entity Behavior Analytics), and cloud security posture tools, depends on machine learning models that can process volumes of log and traffic data far beyond manual analyst capacity, flagging statistically unusual patterns a rules-based system would never catch. A user account suddenly accessing 200 files it’s never touched before, at 3 a.m., from a new device, is exactly the kind of multi-dimensional anomaly ML excels at surfacing.
Alert Triage and Enrichment
Large language models are increasingly used to auto-summarize complex alerts, correlate related events across disparate log sources, and even draft initial incident timelines, meaningfully reducing the manual investigation burden discussed in earlier SOC operations content. This doesn’t replace analyst judgment, but it compresses the time between “alert fires” and “analyst has enough context to make a decision.”
Malware and Phishing Classification
ML classifiers trained on file characteristics and email content patterns can identify novel malware variants and phishing attempts that evade traditional signature and reputation-based detection, since they generalize from learned patterns rather than requiring an exact known-bad match.
Where AI Is Overhyped
- “AI will replace your SOC analysts”: current AI tools augment triage and investigation; they don’t yet reliably make the judgment calls required for ambiguous, novel incidents, or navigate the organizational context needed for containment decisions with real business impact.
- “Our AI-powered product stops zero-days”: genuinely novel attack techniques by definition lack training data for any model to have learned from; AI improves detection of known-pattern-adjacent behavior, it doesn’t grant precognition.
- Black-box explainability gaps: a model that flags something as malicious without explainable reasoning creates its own operational risk: analysts can’t validate, tune, or trust decisions they can’t understand, and compliance/audit requirements increasingly demand explainability.
How Attackers Are Using AI
The same capabilities cut both ways, and defenders need to understand the offensive side to prepare for it:
- AI-generated phishing content eliminates the grammatical tells and awkward phrasing that used to help users spot scams, and can generate highly personalized spear-phishing content at a scale that previously required significant manual research per target.
- Voice cloning for vishing and BEC: a handful of seconds of audio, often scraped from public video or social media, is now enough to convincingly clone a voice for fraudulent phone-based social engineering, including impersonating executives requesting urgent wire transfers.
- AI-assisted vulnerability discovery: attackers are using large language models to accelerate the process of identifying exploitable patterns in code and generating working exploit code faster than manual analysis alone.
- Deepfake-enabled social engineering: video and audio deepfakes are increasingly used in targeted fraud, including at least one widely reported case where a finance employee was convinced to authorize a large transfer after a video call with deepfaked “executives.”
Preparing Your Organization for an AI-Accelerated Threat Landscape
- Update security awareness training to address AI-generated phishing and voice cloning specifically: the old advice of “look for typos and bad grammar” is now actively obsolete.
- Strengthen out-of-band verification processes for financial transactions and credential changes, since these are precisely the workflows AI-enhanced social engineering targets: a callback to a known number defeats a deepfake regardless of how convincing it is.
- Evaluate AI security tools for explainability, not just accuracy claims. A model your analysts can’t interrogate is a model they’ll eventually learn to distrust or ignore.
- Treat AI-assisted code review and vulnerability scanning as an accelerant, not a replacement, for secure development practices: faster feedback loops in your own pipeline help you find and fix issues before attackers, using similar tooling, find them first.
The Bottom Line
AI is neither the security panacea vendors promise nor a fad to dismiss: it’s a genuine capability shift that’s already reshaping both defense and offense simultaneously. Organizations that invest in understanding its real capabilities and limitations, rather than either uncritical adoption or dismissal, will be better positioned as this technology continues to reshape the threat landscape on both sides.