Custom-built AI applications are set to cause major headaches for security teams over the coming years, unless they can get involved in projects early on, Gartner has warned.
The analyst predicted that by 2028, at least half of enterprise incident response efforts would be devoted to managing the fallout from security issues connected to these apps.
“AI is evolving quickly, yet many tools – especially custom-built AI applications – are being deployed before they’re fully tested,” warned Gartner VP analyst, Christopher Mixter. “These systems are complex, dynamic and difficult to secure over time. Most security teams still lack clear processes for handling AI-related incidents, which means issues can take longer to resolve and require far more effort.”
By “shifting left” security teams can ensure adequate controls are built in from the start, he added.
Read more on AI threats: Gartner: 40% of Firms to Be Hit by Shadow AI Security Incidents.
However, Gartner also predicted a growing role for AI-powered security tools. It claimed that, in two years’ time, half of organizations would use AI security platforms to protect third-party AI service usage and custom-built AI apps.
These tools help teams enforce acceptable use policies, monitor activity and apply consistent guardrails across AI apps, protecting them from prompt injection, data misuse and other threats, the analyst claimed.
AI‑powered “identity visibility and intelligence platforms” are also set for take-off as organizations look to improve detection and remediation of human and machine identities, Gartner said.
A Sysdig report last year claimed that machine identities now outnumber human users by 40,000 to one and present 7.5 times more risk. Over-permissioned AI agents are particularly concerning.
Sovereignty Set to Take Center Stage
Gartner’s predictions extended beyond the AI realm.
It claimed that, by next year nearly a third (30%) of organizations will demand “comprehensive sovereignty” of cloud security controls in order to mitigate continued geopolitical risk.
Local regulatory demands and geopolitical turmoil are driving this change in approach, and CISOs must play a key role in defining sovereignty requirements, Gartner said.
However, research from UK startup Arqit last month revealed that, unless handled intelligently, such mandates could be a drag on innovation.
Some 62% of responding organizations cited data sovereignty and privacy risks as the biggest factor slowing AI projects when using a public cloud.
“At the same time, one in six organisations tell us they cannot guarantee sovereign facilities at all, and only 8% can rely on sovereign edge environments today,” said Arqit CEO, Andy Leaver.
“That gap is exactly where security leaders need stronger controls for data in transit, better visibility into cryptographies, and a practical means of building trust across cloud and edge.”
Arqit and others are promoting confidential computing as a means to achieve sovereignty without compromising on security. It’s a technology approach which carves out secure enclaves at a processor level that isolate and protect data in use.