AI’s Evidence Problem

A deep dive into why data sits at the center of securing AI, and what practitioners actually need to see

Throughout my career in cyber I have watched our industry meet every major technology shift the same way, which is with a governance committee, a fresh acronym, and a vendor category to match.

Cloud got CSPM, containers got runtime security, SaaS got SSPM, and AI is now working through its own alphabet soup. Some of that is warranted, given how different these architectures are from what came before.

That said, when you strip the AI security conversation down to what is actually going wrong inside enterprises today, most of it is an old problem wearing a new outfit. Sensitive data going somewhere it should not, into systems nobody inventoried, through accounts nobody manages, leaving behind no record anyone can query afterward.

We have written the acceptable use policies, stood up the AI committees, and run the prompt injection tabletops, and nearly all of it rests on an assumption that does not survive contact with a real environment, which is that we often can’t see what our AI systems are actually doing.

In this article I want to walk through the problem space the way practitioners are running into it, starting with why data became the attack surface, then the visibility gap sitting underneath most AI governance programs, and finally what AI is doing to detection engineering.

From there I will use Cribl, a Resilient Cyber partner whose team walked me through their platform and their AI security approach across two conversations this summer, as an example of how one vendor is coming at it.

The Data Was Always the Point

To level set on how fast this moved, Verizon’s 2026 DBIR has a Shadow AI section drawn from its data loss prevention telemetry, and it found that 45% of employees are now considered regular users of AI on their corporate devices, authorized or not, up from 15% the year before, which is a tripling in twelve months.

67% of users are reaching AI services through non-corporate accounts on corporate hardware, which Verizon notes is actually a slight decrease from the prior year, while Shadow AI has climbed to the third most common non-malicious insider action in that dataset, a fourfold increase in percentage terms. What is leaving matters as much as how much of it, and Verizon found source code to be the most common data type going into external models.

For those who prefer breach outcomes to policy violations, IBM’s Cost of a Data Breach Report 2026 found that incidents involving shadow AI more than doubled this year, to 43% from 20%, and cost more when they happened, at an average of $5.39 million against $4.63 million the prior year.

More than 20% of organizations reported a breach targeting AI models or applications outright. When IBM looked at how those AI systems were actually compromised, the two leading causes were compromised APIs, applications or plug-ins at 27%, and cloud misconfigurations affecting AI workloads, also at 27%.

That last finding is key, because it tells you where the risk actually lives. The AI systems getting breached are not being broken through exotic model attacks. They are falling to integrations, plumbing, and misconfigured Cloud, which is to say the same things that have been breaching everything else for fifteen years.

The governance numbers in the same report are worth highlighting as well. 68% of breached organizations had no AI governance in place to manage AI or detect shadow AI, which is worse than the 63% reported a year earlier, meaning the gap widened rather than closed during a year of exceptionally fast adoption. Among organizations that reported an AI-related breach specifically, 92% lacked proper AI access controls, and only 40% reported using access controls on AI models and data at all.

To be fair, the industry is not standing still. The World Economic Forum’s Global Cybersecurity Outlook 2026 found the share of respondents assessing the security of their AI tools nearly doubled in a year, from 37% to 64%. That is real progress but it also leaves more than a third not assessing those tools at all, which is a tough position to defend in 2026, particularly when 87% of those same respondents called AI-related vulnerabilities the fastest-growing cyber risk of the past year.

Sanctioned Is Not the Same as Instrumented

Now, the instinctive response to shadow AI is to sanction the tools, buy the enterprise tenant, and declare the problem managed. I understand the impulse and it is not wrong, exactly. It just solves a different problem than the one people think it solves.

Moving a Developer from a personal ChatGPT account onto a corporate tenant addresses account governance. It does very little for the data governance underneath it. The prompts still contain source code, the context windows still pull from internal document stores and wikis, the retrieval layer still reaches into systems with their own access models, and the outputs still land somewhere nobody is watching. The organization traded an unmanaged account it could not see into for a managed account it still cannot see into, and then closed the risk item anyways, despite the gap.

ISACA’s 2026 AI Pulse Poll, which surveyed more than 3,400 practitioners across IT audit, governance, cybersecurity, privacy and emerging tech roles, put a number on it. Only 43% said they were completely or fairly confident their organization could investigate and explain a serious AI incident to leadership or regulators. More than half do not know how quickly they could halt an AI system in response to a security incident.

That first figure is, to me, the whole article in one line. Investigating and explaining an incident is not a policy capability or a governance capability. It is an evidence capability, and evidence means logs somebody decided to collect before the thing happened. A majority of ISACA’s respondents could not say they were confident their organization could reconstruct what an AI system did.

Everything else in an AI security program sits downstream of that.

You Cannot Inventory What You Never Logged

Every serious piece of AI security guidance published in the last three years starts in the same place, which is inventory.

NIST has covered this in various publications such as the AI Risk Management Framework which calls for inventorying AI systems and their GenAI Profile, which pushed for organizations to enumerate AI systems and continuously monitor third-party ones being leveraged.

OWASP gets to the same place from the threat side, where the Excessive Agency guidance asks teams to log and monitor LLM extension activity, and is careful to frame that as damage limitation rather than prevention. It’s worth noting that excessive agency also jumped to #3 on the latest OWASP LLM Top 10 as well, due to the impact agents are having in real-world incidents.

The OWASP Top 10 for Agentic Applications, released at the end of 2025, is where this stops being aspirational and becomes an engineering requirement. Goal hijack asks for “comprehensive logging and continuous monitoring of agent activity” plus a behavioral baseline covering goal state, tool-use patterns, and invariant properties such as schema and access patterns.

Tool misuse wants immutable logs of every tool invocation and parameter change, along with monitoring for unusual tool-chaining. Rogue agents wants signed, immutable audit logs of agent actions, tool calls, and inter-agent communication.

Read those as an engineer rather than as a GRC practitioner, and what OWASP is asking for is a behavioral baseline per agent, immutable logs of every tool call, and signed records of agent-to-agent traffic. Every one of them is a telemetry problem before it is an AI problem, and I would wager most organizations reading this could not satisfy a single one today for a single production agent, let alone all of them.

The Cloud Security Alliance published research in March in which 68% of respondents said they cannot clearly distinguish AI agent activity from human activity, and 79% believe agents create new access pathways that are difficult to monitor. The shape of it matches what I hear from practitioners constantly, and it is the same structural problem we have failed to solve for non-human identities generally, which I wrote about previously in an article titled “Identity Is the Agentic AI Problem Nobody Has Solved Yet”.

Now, none of this means the sky is falling, and I want to be careful not to overstate where enterprises actually are. Gartner expects more than 40% of agentic AI projects to be canceled by the end of 2027 on cost, unclear value, or inadequate risk controls. A great deal of what is marketed as the agentic enterprise is pilot work, as organizations stumble through the messy work of trying to operationalize this disruptive and quickly evolving technology.

That is precisely the argument for building the instrumentation now, while these architectures are still taking shape, before the telemetry debt compounds the way our vulnerability backlogs did, and before we spend another decade bolting visibility onto systems designed without it.

We know how that story ends because we have lived it with Cloud, with SaaS, and with every wave before them.

The Volume Problem Nobody Budgeted For

The uncomfortable part is that the answer to all of the above is “log more,” and that runs directly into the economics of every SIEM contract in the industry.

Telemetry growth by itself is an old problem and I won’t rehash all of the details. What makes it different this time is the character of the data rather than just the quantity of it.

Prompt and completion logs, tool call traces, retrieval events, agent-to-agent messages, and MCP server interactions arrive verbose, semi-structured, and inconsistently schematized across every provider, framework, and gateway in the stack. Making that queryable is exactly the kind of undifferentiated engineering work detection teams have no capacity for, and it lands on the same people already drowning in alert backlogs.

So we arrive at the moment where OWASP asks for immutable logs of every tool invocation and the practitioner reading that guidance is doing math on what their ingest tier costs per gigabyte. Better policy does not resolve that tension, as long as the cost of retaining a log stays coupled to the cost of analyzing it, security teams will keep making retention decisions on budget grounds and then discover, mid-incident, that the evidence they needed got dropped at the pipeline six months earlier.

I have written about the structural version of this in “The SIEM’s Structural Problem and Why It Matters Now More Than Ever” and “The Data Layer Is the New Battleground for the Agentic SOC”, so I will leave it there.

The Pyramid Is Shifting

The other half of this, and the half that came up in my conversation with Nicole Beckwith on Cribl’s security side, is what AI is doing to the indicators we have been building detections on for a decade.

For those unfamiliar, David Bianco published “The Pyramid of Pain” back in 2013 and it remains one of the most durable mental models in detection engineering.

It ranks indicator types from hash values at the bottom, through IP addresses, domain names, network and host artifacts, and tools, up to TTPs at the top, ordered by how much pain denying each one causes the adversary.

His read on the bottom of the pyramid has aged extremely well. Any reasonably advanced adversary can rotate IPs “with very little effort,” and if you burn one of them they can usually recover “without even breaking stride.” Hashes are worse, since flipping a single inconsequential bit produces a completely unrelated value.

Bianco came back to this with data in 2022, analyzing VirusTotal metadata for over 11 million unique files submitted during 2021, and found that 91.81% of them were submitted by a single submitter, with only 0.33% seen by more than ten organizations. His conclusion was blunt, and it was to stop using third-party hash values for detection, because the effort to reward ratio does not work.

The governments have since made the same argument one level up. The joint advisory on fast flux published in April of 2025 by NSA, and others notes that a typical fast flux domain may change its IP address every 3 to 5 minutes, which “renders IP blocking irrelevant” because the address is out of use before the block lands.

Beckwith’s framing was that AI has accelerated this shift rather than created it, and I think that is the correct and appropriately un-hyped read. The evidence that adversaries are operationalizing AI is real and it is now documented by the model providers themselves. Google’s Threat Intelligence Group reported in November that for the first time it had identified malware families using LLMs during execution, generating scripts and obfuscating their own code on demand rather than hard-coding it.

One family, PROMPTFLUX, carried a function to have Gemini rewrite its own source hourly, though GTIG noted that function was commented out, which is the kind of detail that separates threat intelligence from marketing. Another, PROMPTSTEAL, was used by APT28 against Ukraine and represents GTIG’s first observation of malware querying an LLM in live operations. Anthropic separately reported a campaign against roughly thirty targets where, by their assessment, AI performed 80 to 90% of the work with humans intervening only sporadically, at perhaps four to six critical decision points.

OpenAI’s October threat report concluded across the activity it disrupted that it found “no evidence of new tactics” or that its models handed threat actors novel offensive capabilities. In the individual operations it wrote up, OpenAI described actors who appeared to be primarily seeking incremental efficiency in workflows they already had.

M-Trends 2026, grounded in over 500,000 hours of Mandiant incident response last year, lands in the same place, noting adversary AI adoption while reporting that, from its own frontline vantage point, the vast majority of successful intrusions still stem from “fundamental human and systemic failures.” Global median dwell time actually rose to 14 days from 11, and exploits remained the most common initial infection vector for the sixth straight year at 32% of intrusions.

So the honest synthesis, less exciting than either the doom or the dismissal, is that AI is not rewriting the threat model from scratch. It is compressing the cost and the cycle time of everything at the bottom of the pyramid, which was already cheap for adversaries and already low-value for defenders.

Regenerate the binary, rotate the infrastructure, rewrite the script, and every atomic indicator you were matching on is stale before your feed refreshes. MITRE’s Center for Threat-Informed Defense makes the structural version of this argument in Summiting the Pyramid, pointing out that a detection built on a hash will miss the same malware altered by a single byte, while TTPs sit at the top precisely because they are the most expensive thing for an adversary to change.

Detecting behavior rather than artifacts has been the stated goal of detection engineering for a decade. What has kept most teams from getting there is not conviction.

It is that behavioral detection requires broad, retained, correlatable telemetry across sources, and that requirement collides head-on with the volume and cost problem in the previous section. Which brings the two halves of this article together.

Four Things, All of Them Telemetry

Strip away the vendor positioning and the requirements for securing AI in an enterprise reduce to four things, and Nicole from Cribl’s framing of them makes a ton of sense.

• First, identify what exists, meaning an inventory of models, agents, tools, plugins, and extensions, including the ones nobody registered.

• Second, observe how those things are actually being used, which means prompts, input context, and session activity rather than a count of API calls.

• Third, track where data is going, meaning what feeds the models, where it is retained, and what leaves the boundary.

• Fourth, alert on malicious activity, including exfiltration, leakage, abnormal usage spikes, compromised accounts, and agents deviating from how they normally behave.

Every one of those four is a telemetry problem. You cannot inventory what does not emit, you cannot baseline behavior without history, and you cannot detect deviation without both.

The AI security tooling market is full of products that quietly assume this substrate already exists, and in most enterprises it does not.

A Look at One Approach in Practice

Which brings me to Cribl, who partnered on this piece and whose team spent two sessions this summer walking me through their platform and their AI security approach.

Cribl is useful as the example here for a specific reason, which is that they came at AI security from the data layer rather than the model layer, which is especially relevant as we as an industry realize the model isn’t a security boundary.

The company positions itself as “Your AI Platform for Telemetry”, They also earned a FedRAMP Moderate ATO in January, which was relevant for me given I came from the public sector both as a contractor supporting agencies, and on the GSA FedRAMP team and I know agencies desperately need innovative capabilities when it comes to AI.

I chatted with Nick Heudecker, who leads market strategy and competitive intelligence at Cribl, and he described customers seeing data volumes climb 80 to 100 percent annually, with some going from one petabyte to 2.5 petabytes per day of ingestion, and most of them not remotely ready for that from a governance or security perspective.

The architectural argument he made is one I have made myself in a different context, which is that a pipeline sitting between sources and destinations functions as a two-way door. It lets you change SIEMs, augment with best-of-breed tools to avoid vendor lock, route the same data to multiple destinations, and avoid a world where your detection strategy gets dictated by whatever your ingest contract makes affordable.

On the four pillars, Cribl’s AI observability app runs on top of that pipeline. Publicly the company pitches it as one place to search, investigate and report on AI telemetry across models, tools and environments, surfacing AI usage, sensitive data and cost.

In the demo that meant an inventory of models, agents, tools, plugins and extensions, extending into prompts, input context and session activity. For the data protection layer, Cribl Guard ships with over 200 configurable policy rules and more than 220 prebuilt detection patterns, organized into rulesets for secrets and credentials, bank and credit cards, medical codes and licenses, network authentication and configs, and PII, running either in an active mode that masks in real time or a background mode that samples and lets you decide what to protect automatically.

I want to be equally clear about the boundaries here, because the Cribl team was during our convos, and that candor is a good part of why I found the conversation worth writing up.

Cribl operates as an aggregator. For bespoke agents running on endpoints it leans on existing EDR telemetry rather than shipping its own sensor, and for tracking where data goes it currently depends on tooling like DLP that you already own.

Once that telemetry is ingested you can drive policy decisions from it and route enforcement accordingly, but the collection depends on instrumentation already in place. For most enterprises that is a feature rather than a limitation, since the alternative is yet another agent on the endpoint. It does mean Cribl’s AI visibility is bounded by the quality of the sources feeding it, and any evaluation should start there rather than with the dashboard.

The fourth pillar is where the demo got genuinely interesting. Nicole’s team had built a threat intelligence platform on Cribl’s app framework to ingest IoCs, and then ran into exactly the wall this article walked through above, which is that the bottom of the pyramid has become ephemeral enough that the effort stops paying for itself.

What they built in response is called APEX, and it maps MITRE ATT&CK TTPs as individual signals against raw telemetry and clusters those signals into behavioral chains, generating detections from the previous 30 days of telemetry to set a baseline, with users able to tailor criticality and monitoring windows to their own risk tolerance.

Publicly, Cribl announced new security capabilities in August that include stream-native detections running on telemetry as it moves through the pipeline, plus detection engineering capabilities out of their CardinalOps acquisition that map detections to ATT&CK, expose coverage gaps, and surface broken and noisy rules before they fail silently.

Two design choices in the capability Cribl demoed are worth pulling out for practitioners. The first is that it works against any log source piping into Cribl without requiring OCSF normalization, with detection engineers working from roughly the top 15 critical fields that stay consistent across sources rather than parsing every field in every source.

Anyone who has been involved in a data normalization project knows how much undifferentiated effort that avoids, and normalization debt is one of the quieter reasons detection coverage stalls out.

The second is that running detections closer to the telemetry layer, rather than after data lands in an analytics platform, shortens the distance between an event happening and an alert firing.

Combined with the correlation work their team described as weeks out at the time of our conversation, clustering activity within time windows so alerts fire against behavioral baselines, the intent is to avoid a world where covering the technique space means authoring and maintaining tens of thousands of individual detections. Whether that holds up at enterprise scale is something practitioners will need to validate in their own environments, and I would want to see it running against a genuinely messy estate before drawing conclusions but the premise makes sense and is promising.

Closing Thoughts

This is far from an exhaustive discussion of AI security, and there are entire domains I have not touched, from model supply chain to inference-time attacks to the agent identity problem, among others.

That said, the the recurring theme is hard to miss. NIST wants an inventory, OWASP wants immutable logs of every tool invocation and a behavioral baseline for every agent and regulators are going to want you to explain what happened. None of that is achievable without a data layer built to carry it, and most security practitioners being surveyed don’t feel confident they could produce that today.

There is a version of this market that sells you a model firewall, a prompt scanner, and yet another dashboard, and leaves you exactly where you started, unable to answer basic questions about what your AI systems did on any given day. I do not think we get out of this by adding another silo. We learned that in the Cloud era and relearned it with SaaS, and the organizations handling AI adoption best so far are the ones treating it as another set of sources and destinations in a pipeline they already understood.

The ask is tough too though, as we are being asked to expand telemetry and add behavioral baselining for an entirely new class of non-human actor in agents at the exact moment most teams are under pressure to cut ingest costs due to runaway bills. Something has to give there, and I would much rather it be the assumption that we must pay analytics prices to retain evidence than the evidence itself.

So, will we build the instrumentation while these architectures are still forming, or spend the next five years reconstructing what our agents did from logs we chose not to keep?

We have run this experiment before, and we know how it goes when security shows up after the fact, which is disjointed tech stacks, open questions in our incident response and the inability to deliver on secure outcomes for the businesses we support.