UpShaqo
Intelligence desk
Enterprise Agents Source-backed analysis

Why Most Enterprise AI Agents Never Leave the Pilot Stage

A new survey of 300 technology executives finds that two-thirds of agentic AI projects stall before production — not because the models are weak, but because the data behind them isn't.

UpShaqo Editorial IntelligenceOctober 7, 20266 min read
Intelligence standard

Independent UpShaqo analysis built from fresh, attributed sources. We explain the impact instead of repeating the announcement.

Read for leverage: focus on the workflow change, the customer problem, and the next action—not only the product announcement.

Picture a procurement agent that can draft a purchase order in seconds but doesn't know the vendor it just selected was flagged for a contract dispute last quarter. The agent isn't broken. It's blind. It has access to data — invoices, vendor lists, pricing tables — but no understanding of what that data means inside the company's own history and rules. That gap, not model quality, is quietly killing most enterprise agent projects before they ever reach production.

A new report from MIT Technology Review Insights, produced in partnership with Neo4j, puts a number on the problem. Based on a survey of 300 data, AI, and technology executives, the research finds that only about a third of agentic AI projects make it into production — and that even sophisticated high-tech firms aren't immune. The culprit, repeatedly, is a shortage of what the report calls knowledge: not raw data, but the contextual understanding that lets an agent reason about a specific organization's situations and act on them reliably.

Data-Rich, Knowledge-Poor

The distinction matters more than it sounds. Most enterprises are swimming in data. Few have organized that data into something an agent can actually reason with. The report breaks the problem into three capabilities organizations need to build: semantic knowledge (what concepts mean and how they relate), episodic memory (what has happened before, and when), and procedural knowledge (how tasks are actually supposed to get done). An agent missing any of the three tends to produce the same failure mode — technically correct outputs that are practically wrong, because they ignore context a human colleague would take for granted.

According to the research, legacy data systems, security and privacy concerns, and a lack of knowledge and context are the three most common points of failure. None of these are new IT problems. What's new is the cost of ignoring them: a stalled pilot used to be a minor write-off. A stalled agentic AI pilot, in a market where competitors are already automating decisions, is a forfeited efficiency gain that's harder to win back.

What the Leaders Do Differently

Not every organization is stuck. The report identifies a smaller cohort of what it calls production leaders — companies where, on average, 61% of agentic projects advance beyond pilot, roughly double the overall rate. The defining trait of this group isn't bigger budgets or flashier models. It's stronger knowledge capabilities, especially around semantics — in other words, better infrastructure for teaching agents what concepts and relationships actually mean inside the business.

Interestingly, the leaders also worry about different things than everyone else. Across the full survey population, data fragmentation — the inadequate sharing of data across systems — is the top-cited challenge, named by 55% of respondents. Production leaders, by contrast, are more likely to flag security and privacy concerns, cited by 72% of that group. The implication, drawn from the report's framing: once an organization solves the structural problem of connecting its data, the remaining friction shifts from "can we get the information" to "should an agent be allowed to use it this way." That's a healthier problem to have, but it's a different one, and it requires different governance.

A Hypothetical Rollout, Walked Through

To see how this plays out in practice, consider a plausible scenario built on the report's findings rather than a real case. A mid-size insurance operator wants an agent to triage incoming claims — checking policy terms, flagging likely fraud patterns, and routing complex cases to adjusters. In the pilot, the agent performs well on simple, high-volume claims where policy language is standardized. It starts failing on edge cases: claims tied to riders negotiated years earlier, exceptions granted during a prior system migration, or disputes referencing a since-retired underwriting rule.

The underlying data exists somewhere — scanned PDFs, an old CRM, a retired ticketing system. But it isn't linked to the active claims database in any machine-readable way, and no one indexed the exceptions as a category an agent could query. The pilot stalls, not because the model misunderstands claims, but because the organization never built the connective tissue between its systems and its history. This is precisely the fragmentation problem the survey's respondents cite most often, and it's exactly the kind of gap a knowledge layer is meant to close.

If the same insurer later built a structured knowledge layer — linking policy documents, exception histories, and underwriting rules into something an agent could traverse rather than guess at — the triage agent's accuracy on edge cases would plausibly improve, and more of the project's scope could graduate from pilot to production. That outcome isn't confirmed by the research; it's a reasonable extrapolation from what separates the report's production leaders from everyone else.

Where the Investment Is Actually Going

The report finds that executives expect the biggest improvement in agent decision quality to come from strengthening the structural foundation between an organization's data and its AI agents, and that the experts interviewed point to a knowledge layer as the primary mechanism for getting there. Concretely, organizations say they'll prioritize spending across a few categories: retrieval infrastructure such as ingestion pipelines, AI-ready APIs, and retrieval-augmented generation; AI evaluation agents that check outputs before they reach a human or a downstream system; and knowledge graphs that explicitly model how entities, events, and rules relate to one another.

Each of these solves a slightly different slice of the problem. RAG and ingestion pipelines widen what an agent can see. AI-ready APIs make that information consumable without brittle custom integration. Evaluation agents catch failures before they become customer-facing mistakes. Knowledge graphs, the piece the report emphasizes most, go further: they encode relationships — this exception overrides that rule, this vendor is linked to that dispute — so an agent isn't just retrieving documents but reasoning over structure.

The Tradeoff Leaders Should Weigh

Here's the analytical catch, not stated outright in the report but implied by its own data: building this infrastructure is slower and less glamorous than deploying a flashy agent demo, and it front-loads cost before any efficiency gain shows up. Organizations that skip straight to agent deployment without the knowledge layer are the ones most likely to land in the 66% whose projects never reach production. Organizations willing to invest in unglamorous data plumbing first are the ones most likely to join the minority seeing real returns.

For technology leaders evaluating their own agentic roadmap, the practical next step suggested by this research is to audit before building: map where semantic, episodic, and procedural knowledge actually lives today, identify which systems are fragmented from the rest, and treat security and privacy review as a scaling requirement rather than a launch blocker. The agents are rarely the bottleneck. The knowledge underneath them usually is.

Sources

#agentic AI#knowledge graphs#enterprise data#RAG#AI infrastructure#AI agents

Two doors. Pick one.

Hire the team.
Or become it.