The Implementation Gap: Why AI Pilots Don't Translate Into Economic Impact

Introduction: The Technology Is Available, the Impact Is Not

Over the past three years, artificial intelligence has evolved from a subject of strategic declarations to an element of everyday corporate practice. Access to models is no longer a barrier: it's purchased by subscription, deployed within company perimeters, and embedded into office and production systems. But it's precisely at this moment that a paradox has emerged which now defines the agenda: the technology has become mainstream, but measurable economic impact has not.

The most resonant formulation of this came from the MIT NANDA initiative's report "The GenAI Divide: State of AI in Business 2025." The authors examined more than 300 publicly disclosed corporate initiatives, conducted structured interviews with executives and employee surveys—and concluded that approximately 95% of generative AI pilot projects produce no measurable impact on financial results, remaining stuck as proofs of concept or local tools for a single team. Success is concentrated in a narrow group: about 5% of implementations reach production deployment and deliver tangible results for the company.

These figures need to be read correctly. They don't mean that AI doesn't work. They mean something else: the divide runs not between "smart" and "dumb" models, but between organizations that have restructured their processes and organizations that have embedded a model into an unchanged process. The former get results, the latter get a pilot report.

Anatomy of the Divide: Five Typical Breaking Points

The first breaking point: data not ready for production. A pilot is launched on a manually prepared data extract designed for demonstration. Production deployment requires a regular, versioned, quality data flow with a clear owner. Between these two states lies engineering work that's rarely budgeted into the pilot, because it doesn't look like "artificial intelligence."

The second breaking point: absence of impact metrics. A huge portion of projects are measured in terms of "employees are satisfied" and "things got faster." But saving minutes doesn't automatically convert to profit. If freed-up time isn't reallocated—to additional volume, to reducing external purchases, to a new product—the company is financing comfort, not productivity. Impact must be tied in advance to a line item in the P&L: cost per operation, cost per request processed, defect rate, cycle time, overtime volume.

The third breaking point: the process remains unchanged. A model embedded into an old procedure inherits its limitations. If the same four-stage visual review is preserved after document generation, cycle time won't change. Real impact emerges where the algorithm is followed by changes to procedures, control points, and distribution of responsibility.

The fourth breaking point: accountability for errors. Once a solution leaves the sandbox, the question arises: who's responsible for an incorrect recommendation—the implementer, the process owner, or the model provider? Until the answer is formalized in regulations, middle managers rationally prefer to keep the system in auxiliary status. And so the pilot lingers for years.

The fifth pitfall is choosing incorrectly between buying and building in-house. The MIT report highlights the distinction between solutions purchased from specialized vendors and those developed internally. Neither model is universal: in-house development makes sense where the process is a source of competitive advantage, while purchasing works for standardized processes. Getting this choice wrong costs more than picking the wrong language model.

The Russian context: plenty of pilots, few strategies

Russia's picture mirrors the global gap with its own specifics. On one hand, large businesses show strong interest: AI is on the agenda at virtually every major company. On the other, the management framework lags far behind investment activity. According to research by MTS Web Services, only about a quarter of companies that already budget for AI spending have a formalized implementation strategy. In other words, money gets allocated before anyone determines what management outcome it's supposed to buy.

The second factor is structural heterogeneity. Data from HSE University's ISSEK shows that the share of organizations using AI technologies averages around 4.8% across the economy, while among companies with over 500 employees it reaches nearly 15%. This gap between large businesses and the rest of the economy means AI currently acts as an amplifier for already strong players rather than a driver of broad-based productivity growth.

The third factor is infrastructure. The cost and availability of computing power remain a tangible constraint: demand for accelerators consistently outpaces supply, making proprietary model deployment expensive and cloud scenarios critically dependent on provider policies.

The fourth factor is talent—and it's the most underestimated. The shortage isn't so much in developers (estimates of the IT specialist deficit in the country run into hundreds of thousands) as in the intermediate layer: data engineers, process owners, and analysts capable of translating a management challenge into a proper model specification and then interpreting its output. This layer determines whether a pilot ever reaches production.

Why this is a macroeconomic question, not corporate fashion

For the Russian economy, productivity has ceased to be an academic topic. Demographic contraction of the workforce, persistent talent shortages, and wage growth outpacing output create scissors that can't be closed through hiring alone. Technological productivity improvement becomes not an option but a condition for preserving margins across entire industries.

That's why the implementation gap isn't an internal IT department problem. If the overwhelming majority of initiatives stall at the experimental stage, the national return on AI investment will fall well short of projections, which for Russia are measured in tens of trillions of rubles through 2035. Forecasts represent the upper bound of what's possible, not the automatic result of technology procurement.

It's also telling how global economic estimates diverge: projections for productivity gains from generative AI among leading analytical centers vary by multiples—from fractions of a percentage point annually to one and a half percentage points over a decade. Such a wide range is itself an indicator: the effect depends not on the technology but on the quality of its implementation.

What Sets the 5% Apart: The Roadmap from Pilot to Production

Experience from successful implementations reveals a fairly consistent set of conditions.

Process selection based on cost of error and frequency of repetition. Maximum impact comes from high-volume, standardized operations that are expensive to service: processing inquiries, initial document verification, defect detection, route and inventory planning. Showcase projects deliver impressive presentations but weak results on the P&L statement.

Pre-announced metrics and baseline. Before launch, document the current metric value, target value, and the deadline at which the project will be deemed unsuccessful. The absence of failure criteria is the clearest sign that a project was never intended for scaling.

Process owner, not technology owner. The person accountable for results must be the business function leader for whom the metric is an operational KPI. A project whose sole sponsor is the CIO rarely survives beyond the demonstration phase.

Redesigning procedures concurrent with implementation. If an algorithm completes a stage, that stage should disappear from the procedure rather than being duplicated by "just in case" oversight.

Institutionalized result verification. Human oversight remains but becomes selective, formalized, and measurable—with tracking of error rates and feedback loops into the model. This is simultaneously a question of efficiency and employee trust in the system.

A transparent conversation about employment. Implementation perceived as a disguised redundancy program meets passive resistance and degradation of input data quality. A clear workforce framework—retraining, task reallocation, revised performance standards—is a technological requirement, not merely an ethical one.

Conclusion: change the metric of success

The main takeaway from global failure statistics is not that the AI market is overvalued, but that it's being measured incorrectly. The number of pilots, the count of connected services, and investment volumes describe activity, not outcomes. What matters economically is a single metric: the share of solutions brought to production deployment, and verified impact in monetary terms.

For companies, this means shifting from experimental logic to operational model logic: fewer pilots, deeper process restructuring, stricter criteria. For public policy, it means redirecting support from demonstration projects to implementation infrastructure: data quality and accessibility, computing capacity, industry-standard solutions for mid-sized businesses, and training for mid-level specialists.

Artificial intelligence truly can become a driver of qualitatively new economic growth. But it is neither an accelerator in itself nor a substitute for management work. It merely reveals an organization's maturity with extreme precision: where processes are documented, data is organized, and accountability is distributed, impact emerges quickly. Where these elements are absent, AI dutifully automates chaos—and the implementation gap reproduces itself once again.