The $100 billion stack gap: Why banking’s agentic AI dreams will become infrastructure nightmares

The financial services industry is at an inflection point when it comes to effectively leveraging AI. Across boardrooms at major banks, the conversation is the same: “We need agentic AI.” The promise is intoxicating — autonomous systems that think, reason and act independently to transform operations. 

But here’s where we need to introspect: A new MIT study reveals that 95% of enterprise GenAI pilots are failing to deliver measurable business impact.

The research, published by MIT’s NANDA initiative, analyzed 150 executive interviews, surveyed 350 employees and examined 300 public AI deployments. The findings are stark: While about 5% of AI pilot programs achieve rapid revenue acceleration, the vast majority stall at the pilot stage, with little measurable impact on profit and loss.

MIT uncovered a critical insight: The failures aren’t due to model quality. They stem from flawed enterprise integration — what MIT calls the “learning gap” between sophisticated AI capabilities and the organizational infrastructure needed to scale.

A $100 billion industry challenge

The “stack gap” represents the hidden costs between what banks budget for agentic AI and what full implementation actually requires. We estimate this gap could exceed $100 billion globally:

• Infrastructure retrofit: In 2024, the financial services industry invested $45 billion in AI, with banking accounting for $31 billion of that. But most infrastructure cannot support agentic AI’s demands. Data center upgrades typically require three to five times the original AI investment budget, translating to more than $40 billion in unplanned costs.
• Integration complexity: For example, UK banks typically use hundreds of different systems. Each AI-to-system integration costs $100,000–$500,000. For global banks needing thousands of connections, integration costs could surpass $25 billion.
• Multivendor coordination: RAND Corporation research shows AI project failure rates of up to 80%, with multivendor complexity a primary factor. Inefficiencies add 30%–50% to costs — potentially wasting $40 billion.
• Pilot-to-production gap: MIT found 95% of pilots failed to scale, while RAND reports 80% overall AI project failures. Applied to banking’s $31 billion annual AI investment, that’s $23 billion in lost opportunity.

The lesser-known truth about agentic AI is that it’s not really about intelligence at all. It’s about the stack.

Why the stack gap is killing AI dreams

Step into many banks and you’ll find an extremely complex technology landscape — decades of systems stitched together. Core banking systems from the 1990s talk to cloud applications through middleware held together with digital duct tape. Data is scattered across hundreds of systems, while infrastructure designed for batch processing now faces real-time demands.

Now imagine asking this technology estate to support autonomous AI agents that must make split-second, compliant decisions with perfect data accuracy. It’s like expecting a dependable, hardy old station wagon to compete in Formula 1.

This is the stack gap — the chasm between what agentic AI demands and what most banks can deliver. Too many providers build “penthouse suites” on quicksand foundations. The truth is simple: You cannot implement agentic AI without simultaneously transforming your entire technology stack.

• ALSO READ → From cockpit to command center: How agentic AI is redefining business operations

The infrastructure-first reality

Most AI programs start with algorithms and fail on infrastructure. At NTT DATA, we advocate the reverse: Start with the robust foundations that enable AI to function.

Take fraud detection. An AI agent identifying a suspicious transaction may need global risk assessment in milliseconds. This requires:

• AI-ready infrastructure that processes at the edge with ultralow latency
• Network architecture that distributes decisions globally without breaching data sovereignty
• Real-time pipelines that correlate information across dozens of systems instantly

As the world’s third-largest data center provider, with a $10 billion investment until 2027 to deliver AI-ready infrastructure, NTT DATA doesn’t just host AI for clients. We also architect the global nervous system that makes agentic AI possible.

• ALSO READ → High-yield AI in financial services depends on secure, scalable networks

From banking backbones to AI nervous systems

At NTT DATA, we know agentic AI can’t thrive on shaky ground. We've proven it before by building BOJ-NET and CAFIS payment systems forming the invisible backbone of Japan's financial system, and now by reinventing core banking with our Integrated Banking Cloud.

We’ve done the same across industries with platforms like TradeWaltz and FEDI, creating trusted digital rails on which entire ecosystems can run. Today, our Smart AI Agent™ Ecosystem takes that foundation global, accelerating the adoption of agentic AI with industry-ready solutions.

We’re not building penthouses on quicksand. We’re engineering the AI-ready nervous system of the future.

Why NTT DATA is different

Unlike AI-only startups, we have spent decades building the global infrastructure that makes autonomous intelligence real. That’s why we’re uniquely positioned to help banks avoid the $100 billion stack gap:

• Complete stack ownership: From data centers and AI algorithms to business transformation
• Global scale: A $30 billion company serving 75% of the Fortune Global 100
• Proven integration: Successful large-scale financial services deployments
• Strategic partnerships: OpenAI Center of Excellence and alliances with all major cloud providers

With NTT DATA, financial institutions don’t just experiment with agentic AI. They scale it.

The SIBOS 2025 reality check

As leaders gather in Frankfurt, Germany, this September for SIBOS 2025, there are some questions to ponder:

• How do you bridge the stack gap to enable large-scale deployment of AI? 
• Can your AI vendor guarantee sub-millisecond latency across global markets?
• Can your cloud provider ensure compliance with every jurisdiction’s data laws?
• Can your systems integrator modernize legacy banking systems without disruption?

The future belongs to full-stack AI

The future of agentic AI in banking isn’t just about smart algorithms. It’s also about smart infrastructure. Those who understand this and partner with providers who can deliver both will lead the industry transformation.

At NTT DATA, we’ve built both.