Your AI strategy is only as strong as the infrastructure it runs on

Most AI strategy conversations start with the technology: Which model should we use? How do we improve our prompts? Should we build or buy? How do we govern AI responsibly?

They’re all important questions. But after more than two decades helping organizations modernize infrastructure and run business-critical environments, I’ve found they’re rarely the ones that determine whether any technology initiative succeeds. Because at enterprise scale, AI succeeds or fails on the foundation beneath the model.

That’s why I believe the most important AI conversation today is about the underlying infrastructure and operating model.

Building infrastructure for how AI actually works

For decades, infrastructure existed to provide reliable computing, storage and connectivity. AI is fundamentally changing what we expect infrastructure to do now.

Every model, workflow and autonomous decision depends on the platform underneath it to route intelligence, move data, protect autonomous systems and support continuous inference across cloud, edge and on-premises environments.

But most enterprise infrastructure was built for predictable workloads, stable traffic patterns and efficiency — not continuous AI inference. With AI, inference workloads fluctuate, GPUs become strategic assets, data moves continuously across environments and autonomous agents require secure, real-time access to multiple systems.

When AI projects hit a wall, it’s often because the cloud wasn’t designed for inference at scale, the network wasn’t built for AI traffic, or security and governance slow every interaction between systems.

Reliability is your most important metric

For years, infrastructure teams measured success by availability. They kept track of how long the system was up, whether the network was running and whether they had met their uptime targets.

With AI, the focus is not on whether your systems are “up” but whether they’re consistently producing the business outcome you expect. For example, your customers don’t care that every server is online if an AI assistant takes 15 seconds to respond, and your employees don’t want to hear that the network is technically available if an AI agent can’t retrieve the information it needs to complete a task.

Reliability means your AI services perform consistently under real-world conditions — not just that the underlying infrastructure stays online.

AI also demands a different way of operating

The first wave of enterprise AI was largely about experimentation. Like many organizations, you probably tested large language models, launched pilot projects, hired AI specialists and explored exciting use cases across the business.

Many of those initiatives proved the technology worked, but far fewer proved your business could run on it. Building an impressive AI model is one thing, but running AI as part of everyday business operations is something else entirely.

Every AI interaction depends on cloud, networking and security working together. A performance issue in the network can look like a problem with the model, while a security policy can introduce delays that users experience as poor AI performance. And simply adding more infrastructure isn’t enough — you can buy more GPUs, cloud capacity, monitoring tools and security products, but on their own, these investments rarely solve the problem. You also need to change how the environment operates.

Instead of managing cloud, networking and security independently, they have to function as a single operating environment for AI. That means shared visibility, coordinated automation and governance that spans every layer of the infrastructure.

Only then can your infrastructure adapt as quickly as the AI running on top of it.

For one of our clients, a global logistics leader, years of technical debt and fragmented cloud, network and security operations were limiting their ability to innovate and scale new technologies. Working with NTT DATA, they modernized their infrastructure foundation and, just as importantly, adopted a more integrated operating model that unified these environments. They are now delivering services more efficiently, are on track to save more than $50 million over 10 years and have built a stronger foundation for AI-driven innovation.

• ALSO READ → Your network is now at the center of the AI conversation

5 ways your infrastructure team needs to evolve

If AI requires a different way of operating infrastructure, what changes in practice?

Here are five shifts that are already emerging in leading organizations:

1. From monitoring to observability

Monitoring tells you that something has failed. Observability tells you why, and what that means for the business. As AI workloads become more distributed across cloud, edge and on-premises environments, that context becomes far more valuable than a flood of isolated alerts. Your organization will increasingly rely on unified observability to understand the health of your entire AI environment and make faster, better operational decisions.

2. From manual operations to automated operations

Rather than scripting repetitive tasks, automation is now about embedding operational knowledge into software so routine decisions happen automatically and consistently. In complex AI environments, automating operational workflows rather than individual tasks will improve productivity and help you identify opportunities for greater cost efficiencies.

3. From technology silos to platform engineering

As we’ve established, AI doesn’t distinguish between cloud, networking and security, and neither should your operating model. Instead of managing each domain separately, adopt platform engineering to provide shared services, consistent governance and reusable infrastructure for your developers and operations teams.

4. From reactive support to reliability engineering

Traditional IT responds after incidents have occurred. Reliability engineering starts from the assumption that failures are inevitable, then designs systems to minimize their impact before customers or employees notice them. Once AI is embedded in your everyday business processes, success will increasingly depend on preventing disruption rather than simply recovering from it quickly.

5. From infrastructure management to infrastructure economics

Perhaps the biggest shift is how infrastructure is measured. Instead of focusing primarily on cost, use and uptime, start evaluating your infrastructure by the business outcomes it enables: AI productivity, customer experience, operational resilience and revenue growth. In this way, your infrastructure becomes a strategic investment in enterprise performance.

The kinds of operational shifts I describe above are already helping organizations improve resilience and performance. When a global truck manufacturer found themselves battling recurring issues across a complex multivendor network, the consequences rippled through operations and drove up support costs. With NTT DATA’s help, the company adopted a platform-led approach to unify and manage their network infrastructure. Now, with end-to-end visibility, continuous monitoring, predictive analytics and outcome-based services, they have improved reliability, resolve issues faster and keep factories and logistics running more smoothly.

• ALSO READ → Cloud is no longer infrastructure — it’s the execution layer of AI

Questions to pose at the next board meeting

Organizations will typically move from manually managed environments to automated, observable and reliability-focused operations before eventually allowing AI to optimize parts of the infrastructure itself. Few are at that final stage today. What matters is understanding where you are, where you need to go next and how each step strengthens the operational foundation for AI.

As you plan that journey, the conversation needs to move into the boardroom. At the next board meeting, these are the questions worth asking:

• Is our infrastructure designed for continuous AI workloads?
• Can our operating model scale faster than our AI adoption?
• Are we measuring reliability in business outcomes rather than technical metrics?
• Have we automated enough of our operations to support enterprise-scale intelligence?
• Does leadership have unified visibility of cloud, network and security?

Ask a different first question

Powerful AI models are becoming available to everyone, so your competitive advantage will come less from the model you choose and more from how quickly, securely and reliably you can put it to work.

The next time your organization starts an AI strategy discussion, don’t begin by asking which model to deploy. Instead, whether your infrastructure and operating model are ready to support it.