The cloud-first consensus is quietly reversing
For most of the past decade, enterprise IT had one default answer: put everything in the public cloud. AI is rewriting that. As models move from pilots to mission-critical infrastructure, the limits of a cloud-only approach - latency, data sovereignty, regulatory compliance, and cost - are pushing companies to bring AI workloads back behind their own walls. Purpose-built private infrastructure for training and inference is becoming a central pillar of enterprise strategy rather than a niche concern.
The numbers behind the shift
The spending signals are hard to ignore:
- IDC reported enterprise compute and storage hardware for AI grew 166% year-on-year in Q2 2025, while Gartner pegged 2025 AI spending at US$1.5tn, with data-center systems up nearly 47%.
- The GPU server market, worth US$171bn in 2025, is forecast to hit US$730bn by 2030.
- For firms in regulated industries or under data-residency laws, the cloud isn't just costly - it can be a legal risk, with confidentiality obligations sometimes requiring on-premise deployment outright.
What changed on the supply side is that the hardware caught up: liquid-cooled GPU servers built on NVIDIA's Blackwell architecture, available through Dell, HPE, and Lenovo, now deliver petaflop-scale inference in racks a company can own and secure itself. Most organizations are landing on a hybrid model - public cloud for elastic, non-sensitive work; private data centers for inference and fine-tuning; edge for latency-critical tasks.
What it means for the data center
Bringing AI in-house is not just racking more servers. Densities can reach 100 kilowatts per rack, which makes traditional air cooling inadequate and turns power resilience, grid connectivity, and thermal management into strategic concerns - the data center becomes, in effect, an AI factory.
Who's furthest ahead
The piece profiles three very different adopters. Goldman Sachs has built a private agentic stack and became the first major bank to roll out Cognition's autonomous engineer Devin across its 12,000 developers, reporting three-to-four-times productivity gains in software lifecycle work - funded partly by capital redirected from its retreat from consumer banking. Siemens pushes AI onto the factory floor via its Industrial Edge platform and is building modular, lower-carbon data-center units. And NTT DATA runs agentic AI inside its Cyber Defense Centers to protect private infrastructure, cutting alert volumes by up to 90%. The throughline: on-premise AI is now as much an engineering and security discipline as a software one.