Follow ZDNET: Add us as a favorite source On Google.
ZDNET Highlights
- The cloud-first approach needs to be reconsidered.
- AI contributes to increasing cloud costs.
- A hybrid model assures the best of both worlds.
About a decade ago, the debate raged between cloud and on-premises computing. Cloud easily won that battle, and it wasn’t even close. However, people are now reconsidering whether the cloud is still their best option for many situations.
Too: Cloud-native computing is set to explode thanks to AI inference work
Welcome to the age of AI, in which on-premises computing is starting to feel good again.
there is a movement going on
Recently, existing infrastructure configured with cloud services may not be ready for emerging AI demands Analysis Warned by Deloitte.
“Infrastructure built for cloud-first strategies cannot handle AI economics,” said the report, written by a team of Deloitte analysts led by Nicolas Merizzi.
Too: 5 Essential Cloud Tools for Small Businesses in 2025 (And My Top 10 Money-Saving Secrets)
“Processes designed for human workers don’t work for agents. Security models built for perimeter defense don’t protect against threats moving at machine speed. IT operating models built for service delivery don’t drive business transformation.”
According to Deloitte analysts, to meet AI needs, enterprises are moving from primarily cloud to considering a hybrid mix of cloud and on-premises. Technology decision-makers are taking second and third looks at on-premises options.
Too: Want real AI ROI for business? This could finally happen in 2026 – here’s why
As the Deloitte team describes it, “There is a movement underway from cloud-first to strategic hybrid – cloud for elasticity, on-premises for stability, and edge for immediacy.”
four issues
Deloitte analysts cited four burning issues that arise with cloud-based AI:
- Rising and Unexpected Cloud Costs: He believes the cost of AI tokens has dropped 280x in two years – yet “some enterprises are still running into the millions with monthly bills.” Excessive use of cloud-based AI services “can lead to frequent API hits and rising costs.” There’s even a tipping point at which on-premises deployment makes more sense. “This may occur when cloud costs begin to exceed 60% to 70% of the total cost of acquiring an equivalent on-premises system, making capital investment more attractive than operating expenses for projected AI workloads.”
- Latency issues with the cloud: AI often demands near-zero latency to take action. “Applications requiring response times of 10 milliseconds or less cannot afford the inherent latency of cloud-based processing,” the Deloitte authors explain.
- On-premises promises greater flexibility: Flexibility is also part of the key requirements for fully functional AI processes. These include “mission-critical functions that cannot be interrupted, requiring on-premises infrastructure in case the connection to the cloud is disrupted,” the analysts say.
- Data Sovereignty: Some enterprises “are scaling back their computing services, not wanting to be completely dependent on service providers outside their local jurisdiction.”
Too: Why are some companies retreating from the public cloud?
three tier approach
The Deloitte team said the best solution to the cloud vs. on-premises dilemma is to go with both. They recommend a three-tier approach, which includes the following:
- Cloud for elasticity: To handle variable training workloads, burst capacity requirements and experimentation.
- On-premise for continuity: Run production estimates at estimated costs for high-volume, sustained workloads.
- Edge for urgency: This means AI within edge devices, apps or systems that “handles time-critical decisions with minimal latency, especially for manufacturing and autonomous systems where split-second response times determine operational success or failure.”
This hybrid approach resonates as the best route for many enterprises. Milankumar Rana, who most recently worked as a software architect at FedEx Services, is strongly aligned with the cloud for AI, but sees the need to support both approaches where appropriate.
“I’ve built large-scale machine learning and analytics infrastructure, and I’ve seen that almost all functionality, such as data lakes, distributed pipelines, streaming analytics, and AI workloads based on GPUs and TPUs, can now run in the cloud,” he told ZDNET. “Because AWS, Azure, and GCP services are so mature, businesses can grow quickly without spending a lot of money.”
Too: How AI agents can eliminate waste in your business – and why it’s smarter than cutting costs
Rana also tells customers to “maintain some workloads on premises where data sovereignty, regulatory considerations, or very low latency make the cloud less useful,” he said. “The best way to do things right now is to use a hybrid strategy, where you keep sensitive or latency-sensitive applications on premises while using the cloud for flexibility and new ideas.”
Whether using cloud or on-premises systems, companies should always take direct responsibility for security and monitoring, Rana said. “Security and compliance are the responsibility of all individuals. Cloud platforms include strong security; but, you must ensure compliance with regulations for encryption, access, and monitoring.”
