The Data Center Never Went Anywhere
Every few months somebody writes about moving their infrastructure out of the public cloud, Hacker News loses its mind, and we have the same argument we’ve been having for the last twenty years. One group thinks the cloud was a giant waste of money, the other thinks anybody running their own hardware in 2026 must enjoy getting paged at 3 a.m.
Meanwhile, Amazon, Microsoft, Google, and Meta are spending hundreds of billions of dollars building data centers. AI companies are buying GPUs faster than manufacturers can produce them, and businesses that spent the last decade moving everything into the cloud are suddenly looking at colocation again.
I’ve spent most of my career in telecom and collaboration, which means I’ve been around plenty of infrastructure decisions that looked great on paper until somebody had to support them. I’ve watched customers move from servers in closets to colocation facilities, from on-premises phone systems to hosted collaboration platforms, and eventually into the public cloud. Now we’re having conversations about bringing workloads back, buying GPU infrastructure, and figuring out whether it makes more sense to rent computing capacity or own it.
What’s interesting is that none of this is particularly new. We’ve been moving computing resources back and forth between centralized and distributed infrastructure since the 1960s. The technology changes, the costs change, and we come up with new terminology for it, but a lot of the decisions are surprisingly familiar.
We Were Renting Computers Before AWS Existed
In 1961, John McCarthy suggested that computing would eventually become a public utility. Instead of purchasing expensive equipment, businesses could pay for access to computing resources the same way they paid for electricity or telephone service.
IBM introduced the System/360 in 1964, and the pricing gives you a pretty good idea why McCarthy thought this would work. A basic system rented for around $2,700 per month, while larger configurations could cost $115,000 per month. Purchasing one outright could run as high as $5.5 million. IBM generally rented these systems rather than selling them, and standard contracts covered a forty-hour workweek. If you needed additional computing time, you paid for it.
Companies like Tymshare took the concept further by selling access to centralized computers. You connected through a terminal, submitted your work, and paid for the computing time you consumed. There were limitations, obviously. We’re talking about 300 and 1200 baud connections, teletype terminals, and computing resources that were shared across multiple customers. You weren’t exactly spinning up a development environment and deploying an application before lunch. But if you needed a few hours of computing each week, renting made a lot more sense than purchasing a mainframe.
Then Digital Equipment introduced the PDP-8 in 1965 for $18,500, and suddenly departments could afford computers of their own. They didn’t need to schedule time on the company’s mainframe or wait for somebody in the data processing department to run a job. They could walk over to their own machine and use it. It wasn’t as powerful as a mainframe, but having access to a less powerful computer whenever you wanted turned out to be pretty useful.
This is essentially the first version of what we’re talking about today. Businesses were willing to purchase smaller systems because the cost of owning them had become reasonable, and the additional control was worth more than access to a larger shared machine.
Then We Put a Server in Every Closet
The personal computer pushed things further, and by the 1990s client-server computing was everywhere. PCs were on desks, applications ran on servers, and just about every business had some combination of file servers, database servers, email servers, and whatever else the IT department needed to keep running.
The problem was that we weren’t particularly good at sharing physical hardware between applications. Your email server was a physical machine. Your database was another physical machine. Your application server was another. Add a test environment, a backup system, and a few more applications, and suddenly you’ve got a room full of servers that spend most of their lives doing practically nothing.
A 2008 report from McKinsey and the Uptime Institute found that utilization in enterprise data centers rarely exceeded 6%. Other estimates from the period put average utilization somewhere between 5% and 18%.
Think about that. You purchase a server, pay to power it, cool it, maintain it, license the operating system, and eventually replace it, while using a tiny fraction of its computing capacity. Sounds ridiculous now, but there were legitimate reasons for doing it. Dedicated hardware was predictable, software isolation was difficult, and overprovisioning was usually cheaper than explaining an outage to management.
VMware helped change that when it introduced ESX Server in 2001. Virtualization allowed multiple operating systems to share the same physical machine, and companies started consolidating ten, fifteen, or twenty servers onto a single host. You could finally use the equipment you’d already paid for without giving every application its own box.
That also laid much of the foundation for the modern public cloud. If you can safely divide a physical computer into smaller virtual machines, there’s no reason those virtual machines have to belong to the same company.
Amazon Wasn’t the First Company to Try This
Before AWS, several companies had already tried selling computing infrastructure as a service. Exodus Communications was a major colocation and hosting provider during the dot-com boom. Loudcloud offered managed infrastructure before eventually becoming Opsware. HP, IBM, and Sun Microsystems all experimented with utility computing.
Sun announced its Grid service in 2005 at roughly $1 per CPU-hour. That wasn’t an outrageous price, especially compared with what computing infrastructure cost only a few years earlier.
The problem was getting access to it.
A lot of these services still involved contracts, procurement processes, sales teams, minimum commitments, or job submission models better suited to scientific computing than general-purpose applications. They were trying to sell computing like traditional enterprise infrastructure, and that’s a pretty terrible way to encourage somebody to experiment with a new idea.
Amazon launched S3 in March 2006 and EC2 in August of the same year. The original EC2 instance cost ten cents per hour. You could create an account, request a server through an API, run whatever you needed, and shut it down when you were finished. No hardware purchase. No waiting on procurement. No capacity planning meeting involving twelve people and a spreadsheet nobody wanted to maintain.
That’s what made AWS different. The idea of renting computing power wasn’t revolutionary, but making it available on demand to practically anybody with a credit card was.
And it worked. Businesses could launch applications without purchasing infrastructure, expand into new markets without building data centers, and scale their environments as demand increased. Managed services made things even more attractive because now you could avoid operating the databases, queues, and other infrastructure your applications depended on.
Netflix started its migration to AWS following a major database failure in 2008 and finished moving its streaming infrastructure in early 2016. That’s roughly seven years, which is worth remembering the next time somebody describes a cloud migration as a quick infrastructure project.
Capital One made a similar bet, closing eight data centers and exiting its remaining domestic facility in 2020. Plenty of other companies followed. For organizations that didn’t want to operate data centers, the cloud offered a pretty compelling alternative.
But something interesting was happening at the same time. Google, Facebook, Microsoft, and other companies operating at enormous scale continued building their own infrastructure. They weren’t paying somebody else to host everything because, at their scale, owning the equipment made financial sense.
Apparently buying servers wasn’t such a bad idea after all.
Dropbox Figured Out the Math
Dropbox is probably the best-known example of a company moving a significant workload out of the public cloud.
The company originally relied heavily on Amazon S3 to store customer files, but as the business grew, so did the storage bill. In 2015, Dropbox began moving data onto infrastructure it operated itself through a project called Magic Pocket.
When Dropbox filed to go public in 2018, the financial results became public. Its infrastructure optimization reduced cost of revenue by $39.5 million in 2016 and another $35.1 million in 2017. That’s approximately $74.6 million over two years.
Those are real savings, but the important part is understanding why it worked.
Dropbox had an enormous amount of storage, relatively predictable growth, and an engineering team capable of building and operating the infrastructure. Customer files don’t disappear overnight, and storage demand is generally easier to forecast than an application that suddenly experiences a massive spike in traffic.
Dropbox was paying for the convenience of renting infrastructure even though a large portion of its workload didn’t need the flexibility that came with it. Once the company reached sufficient scale, buying equipment and operating it in colocation facilities became a better financial decision.
It also didn’t abandon AWS entirely. Some workloads stayed in the cloud because that’s where they made sense.
In 2021, Andreessen Horowitz published The Cost of Cloud, a Trillion Dollar Paradox, arguing that large software companies were leaving substantial amounts of money on the table by continuing to rent infrastructure they could operate themselves for less.
The piece got plenty of criticism, and some of it was deserved. Comparing a cloud bill against the purchase price of hardware without factoring in engineering, maintenance, facilities, depreciation, and the cost of operating everything yourself isn’t a particularly useful exercise.
Still, the underlying argument was reasonable. If you have a predictable workload running around the clock, at some point the cost of renting infrastructure may exceed the cost of owning and operating it.
The challenge is figuring out where that point is for your business.
And Then Cloud Bills Got Interesting
From 2009 through much of 2021, interest rates were extremely low and businesses were spending aggressively on growth. If public cloud allowed you to launch products faster, avoid upfront infrastructure purchases, and scale without worrying about capacity, paying a premium was often an easy decision.
That changed in 2022 and 2023 when interest rates climbed and companies started scrutinizing expenses. Cloud spending became an obvious target, particularly for businesses that had spent years accumulating services without paying much attention to what they were actually using.
AWS growth slowed during that period as customers began optimizing their environments. Most of them weren’t leaving the cloud. They were right-sizing instances, purchasing reserved capacity, shutting down unused resources, and cleaning up the mess they’d accumulated.
Some went further.
37signals, the company behind Basecamp and HEY, announced in October 2022 that it planned to move much of its infrastructure out of the public cloud. The company had been spending around $3.2 million annually and decided to purchase its own Dell servers and operate them in colocation facilities.
By 2024, it reported reducing annual infrastructure spending to roughly $1.3 million, saving around $2 million. Its remaining S3 storage eventually moved to owned Pure Storage equipment after an existing contract expired in 2025, pushing its projected five-year savings above $10 million.
Ahrefs made a similar argument in 2023, estimating that its own fleet of 850 servers was dramatically cheaper than equivalent AWS capacity. The company estimated savings approaching $400 million over three years, although the comparison relied on assumptions about AWS instance sizes and pricing that make the exact number debatable.
Both examples are worth paying attention to, but there’s a pretty important catch. These companies already had the technical expertise needed to operate their infrastructure. They weren’t hiring an entirely new IT department to manage servers they’d never touched before.
A company with three developers and an application that might triple in usage next month probably shouldn’t copy the infrastructure strategy of Dropbox or 37signals simply because somebody published an impressive savings figure.
And moving out of AWS doesn’t mean building your own data center. Dropbox, Ahrefs, and 37signals all used colocation facilities. They owned the hardware while somebody else handled the building, power, cooling, physical security, and connectivity.
That’s a distinction that tends to get lost whenever somebody announces they’re leaving the cloud.
VMware Managed to Make Owning Expensive Again
There’s another part of this that deserves attention because it’s a good example of how infrastructure economics can change without the underlying technology changing at all.
Broadcom completed its acquisition of VMware in November 2023 and started restructuring the product portfolio around subscriptions and bundled licensing. Customers subsequently reported substantial price increases, sometimes several times what they’d previously paid.
For organizations with large VMware environments, licensing suddenly became a significant factor in the decision to keep workloads on-premises. Some began evaluating alternatives like Proxmox, KVM, and OpenNebula, while others reconsidered their broader infrastructure strategy.
I find the irony pretty funny. VMware helped make owning infrastructure cheaper by allowing businesses to consolidate servers onto shared hardware. Two decades later, the cost of VMware itself became a reason some of those same businesses started looking elsewhere.
And this is why comparing infrastructure costs is more complicated than dividing the price of a server by the number of hours it runs. Hardware, software, support, licensing, power, cooling, staffing, and replacement cycles all matter. Change one of those costs substantially and the entire calculation can change.
AI Makes This a Much Bigger Conversation
Everything we’ve talked about so far applies to traditional enterprise computing, but AI has introduced an entirely different set of infrastructure requirements.
Training a frontier AI model requires enormous clusters of accelerators working together over extremely fast networks. We’re talking about investments measured in billions of dollars, and there are only a handful of organizations capable of operating at that scale.
Inference is a different problem. Once a model has been trained, somebody has to run it every time a user submits a prompt, an application requests an answer, or an AI agent performs a task. That means GPUs processing requests continuously, often alongside databases and other systems containing the information those models need.
Some of that demand is unpredictable. Some of it is remarkably consistent.
And that brings us right back to the same infrastructure decision Dropbox faced.
An Nvidia H100 might cost approximately $30,000 to purchase, depending on the configuration and market conditions. Renting equivalent capacity can range from around $1.50 per hour with specialized GPU providers to considerably more through hyperscaler on-demand offerings.
At $2.50 per hour, renting a GPU continuously costs about $21,900 per year before considering other infrastructure expenses. At $12 per hour, you’re spending more than $100,000 annually to rent capacity associated with a card that might cost $30,000 to purchase.
Obviously, a GPU doesn’t operate by itself. You need a server, networking, power, cooling, support, and people capable of maintaining everything. There’s also the possibility that Nvidia releases something substantially better before you’ve recovered your investment.
But you can see why organizations with predictable, round-the-clock inference workloads are starting to consider owning the equipment.
Now look at the other side. If you purchase a $30,000 GPU and only use it a few hours every day, you’ve tied up a lot of money in something that spends most of its time sitting idle. That’s basically the enterprise server problem from twenty years ago, except the hardware is considerably more expensive.
Utilization matters. It mattered when companies were sharing mainframes, it mattered when VMware started consolidating servers, and it matters when deciding whether to purchase AI infrastructure.
There’s also the issue of how quickly these accelerators lose value. Nvidia continues releasing new architectures, and newer GPUs can offer substantial improvements in performance and efficiency. A card that’s economically attractive today may be far less competitive a few years from now, even if it still works perfectly well.
A GPU purchase is ultimately a bet that you’ll use enough of its capacity, for long enough, to recover the investment before something better changes the equation.
The Bigger Problem Might Be Finding Somewhere to Plug It In
One of the more interesting developments in AI infrastructure is that purchasing the hardware isn’t necessarily the hardest part anymore.
Power is becoming a serious constraint.
Nvidia’s GB200 NVL72 is a liquid-cooled rack containing 72 GPUs, with power requirements around 120 kilowatts. Compare that with enterprise data centers designed around five or ten kilowatts per rack, and you can see why somebody can’t simply order a few of these systems and roll them into an existing server room.
Now multiply that by thousands of racks.
Microsoft, Amazon, Google, Meta, and other operators are investing enormous amounts of money in new data center capacity, and getting sufficient electrical power delivered to those facilities has become a major challenge. Grid interconnection timelines can stretch for years, particularly in markets already experiencing rapid data center growth.
In Northern Virginia, one of the largest data center markets in the world, available capacity has become extremely tight. Demand for power and suitable facilities is growing faster than infrastructure can be built to support it.
Microsoft has publicly discussed having computing hardware that couldn’t be deployed because the company lacked the necessary facility capacity. Think about how strange that is. One of the largest cloud providers in the world has customers willing to pay for computing resources and equipment available to serve them, but not enough places to install that equipment.
Companies are responding in some pretty extraordinary ways.
xAI built its Colossus cluster in a former Electrolux facility in Memphis and used temporary natural gas turbines while additional utility capacity was being arranged. Meta is investing in enormous AI campuses with dedicated power arrangements. Microsoft signed a long-term agreement supporting the planned restart of a nuclear generating unit at Three Mile Island.
We’re now at a point where companies that built their businesses around software and computing services are making major investments in electricity generation and transmission simply to keep expanding.
And the money involved is getting difficult to comprehend.
Amazon, Microsoft, Alphabet, and Meta are collectively planning hundreds of billions of dollars in capital spending during 2026. Specialized GPU cloud providers like CoreWeave are borrowing heavily to purchase equipment and develop facilities, while customers are entering long-term commitments to secure access to computing capacity.
Some of those arrangements involve GPUs being used as collateral for loans. Others involve customers making substantial financial commitments before the infrastructure is even available.
There’s nothing inherently wrong with financing infrastructure that way, but it introduces risks that aren’t always obvious when you’re looking at revenue growth or announced contracts.
These companies are making enormous bets on continued demand for AI computing. If demand grows as expected, the investments could make sense. If GPU prices fall faster than anticipated, new architectures make existing equipment less competitive, or demand doesn’t materialize on schedule, the financial picture could look very different.
We’ve seen versions of that story before, particularly during the dot-com infrastructure boom.
I’m not suggesting AI is another dot-com bubble. The demand for computing is very real. But real demand doesn’t automatically mean every infrastructure investment supporting it will produce a good return.
So Are Companies Actually Moving Back?
Yes, but probably not in the way some of the headlines suggest.
Industry surveys show that a large percentage of enterprise CIOs are considering or actively moving selected workloads out of public cloud environments. Barclays surveys have reported figures above 80%, while IDC research has identified similar interest in moving some computing or storage workloads back.
The key word is some.
An organization moving one large database out of AWS counts as repatriation. So does an organization moving thousands of virtual machines. Those are very different projects, and treating them as equivalent makes the trend look much larger than it necessarily is.
Public cloud isn’t going away. In fact, the hyperscalers are investing more heavily in their infrastructure than ever before. The market is growing, AI demand is increasing, and businesses still need the flexibility and services these platforms provide.
What’s changing is how organizations decide which workloads belong there.
For a startup launching a new application, renting computing capacity is probably still the obvious choice. For a business running a stable application with predictable demand, purchasing infrastructure might be worth evaluating. For a large enterprise with strict data requirements or continuous AI inference workloads, a combination of owned equipment and public cloud services may be the most practical approach.
And for plenty of organizations, paying somebody else to manage all of this is worth the additional cost.
Where I Think This Goes
I don’t think we’re heading toward a world where everybody abandons the public cloud and starts purchasing servers again. I also don’t think the idea that everything eventually belongs in the cloud holds up particularly well anymore.
What I expect is more of a hybrid approach, although I know we’ve been using that term for about fifteen years.
Companies will continue using public cloud for applications that need elasticity, managed services, and geographic flexibility. They’ll operate predictable workloads on owned equipment where the economics justify it. Some AI models will run locally, others will be consumed through APIs, and many organizations will use a combination depending on the application.
Colocation is likely to become an even more important part of that picture, particularly for businesses that want to own high-density computing infrastructure without building facilities themselves.
There are limits, though. Owning a few racks in a colocation facility is one thing. Finding five megawatts of available power to support a large AI deployment is something else entirely. At that point, you’re competing for resources with companies spending tens or hundreds of billions of dollars on infrastructure.
I also think we’re going to see a fair number of businesses purchase AI hardware they never fully utilize. Everyone wants to have an AI strategy, and buying equipment can feel like progress. But owning a rack full of expensive GPUs doesn’t mean you’ve built anything useful, and it’s a pretty expensive way to discover your actual workloads would have been cheaper to run through an API.
The same basic questions that applied to mainframes in the 1960s still apply today. How much capacity do you need? How often are you going to use it? What’s the complete cost of operating it? And what are you giving up by choosing one approach over another?
In 1964, renting a mainframe made sense because purchasing one was prohibitively expensive. By the 1970s, smaller computers became affordable enough that businesses started buying their own. In the 2000s, virtualization made existing infrastructure more efficient while AWS made renting infrastructure incredibly convenient. Now AI is creating new reasons to own hardware, along with some very expensive reasons not to.
I work with customers that have perfectly good reasons to maintain on-premises infrastructure, and others that are better served by moving nearly everything into hosted platforms. I’ve also seen plenty of technology decisions made because something was fashionable rather than because the business actually needed it.
Cloud computing isn’t inherently better than owning infrastructure, and owning infrastructure isn’t automatically cheaper than renting it.
The answer depends on what you’re running, what it costs, and who’s going to deal with it when something breaks.
After sixty years of moving computing resources around, you’d think we’d have figured that out by now.