In 1924, David Hilbert described a thought experiment: a hotel with infinitely many rooms, all occupied. A new guest arrives. No problem. Move every guest from room n to room n+1. Room 1 opens up. The hotel was full before and it's full now, but it has one more guest.^[1]

Cloud computing marketing works on roughly the same premise. Need more capacity? Just add instances. Storage full? Provision more. Traffic spike? Auto-scale. The infrastructure stretches to accommodate whatever you throw at it. Always room for one more.

The difference is that Hilbert's Hotel is a mathematical abstraction. The cloud is a building full of servers that need electricity.

The Fine Print of "Unlimited"

Every major cloud provider advertises elastic, on-demand scalability. And to be fair, the abstraction is impressive. You can spin up a thousand virtual machines in minutes. You can store petabytes without buying a single hard drive. For most workloads, most of the time, the cloud genuinely feels infinite.

But it isn't. And the moments when that becomes obvious tend to be expensive.

Cloud services have limits. They're documented, but rarely advertised. DynamoDB has account-level throughput caps. Lambda has concurrency limits per region. S3 has request rate thresholds per prefix. API Gateway has throttling defaults. These aren't bugs; they're the finite walls of a building marketed as Hilbert's Hotel.^[2]

The limits exist because the underlying infrastructure is physical. Data centers have a fixed number of racks, a fixed power budget, a fixed cooling capacity. When a region runs out of a particular instance type, you get capacity errors. When enough customers scale simultaneously, everyone hits the ceiling together.

The Thundering Herd

The most revealing moments come during crises, when everyone needs to scale at once.

In early 2020, the pandemic sent the world online simultaneously. Video conferencing platforms buckled. Grocery delivery apps crashed. Government unemployment sites collapsed under load. Cloud providers themselves experienced degraded performance as customers across every industry tried to scale at the same time.^[3]

This is the thundering herd problem applied to infrastructure. Hilbert's Hotel can accommodate infinite new guests because the thought experiment doesn't account for the time it takes to move everyone, or the hallways getting jammed, or the elevator breaking. Real infrastructure has all of those constraints.

Major cloud outages reinforce the point. When a large provider's region goes down, it doesn't just affect one customer. It cascades. In December 2021, an outage in a single provider's US-East-1 region disrupted services across thousands of companies, from streaming platforms to smart home devices to delivery apps.^[4] The "infinite" hotel turned out to have a single point of failure in its lobby.

The Cost Curve

Even when the cloud can scale, the economics often can't.

Cloud pricing is designed to feel cheap at small scale. The free tier is generous. The first few thousand requests cost fractions of a cent. But the cost curve is rarely linear. As you scale, you hit pricing tiers, data transfer charges, cross-region fees, and support costs that compound in ways the initial estimate didn't capture.

Startups regularly discover this the hard way. A prototype that cost $50 a month in development becomes a $50,000 monthly bill in production. The infrastructure scaled beautifully. The budget didn't.^[5]

There's a Gabriel's Horn quality to cloud economics. Gabriel's Horn is a mathematical shape with finite volume but infinite surface area: you can fill it with paint, but you can never paint its surface.^[6] Cloud infrastructure can deliver finite value (your application works) while generating infinite surface area in costs (monitoring, security, networking, compliance, support). The value converges. The bill diverges.

The Serverless Illusion

"Serverless" computing pushes the abstraction further. You don't manage servers. You don't think about capacity. You write functions and the platform runs them. It's Hilbert's Hotel where you don't even see the rooms.

But the servers are still there. They're just hidden behind another layer of abstraction. And that abstraction has its own limits: cold start latency, execution time caps, memory ceilings, payload size restrictions. The Plato's Cave series on this blog explored how abstraction layers hide reality. Serverless is a particularly elegant cave. The shadows on the wall look like infinite capacity. Behind the wall, there are still racks and cables and power supplies.

The CAP theorem formalizes a related constraint: in a distributed system, you can have at most two of consistency, availability, and partition tolerance.^[7] No amount of scaling changes this. It's a mathematical limit, like the halting problem. You can throw infinite hardware at a distributed system and still not get all three properties simultaneously.

Finite Hardware, Honest Engineering

None of this means the cloud is bad. It's genuinely transformative. The ability to provision infrastructure on demand, without capital expenditure, has enabled an entire generation of companies that couldn't have existed otherwise.

But the marketing promise of infinite scalability deserves the same skepticism you'd apply to any claim of infinity. The cloud is not Hilbert's Hotel. It's a very large, very well-managed, very expensive hotel with a finite number of rooms, a finite power supply, and a front desk that occasionally crashes.

Honest engineering starts with understanding the limits of your infrastructure, not just its capabilities. Document the service quotas. Model the cost curve at 10x and 100x your current load. Plan for the outage that the provider says won't happen. Test what happens when auto-scaling hits the ceiling.

Hilbert's Hotel is a beautiful thought experiment about the strange properties of infinity. The cloud is a beautiful engineering achievement about the impressive properties of very large finite systems. Confusing the two is where the trouble starts.

References

[1] David Hilbert, "On the Infinite," translated by Erna Putnam and Gerald J. Massey, in Paul Benacerraf and Hilary Putnam (eds.), Philosophy of Mathematics: Selected Readings, 2nd ed., Cambridge University Press, 1983. https://www.cambridge.org/core/books/philosophy-of-mathematics/on-the-infinite/B91A325262EF69CA314F13B1B9CD98C2

[2] Amazon Web Services, "AWS Service Quotas," AWS Documentation. https://docs.aws.amazon.com/general/latest/gr/aws-service-information.html

[3] Thomas Claburn, "Network sniffers find COVID-19 did not break the internet – though it was behind a massive jump in outages," The Register, August 5, 2020. https://www.theregister.com/2020/08/05/covid19_internet_impact_thousandeyes

[4] Renato Losio, "AWS US-EAST-1 Outage: Postmortem and Lessons Learned," InfoQ, December 18, 2021. https://www.infoq.com/news/2021/12/aws-outage-postmortem

[5] Mahdi Hazaveh, "AWS Billing Nightmares: Real Stories of Cloud Bills Gone Horribly Wrong," TechWithMahdi, 2025. https://techwithmahdi.com/aws-billing-nightmares-real-stories/

[6] Evangelista Torricelli, "De solido hyperbolico acuto" (On the Acute Hyperbolic Solid), 1643. Discussion and context available at: https://imaginaryinstruments.org/torricellis-trumpet-or-gabriels-horn/

[7] Eric Brewer, "CAP Twelve Years Later: How the 'Rules' Have Changed," IEEE Computer, Vol. 45, No. 2, February 2012. https://www.infoq.com/articles/cap-twelve-years-later-how-the-rules-have-changed/