Why Your Cloud Data Platform Bill Doubled

Your cloud data platform bill doubled and nobody can fully explain why. The platform team says usage grew. Finance wants a forecast. Someone suggests moving back on-premises, which nobody actually wants.

This conversation happens at almost every organisation about eighteen months after migration, and it's rarely because the platform was the wrong choice. It's because cloud data platforms make consumption invisible until it's a bill — and a handful of specific patterns account for most of the surprise.

Why the bill surprises you

On-premises, capacity was a fixed cost decided annually. Bad queries were free at the margin; they just ran slowly and someone grumbled.

In the cloud, every inefficiency has a price and the meter never stops. A badly-written query that used to be a performance annoyance is now a line item. And because compute is elastic, nothing forces the conversation — it just scales to accommodate the waste and bills you.

Worse, the cost is disconnected from the person causing it. An analyst runs an unfiltered query against five years of claims history; the bill arrives at the platform team a month later with no attribution.

Where the money actually goes

1. Idle and oversized compute. Warehouses left running with no queries, or sized for a peak that happens twice a month. Auto-suspend set to sixty minutes instead of two. This is usually the single biggest and easiest win.

2. Full scans that should be pruned. Tables without sensible partitioning or clustering, so every query reads everything. In insurance this bites hard — claims and policy history tables are enormous and most queries want a narrow date or product slice.

3. Dashboards on auto-refresh. A dashboard refreshing every fifteen minutes against a heavy query, viewed by three people, twice a week. Multiply across an organisation. Nobody chose this; it was a default.

4. Pipelines that reprocess everything. Full reloads where incremental would do. This is often historical — the pipeline was written when the table was small and nobody revisited it as it grew.

5. Duplicated data. Four teams each maintaining their own copy of policy data because nobody trusted the shared one. You're paying four times for storage and compute, and getting four inconsistent answers. This is a data-trust problem showing up as a cloud bill.

6. Dev and test at production scale. Full copies of production for every environment, running continuously.

What actually controls it

Attribute cost to teams. The highest-leverage change, and it's organisational rather than technical. Tag warehouses and workloads by team and report spend back to them. Behaviour changes fast when a team sees their own number — and doesn't change at all while cost is a shared abstraction.

Set aggressive auto-suspend and right-size. Suspend after a minute or two of idle. Size for the common case, not the peak; let the peak be a little slower or use a separate larger warehouse for it.

Separate workloads. Don't let a heavy data-science job and the finance team's month-end share compute. Separation gives both predictability and attribution.

Partition and cluster on how data is actually queried. Usually date plus one dimension. This is often a single afternoon's work that halves scan volume permanently.

Make dashboards pull, not push. Refresh on view or on a sane schedule. Ask what decision changes if it's fifteen minutes fresher — usually none.

Fix the duplication by fixing the trust. Teams copy data because they don't trust the shared version. Address the quality and definitions and the copies disappear on their own — a cost saving that arrives as a side-effect of doing the right thing anyway.

Budget alerts, not bill reviews. Alert when a workload exceeds its expected spend, in near-real time. A monthly review is an autopsy.

The trap: cutting cost by cutting capability

Worth flagging, because the pendulum swings hard. Once cost becomes a board topic, the instinct is to restrict access, throttle queries, and reduce data retention.

That saves money and destroys the value you migrated for. The point of the platform was making data available; locking it down to control spend means paying cloud prices for on-premises behaviour.

The better frame is efficiency, not austerity: the same work for less, through right-sizing, pruning, and eliminating duplication — not less work. Almost every organisation has enough pure waste that they never need to touch legitimate usage.

The point

Cloud data platform costs aren't inherently out of control. They're unattributed, which means nobody owns them, which means nobody optimises them.

Attribute the spend, right-size the compute, prune the scans, and remove the duplication that exists because people don't trust each other's tables. Most organisations find 30–40% without touching a single legitimate use case.

And notice the recurring theme: the biggest saving usually comes from fixing data trust — the same fix that unblocks everything else.

We design and optimise cloud data platforms — Snowflake, Databricks, and the modern lakehouse — for cost as well as capability. More at IntelliBooks.