Data Contracts: Ending the "Someone Changed a Field" Outage
Here's a failure everyone in data engineering recognises. A dashboard breaks. Or worse, it doesn't break — it just starts showing numbers that are subtly wrong. Days later someone traces it to an upstream team who renamed a field, changed a code, or dropped a column in a release nobody outside their team read. Nobody did anything wrong, exactly. The upstream team owned that system and changed it. They had no idea a pricing model depended on that field, because nobody ever told them. That's the problem data contracts exist to solve, and it's the reason mature data platforms stay reliable while immature ones lurch from breakage to breakage. What a data contract is A data contract is an explicit, versioned agreement between a data producer and its consumers about what will be provided and what guarantees come with it. Concretely: Schema — fields, types, nullability Semantics — what each field actually means ("policy_status = ACTIVE means bound, in-for...