Data Contracts: The Less Glamorous Side of Reliable ML
A model expects a field called amount. The upstream service continues sending a number, so the schema check passes. But the value now represents a different currency or a different aggregation period. Nothing crashed; the meaning changed.
That is the kind of boundary I would want a data contract to make visible. A contract is more than a list of columns. It describes what the data means, when it is available, who owns it, and what happens when the agreement stops holding.
Start with semantics
For each important field, specify units, allowed values, null meaning, and the entity being described. Does a missing quantity mean zero, unknown, not applicable, or not yet reported? Those meanings should not collapse into one default without a deliberate decision.
Keep the contract narrow enough to maintain. Focus first on fields whose change could materially affect the task. A document listing every possible property but owned by nobody can become less useful than a small enforced agreement.
Make time part of the schema
Distinguish when an event occurred, when it was recorded, and when it became available to the forecasting or decision system. Those timestamps can differ. Historical evaluation needs to respect the information that was actually available at the prediction cutoff.
For an illustrative shipment dataset, observed_at might describe the event and available_at its arrival in the analytical system. The contract should also specify timezone, late-arrival handling, and whether corrected records replace or supplement earlier versions.
Define validation and failure behavior together
Type and range checks catch some issues. Freshness, completeness, and category checks catch others. Distribution checks can reveal a change worth investigating, but a changed distribution is not automatically invalid data.
Each consequential check needs an action. A critical unit mismatch might quarantine a batch. A newly observed category might trigger review while allowing a documented fallback. A late optional feature might permit a degraded mode. Avoid a generic “log warning and continue” policy for every failure.
Version changes across the boundary
Agree how producers announce changes, how consumers test compatibility, and how long old and new formats coexist. Test candidate transformations against representative historical and current data before changing production behavior.
A version label alone does not establish compatibility. A renamed field may be easy to migrate; an unchanged field with a new business definition may not be. Include semantic changes in the review, not only schema diffs.
Check training and serving separately
A feature can be calculated correctly in a notebook and differently in the live service. Compare definitions, timing, missing-value handling, and access to source data across both paths. Do not let a convenient training-time join introduce information that serving will never have.
Tools such as TensorFlow Data Validation can help detect schema anomalies, skew, and drift. They provide checks, not ownership. Someone still needs to decide which changes are expected, which are harmful, and what response is appropriate.
A compact agreement
- Producer and consumer owners, with an escalation route.
- Field meaning, units, entity keys, and time semantics.
- Freshness and completeness expectations.
- Validation rules and explicit failure modes.
- Versioning, migration, and retention expectations.
The contract pays off when a change occurs, not when the document is first written. Exercise it during an integration change and an incident. If it helps the team identify responsibility and restore a meaningful input, it is doing useful work.
Continue with monitoring the boundary and avoiding time leakage in forecasting.