Data quality Enforcement

A record is golden only while every applicable rule passes.

Enforcement is what makes quality a property of the golden record itself. A record is in flow while checks run and corrections are pending, golden once every applicable rule passes, and quarantined the moment a rule fails later.

THREE STATES, TWO GATESIn flowchecks running · corrections pendingGoldenevery applicable rule passedQuarantineda rule failed after promotiongate 1rules passgate 2check failscorrection applied and re-checked → back through gate 1WHAT EACH DOOR DOES, BY STATEdoorin flowgoldenquarantinedgridbadge: pending · edit allowedservedbadge: quarantined · edit allowedheadless APIreturns the record flaggedreturns the recordreturns the last golden version, flaggedSQL and pivotsvisible in a separate viewvisibleexcluded from totals by defaultwrite-backheldrunsheld until corrected
Three states and two gates, and what the grid, the headless API, SQL and the write-back do in each state.
3
states of a record
product
2
gates: promotion and quarantine
product
4
doors that read the state: grid, API, SQL, write-back
product
1
contract per consumer per view, with thresholds per dimension
product
The three states

What each means, and how a record moves

in flow
Checks running, corrections pendingA new or changed record with applicable rules still to run, or with an open ticket. Served with a flag; excluded from totals by default; write-back held.
golden
Every applicable rule passedThe certified version of the truth. Served at every door; write-back runs; contracts count it.
quarantined
A rule failed after promotionA later check, a changed rule, an expired verification or a contract breach failed the record. Served as the last golden version with a flag; write-back held until corrected; consumers of the record are told.

A quarantined record returns to in flow when a correction is applied, and passes gate one again when its rules pass. The state is data on the record, so a question 'as of 31 March' shows the state the record had on that date, and the promotion and quarantine events are entries in the audit trail.

The gate

What 'every applicable rule passes' means in practice

The applicable setThe rules that apply to a record follow its type and status; excused rules are listed on the record with the excusal's owner.

The gate is strict on purpose: a golden record is a promise to every consumer, and the promise is only worth making if it holds. Weighted fields keep the all-pass rule; a domain owner can relax the gate for low-weight groups with a threshold, and the relaxation is a facet with a history.

Because the state is enforced at the platform, every door behaves the same way. A satellite application calling the headless API receives a failed record only with its state and its reason; a pivot excludes quarantined rows from totals unless asked to include them; a write-back waits. The consumer's own code needs no quality logic.

Contracts with consumers

What a consuming system relies on, and what happens on a breach

A contract states thresholds per dimension for one view, signed by the consumer and the owner.

Planning ← golden materialsmetcompleteness of planning attributes ≥ 98% · freshness < 6 h · actual 99.1% · 3 h 12 min
Finance ← golden customersmetuniqueness 100% · accuracy of tax ids ≥ 99% · actual 100% · 99.4%
Treasury ← supplier bank detailsbreached · feed heldtimeliness: verified < 12 months for 100% of active suppliers · actual 96.7% (312 aged)
Sales operations ← golden customers · contact groupdraftcompleteness ≥ 85% per field · phone at 38% · scope exclusion under review

A breach does three things: it notifies the consumer with the records and the dimension, it holds the feed or the write-back if the contract says so, and it opens a ticket for the owner with the root cause. The treasury contract above held the bank-details feed on 312 aged verifications; the verifier re-ran them the same morning.

What the consumer sees

Four doors, one state

DoorIn flowGoldenQuarantined
GridBadge 'pending'; editable; the open ticket one click awayServedBadge 'quarantined' with the failing rule; editable
Headless APIReturns the record with a status field and the pending checksReturns the recordReturns the last golden version with status 'quarantined' and the reason
SQL and pivotsIn a separate view; excluded from totals by defaultVisibleExcluded from totals by default; a flag column names the rule
Write-backHeldRuns, acknowledgedHeld until the correction passes the gate
Assistant and agentsAnswers name the state before the valueAnswers as factAnswers with the last golden value and the reason it is held
Terms on this page

In flowA record whose checks are running or whose corrections are pending.
GoldenA record that passed every applicable rule and was promoted; the certified version of the truth.
QuarantinedA promoted record that failed a later check, a changed rule, an expired verification or a contract.
Publish gateThe condition for promotion: every applicable rule passes, or the domain's threshold for low-weight groups.
Quality contractThresholds per dimension for one view, agreed between a consumer and the owner, with the action on breach.
Questions

Does a strict gate mean fewer golden records?

At first. On the showcase supplier domain 312 of 9,340 records were held on day one, most by the bank-details timeliness rule; after the verifier's first run 41 remained. The number of held records is the work list, which is what the gate is for.

Can a consumer read in-flow records?

Yes, with the status field, through a view that says so. A planning feed that wants only golden reads the golden view; an analyst who wants everything reads the full view with the state as a column.

Who signs a contract?

The owner of the domain and the owner of the consuming system, on the contract's page; the platform monitors it and reports breaches to both.

What about records with a rule they will always fail?

A rule that a class of records will always fail becomes an excusal, owned and reasoned, and the excusal is visible on the scorecard. A record with no basis for a fix goes to the steward as it is.

See quality run on your own domain

Bring one domain and two systems. In a week the sources are profiled, the ten costliest fields are under rules, the first corrections are written back with their acknowledgements and the publish gate is on.