Data quality Services

Six service agents do the wider jobs.

Rule agents check one thing each. The service agents do the jobs that span a source or a domain: profiling what the data looks like, finding duplicates, verifying values against the world, proposing and applying corrections, watching the sources for anomalies, and explaining a finding to whoever asks.

Profileshape of the data01profiler agentCheckall applicable rules02rule agentsProposefix with evidence03corrector agentDecideauto, or a person04guardrails · stewardWrite backto the source05connector · ackLearnfrom every decision06model · per actionTHE LOOP, ON EVERY RECORD OF EVERY DOMAINeach accepted or rejected fix tunes the next proposalGolden record: in flow until every applicable rule passes, then promoted; quarantined the moment one fails laterin flow · golden · quarantined — the state is visible at every door: grid, API, SQL, write-backRules are the smallest agentswhat · where · when, in a sentenceAutonomy is earned per action typeL0 observe → L3 act within limitsEvery step lands in the audit trailwho or what, why, before, after, ack
Where the services sit on the loop: the profiler measures, the rule agents check, the corrector proposes and applies, the monitor watches, and every decision feeds the model.
6
service agents
product
1
register for rules and services alike
product
14,200
corrections in a month on the showcase estate, 96% accepted
showcase
28 s
to trace a finding to its root cause through lineage, showcase
showcase
The six services

What each does, what it produces, what it may do alone

The level shown is the usual starting point; every level is per action type and is earned upward.

ServiceWhat it doesWhat it producesUsual level
ProfilerReads a source and learns its shape: fill rates, formats, distributions, value lists, key candidates, drift over timeColumn scores per dimension, cross-source comparison, suggested rules for the weakest fieldsL1: proposes rules; writes nothing
Duplicate finderRuns the domain's match rules on new and changed records, and periodically over the whole domainPairs and clusters with a confidence band and the evidence that joined them, queued for the stewardL1 to L3: merges alone only above a band the owner sets
VerifierChecks tax ids, registrations, addresses, bank details and sanctions lists against outside servicesA verification result with its reference, date and evidence, and a correction proposal where the service returns a better valueL2: applies confirmed corrections after approval; two-person rule on bank details
CorrectorTurns findings into proposals: standardisations, list mappings, derived values, values taken from the winning sourceTickets with the before, the after, the basis and a confidence; bulk proposals for identical fixesL1 to L3 per action type: standardisations climb fastest
MonitorWatches volume, freshness, structure and value distributions on every source, with learned patternsAnomaly tickets with the root cause traced through lineage; pauses on a feed; notices to consumersL3 for pausing a feed; L1 for everything that changes data
ExplainerAnswers 'why is this record quarantined' and 'what changed here' in plain words for the grid, the API and the assistantAn explanation with the rule, the evidence and the ledger entries behind itL0: reads and reports
How a service is different from a rule

Same blocks, wider scope, more tools

A rule agent has one directive and a narrow scope. A service agent has several directives, a scope that may span a source, and tool grants that let it call the match engine, a verification service or a write interface. Both are described by the same eight blocks, live in the same register and climb the same ladder, which is why a steward who has managed rules can manage services.

Services are also where the models live. The corrector's confidence comes from a model trained on the estate's own accepted and rejected fixes; the monitor's expected ranges come from a model of the source's own rhythm; the duplicate finder's bands come from the match engine's scoring. Each model is an implementation detail behind a directive, and each is evaluated on golden cases like any rule.

Services are shipped with the platform and configured per domain; they are the reason a first week on a new domain is a week. The profiler runs on Monday with no rules written, and by Monday evening the owner can see which fields cost the most.

Where a programme needs an agent beyond the shipped six, the same eight blocks describe it: a mission, a scope, directives in sentences, tool grants, a trigger, guards, an escalation route and an eval. Definitions can be co-drafted with an outside model, with metadata crossing the boundary and row data staying inside.

A day on the showcase estate

What the services did between 02:00 and 09:00

Illustrative, from the showcase supplier and material domains; figures labelled showcase.

02:00
Rule agents run27 material rules and 19 supplier rules over 61,004 and 9,340 records. 1,412 findings.
02:14
Monitor pauses a feedSAP change capture dropped 95%; the feed was paused, the engineer paged, and consumers of the affected views told. Recovered 02:58; backlog replayed.
03:10
Corrector proposes1,204 country standardisations at 0.99 confidence applied at L3 within budget; 208 planner assignments at 0.71 queued for the planning centre.
06:30
Verifier runs the aged records312 bank details older than twelve months re-verified; 9 mismatches held for treasury under the two-person rule.
07:00
Duplicate finder38 new supplier records matched; 3 pairs above the auto-merge band merged with their evidence, 11 queued.
08:30
Steward's queue219 tickets sorted by confidence and value at stake, with the explainer's one-line reason on each.
Questions

Can the services run against the source systems directly?

Yes, through the same read-in-place connectors the rest of the platform uses, with each source's own permissions applied. Writes go through the released interfaces and are acknowledged.

How much of this is automatic on day one?

The profiler, the monitor's observation and the explainer run from the first day at L0 or L1. Anything that changes data starts at L1 and climbs through the gate described on the autonomy page.

Where are the outside verification services?

Configured as tools with a mode, per agent: tax authorities, company registries, postal services, bank verification and sanctions lists. The verification page lists them.

See quality run on your own domain

Bring one domain and two systems. In a week the sources are profiled, the ten costliest fields are under rules, the first corrections are written back with their acknowledgements and the publish gate is on.