A figure from someone else’s question set says little about your estate. Accuracy testing uses fifty questions your own people ask, each with the answer its owner agreed.
Your team supplies the questions and the expected answers; Juno supplies the run and the evidence.
The set starts with questions people already ask. A planner, a buyer, a director and a finance lead each give ten to fifteen from a normal week, in their own words, including the awkward ones.
Each question then gets an expected answer, agreed with the person who owns that number. For a total it is a value and a period. For a list it is the rows that belong in it. For a question that sits outside the registered data, the expected answer is that Juno says so.
Juno runs the whole set and records each answer with the terms it used, the systems it read, the time it took and the query. Every part of a run stays visible, so a failure can be traced to its cause.
The scorecard reports the three scores per question and groups them by kind of question. Your team reads it, fixes what is weak, usually a mapping or a missing word, and runs the same set again. The history of runs stays.
One number for correctness hides too much, so three are reported and the declined answers are counted apart.
The answer matches what the owner of the number agreed. Correct, declined and wrong are counted apart, because a question Juno declines costs far less than a confident wrong answer.
The answer arrives inside the time your team set, measured from the question to the result. The showcase estate uses ten seconds, with an average of 4.2 s.
The reader saw the rows their role allows, and a restricted column was named. A row that should stay hidden counts as a failure even when the number is right.
A published study in 2024 reported its result as 72% correct, 8% declined and 20% wrong. Juno reports the same three numbers, because a single accuracy figure hides the difference between a question answered wrongly and a question answered with “this sits outside the data I hold”.
One run, fifty questions, three scores, and a named cause behind every failure.
The scorecard groups the questions by kind, because the causes differ. A question about the status of an order reads one mapped object and scores highly from the first run. A root-cause question joins several objects, so a missing mapping shows up there first.
Each failed question carries its cause: a column that was mapped to the wrong attribute, a word the vocabulary was missing, a restricted column, or a source that answered slowly.
The three declined answers in the showcase run were questions about supplier contract terms, which sit outside the registered data. That is the answer the owners agreed, so those three count as correct behaviour and are reported apart.
The two wrong answers were both mapping faults on one custom field. Fixing the mapping and running the set again took a morning, and the next run scored 47 of 50.
The number belongs to a date, a model and a version of your estate, so it is published with them.
A test run is tied to a version of your estate: a mapping version, a vocabulary version and a rule version. A change to any of them starts a new run, so the published number belongs to what is live.
Runs are also scheduled, usually weekly, and are required before an agent moves up an autonomy level. The evidence an approver reads is the scorecard of the questions that agent depends on.
A published comparison in 2023 scored a general model at 16% correct over raw tables and 54% over a knowledge graph of the same data. Every vendor figure, this one as well, belongs beside its question list.
Showcase estate.
Before the cancellation policy went live, A. Novak and L. Tan built a set of fifty questions from a normal week: shortages, open orders, supplier lateness, cover, cancellation windows and two questions about contract terms that sit outside the registered data.
The first run scored 45 correct, 3 declined and 2 wrong. Both wrong answers traced to the custom cancellation window field. The mapping was corrected on Thursday morning, the set ran again and scored 47 of 50, and that scorecard went to J. Ridley with the request to release the agent at supervised level.
With answers measured and the vocabulary taught, the next stage puts rules to work: agents that act at the autonomy level you set.
Fifty is enough to cover the kinds of question a team asks in a week and small enough for owners to agree every expected answer. Sets grow as more subject areas are registered.
The person who owns the number: a planner for cover, a buyer for supplier performance, a finance lead for value. Their name stays on the expected answer, so a later dispute has a starting point.
It is counted apart from a wrong answer. Where the owners agreed that the question sits outside the registered data, the decline is the right behaviour and scores as correct.
After a change to a mapping, a vocabulary or a rule, on a weekly schedule, and before an agent moves up a level. Each run keeps its date and its evidence.
Each one builds on the one before it.
Reads your rows in place.
Connect and move ↗One record per real thing.
Master and govern ↗Apps for every decision.
Operate and decide ↗Asks and acts, at your level.
Ask and act ↗Bring two systems and one decision your planners make every week. In the first week we register the data, confirm the mapping with your team, answer your questions in ChatLima and run the first agent at the level you choose.
