Separate factual errors from AI workflow failures
An AI evaluation should record what failed, where the first causal failure occurred, and which team can repair it. Separate factual correctness from workflow completion and permission enforcement, then preserve linked symptoms so a single incident is neither counted repeatedly nor reduced to one vague hallucination label.
Inside this article 5 sections +
A support assistant can state the correct policy and still fail the task by sending the answer to the wrong account. It can also follow the workflow perfectly while inventing a refund entitlement. Calling both outcomes “AI accuracy problems” makes it difficult to choose the right fix.
Build an error taxonomy around the product's decision boundaries. The aim is not to produce a sophisticated list of labels; it is to help a reviewer explain what failed, identify the earliest repairable cause, and assign the work without losing the customer-visible consequence.
Grade the answer and the workflow separately
Answer grading asks whether claims are supported, complete enough for the question, and appropriately qualified. Workflow grading asks whether the system respected permissions, selected allowed actions, maintained state, and reached the defined completion condition. Keep both results even when one is an immediate release blocker.
For an illustrative cancellation request, the answer might correctly explain that an annual plan ends on its renewal date. If the workflow then cancels another subscription, factual correctness does not mitigate the action error. Conversely, a correctly targeted draft that promises a refund absent from policy is a content failure even if no external action occurred.
Add a separate measurement-validity check. Sometimes the test harness provides stale fixtures, the grader expects obsolete policy, or a tool stub behaves differently from production. A failing test may reveal a product defect or an invalid experiment. Treat that distinction as something to investigate, not an excuse to discard uncomfortable results.
Record cause, symptoms, and ownership
| Observed failure | Primary category | Repair owner | Regression assertion |
|---|---|---|---|
| Answer cites a retired refund rule | Evidence selection | Retrieval and content owners | Only applicable policy versions enter context |
| Current exception is omitted | Generation fidelity | AI feature team | Answer preserves the supplied exception |
| Correct action affects another tenant | Authorization boundary | Application security owner | Cross-tenant target cannot execute |
| Timeout causes two messages | Effect reconciliation | Workflow integration owner | One business operation produces at most one intended send |
| Grader expects last year's deadline | Evaluation defect | Evaluation owner | Reference policy matches the case date |
Store a case identifier, configuration version, evidence snapshot, observed result, primary category, secondary symptoms, severity, confidence in the diagnosis, and owner. Keep a link to a redacted trace rather than copying sensitive input into every spreadsheet cell.
Use one primary category for aggregate incident counts and allow multiple secondary tags. If a stale document produces three unsupported sentences, count one affected task in the task failure rate while preserving the three claim failures for content analysis. State the denominator whenever reporting either measure.
Locate the earliest boundary that broke
Review the execution in order: authorized input, selected evidence, model result, validation decision, proposed action, executed effect, and final state. Ask whether each stage received a valid input and honored its contract. The earliest broken contract often suggests the primary cause, but verify that later controls were not also supposed to catch it.
For example, an unauthorized document entering retrieval is an access failure even if the generator never quotes it. If a final output filter catches the disclosure, record the prevented exposure and the failed upstream boundary. A safe-looking final response should not erase evidence of a broken permission control.
Use counterfactual reruns carefully. Replacing stale evidence with the correct passage can test whether retrieval caused the answer failure. It does not prove the generation stage is reliable on all other cases. Preserve the original run and change one factor at a time so the diagnosis remains inspectable.
Allow an unresolved diagnosis state. Forcing reviewers to choose a confident cause when the trace is incomplete creates misleading ownership data. Assign the next investigative step and improve instrumentation. Unknown cause is more useful than a convenient hallucination label attached to every failure.
Keep reviewers consistent without making the taxonomy rigid
Write a short decision guide with positive and negative examples for each category. Have two reviewers classify a sample independently, then discuss disagreements. If they repeatedly disagree between evidence selection and generation, the trace may not show the exact context supplied to the model.
Separate severity from category. A schema error can be harmless if the workflow visibly requests review, or serious if a permissive parser substitutes a dangerous default. Severity depends on the consequence and containment, not the technical layer's name.
Version the taxonomy when meanings change. Keep a mapping for historical reports so a sudden rise in one category is not merely a renamed label. Avoid adding a new category for every incident; use secondary tags for details such as language, document type, or provider adapter.
Turn the ledger into a repair loop
Prioritize recurring causes and high-consequence boundaries. A fix should name the contract it changes, the tests that demonstrate it, and any new tradeoff. After deployment, rerun the original failures plus neighboring cases so a local patch does not merely memorize one example.
Report task success, category distribution, unresolved diagnoses, and escaped failures together. A declining factual-error rate is encouraging, but it should not distract from a rise in incomplete workflows or unauthorized attempts. The taxonomy supports judgment; it does not replace release criteria.
Use model upgrade evaluation for paired release decisions and citation verification for claim-level support checks. Our AI integration work connects these measurements to practical remediation.