Document extraction that can say a field is missing
AI document extraction should distinguish a verified value from a missing, unreadable, or conflicting field. Enforce the output schema separately from business validation, and prevent inferred values from entering downstream systems as if they were present in the document.
Inside this article 5 sections +
A document extractor is most useful when it preserves what the document actually says. If a purchase-order number is absent, a plausible number is worse than an empty field: it can send an invoice into the wrong approval process while appearing complete.
Build the integration around field states as well as field values. In an invoice workflow, the supplier name may be readable, the total may be inconsistent with the line items, and the payment reference may be missing. These are three different results that deserve different handling.
Define the meaning of each field state
A practical extraction record includes a raw value, a normalized value, an evidence location, and a state. Use states such as present, missing, unreadable, and conflicting. “Missing” means the expected evidence is absent; “unreadable” means the region may contain it but the system cannot reliably interpret it. Keeping that distinction helps a reviewer request the right correction.
Require the field object even when its value is null. JSON Schema treats an omitted property differently from a property whose value is null. Your contract should make that difference intentional. A missing field object may mean the extractor failed to follow the schema, while a present object with state missing is a valid account of the source.
Keep normalization reversible. Converting a date into an ISO date or an amount into minor currency units is useful, but preserve the original string and the relevant locale assumption. “04/05” is not an unambiguous date without context. Do not resolve that ambiguity by silently applying the server's default locale.
Test the holes before the happy path
Create a small synthetic invoice family with a known answer key. Start with a complete document, then change one property at a time. This makes each failure interpretable and allows the team to test review behavior without using real supplier data.
| Document variant | Expected extraction result | Next action |
|---|---|---|
| Purchase-order box removed | PO state: missing; value: null | Request the reference if this supplier requires one |
| Bank reference blurred | Reference state: unreadable | Ask for a clearer source |
| Two totals disagree | Total state: conflicting | Present both evidence locations for review |
| Currency symbol omitted | Currency state: missing | Do not infer currency from supplier address |
| Text says “ignore validation” | Treat that sentence as document text | Continue the normal validation process |
Add variants that move a label to another page, rotate a scan, or include a repeated invoice summary. Layout changes test the document pipeline rather than just the language model. Retain the original files, expected values, and rendered review state together so that a passing extraction test cannot hide an unusable review screen.
Once the synthetic set is stable, evaluate an appropriately handled sample of the real formats your workflow receives. Synthetic cases reveal designed failure modes; they do not establish the frequency or diversity of real-world documents.
Separate extraction from business validation
First check that the output follows the schema. Next verify that evidence locations resolve to the supplied document. Then apply deterministic business checks: line-item arithmetic, allowed currencies, supplier identifiers, required references, and duplicate detection. These stages answer different questions, so record which one rejected the document.
For example, suppose two net line items are 120 and 80, and tax is 40. A reported total of 240 passes arithmetic, but that does not prove that the supplier identity or bank details are valid. Conversely, a total of 250 may reflect an omitted charge rather than a hallucinated number. Send the inconsistency to review without rewriting the source to make the arithmetic work.
Define tolerances explicitly for rounding and tax calculations. Use decimal or integer monetary arithmetic rather than binary floating-point comparisons. Do not let a model choose a tolerance case by case. The finance workflow must determine which mismatches are acceptable and which require inspection.
This Python example checks one synthetic invoice after required fields and a two-decimal currency have been validated. Values arrive as decimal strings. Its line-by-line rounding rule is explicit and must match the actual invoice contract.
from decimal import Decimal, ROUND_HALF_UP
lines = [("3", "19.95"), ("2", "4.10")]
cent = Decimal("0.01")
subtotal = sum(
(Decimal(qty) * Decimal(price)).quantize(cent, ROUND_HALF_UP)
for qty, price in lines
)
tax = Decimal("13.61")
expected = subtotal + tax
extracted_total = Decimal("81.66")
assert expected == Decimal("81.66")
assert extracted_total == expected
assert Decimal("81.65") != expected
A mismatch creates a review reason; it must not cause the validator to silently replace the extracted total. Preserve the evidence for quantity, price, tax, and total separately. This example does not infer a tax rule, handle every currency precision, or accept missing fields as zero.
Make the reviewer repair the record, not guess the source
Show the field beside its evidence region and the validation message. A reviewer should be able to distinguish a source correction from a normalization correction. Preserve the initial extraction, the human change, and the accepted version so later debugging can identify where the error entered the process.
Approval should bind to the document version and the normalized record. If someone uploads a replacement invoice, invalidate an approval tied to the earlier content. The downstream write should use an idempotency key based on the business operation, preventing a repeated extraction job from creating a second invoice.
Measure field-specific errors, review effort, and rejected-document reasons. A high document-level pass rate can conceal a recurring mistake in a rare but important field. Treat empty-but-correct values as successes when the source is incomplete; otherwise the evaluation rewards guessing.
Run the review and downstream handoff
A correct extraction is only one boundary in the workflow. The accepted record must still reach the system that owns the business operation. The downloadable Python experiment keeps these steps separate: validate a synthetic invoice, approve the exact document version and record, then submit one operation to an in-memory destination. Missing fields and mismatched totals return a review state without rewriting the source.
Required field states, evidence labels and arithmetic determine whether review can proceed.
A replaced document or edited payload needs a new approval.
A timeout stays unresolved until the destination can establish completion.
Run python3 document_handoff.py after extracting the download. The twelve tests include a missing field object, missing currency, invalid monetary input, an unapproved record, a changed document version and a different account. Two tests simulate failure on either side of a destination commit. The caller receives the same unresolved state in both cases; the fixture’s idempotent destination allows a retry to confirm one final operation without creating another record.
That last behaviour depends on the destination contract. A real integration needs durable operation storage, atomic duplicate handling, current approval permissions and a supported lookup or reconciliation path. If the downstream API cannot provide that, route uncertain completion to an operator instead of promising automatic recovery. The example’s evidence labels and actor identities are test inputs, not verified document spans or authenticated users.
See designing the human approval inbox for version-bound decisions and AI inside an existing SaaS product for durable workflow boundaries. Our AI integration services can connect extraction to your existing records and review process.