When internal documents disagree, preserve the conflict
A knowledge assistant should resolve conflicting documents using explicit scope and authority rules before composing an answer. When equally applicable sources still disagree, show the unresolved conflict and route it to an owner rather than inventing a compromise.
Inside this article 4 sections +
Two retrieved documents can both be relevant and still disagree. A launch note might promise a feature to every customer while the current product policy limits it to annual plans. Asking the model to synthesize the documents can produce a smooth answer that quietly erases the disagreement.
Build an explicit conflict policy around the question being asked. The task is to determine which sources apply and whether they can support one answer. Retrieval similarity is evidence of topical relevance; it is not a vote on organizational authority.
Check whether the apparent conflict is real
Begin with scope. Documents may describe different products, customer tiers, regions, contract versions, or effective periods. “Exports are retained for seven days” and “Exports are retained for thirty days” can both be correct if one describes temporary trial exports and the other describes an enterprise archive.
Store applicability metadata where possible, and preserve the text that establishes it. A document title may be incomplete, and a scope restriction may appear several paragraphs above the retrieved sentence. The assistant needs that context before it treats two statements as contradictory.
Clarify missing user context when it changes the answer. If the question does not identify the relevant plan, a conditional answer may be sufficient: explain the two applicable cases. Do not select the more generous policy simply because it sounds more helpful.
Use authority and effective dates deliberately
Model source types such as approved policy, implementation guide, proposal, meeting note, and local exception. Let the business define how they relate. A signed exception might govern one account while the general policy governs others. A draft proposal should not override an approved policy merely because it contains a more recent date.
Distinguish publication, modification, approval, and effective dates. A document corrected today may describe last year's rules. A future policy may be published now but not yet apply. If the user asks what was allowed on a past date, the assistant needs historical applicability rather than the newest version.
Record explicit supersession links when the source system supports them. They are stronger evidence than lexical similarity between two titles. Missing metadata should remain visible as missing; a model-generated guess about authority must not become authoritative metadata on the next indexing run.
Write different answers for different conflict states
For an invented policy pair, suppose an approved guide says annual-plan exports remain available for thirty days after generation, while a deprecated help article says seven days. If the approval and supersession records are clear, a useful answer says: “For annual plans under the current export policy, availability is thirty days. The seven-day article was superseded.” It cites the governing passage and the supersession evidence.
Now remove the supersession record and make both documents approved for the same period. The answer should change: “Two approved sources give different retention periods for annual-plan exports: seven days and thirty days. I cannot establish which governs this request.” It should identify the owner or next step without choosing a midpoint or the more recent upload.
For genuinely different scopes, a conditional answer is better: “Trial exports use the seven-day period; annual-plan exports use thirty days.” The assistant should ask for the plan only if it cannot otherwise answer the user's practical question. Avoid turning every harmless difference into an escalation.
Make conflict resolution improve the source system
Capture unresolved conflicts as structured issues with source versions, conflicting claims, applicability, and the question that exposed them. Assign an owner responsible for the policy area. The resolution should update the source or its metadata so future requests benefit; a one-off prompt patch hides the underlying knowledge problem.
Cache answers with their source-version dependencies. When one side of a conflict changes, invalidate or revalidate answers derived from it. Keep historical records where appropriate, but prevent a resolved conflict from returning because an old answer cache outlives the source correction.
Test with synthetic pairs that differ in one dimension: scope, effective date, authority, or wording. Include unresolved cases whose correct output is an explicit conflict. Evaluate whether the assistant preserves the important qualifiers, not whether it always produces one confident recommendation.
For evidence checking, read citation verification. For source lifecycle signals, see knowledge freshness monitoring. Our AI integration work helps connect those controls to the documents your teams maintain.