Choose chunk boundaries for policy documents
Policy-document chunks should preserve the rule together with the conditions and exceptions needed to interpret it. Compare chunking strategies under the same retrieval and context budget, measuring whether answers receive complete applicable evidence rather than merely a relevant sentence.
Inside this article 4 sections +
A policy rule rarely stands alone. Eligibility may be defined in the previous paragraph, an exception may appear under the next heading, and a table may change the rule for one customer group. A chunk containing the right keyword can still omit the information that makes the answer correct.
Choose boundaries by testing whether the retrieved evidence preserves the policy’s meaning. Chunk size is one parameter in that experiment, not a universal setting to copy from a tutorial. Your questions, document structure, and available context budget determine what useful evidence looks like.
Start with the smallest complete policy unit
Use an invented equipment-return policy as a working example. The main paragraph says equipment must be returned within fourteen days. The next paragraph says employees on approved leave may request an extension. A table identifies which device categories require a prepaid shipping label.
A question about the ordinary deadline needs the main rule. A question about a person on leave needs both the rule and the exception. A question about shipping needs the relevant table row and its headers. These are different evidence requirements even though they belong to one document section.
Preserve headings, list relationships, table headers, and source locations during parsing. If extraction loses that structure, a clever splitter cannot reliably reconstruct it. Inspect the extracted representation before tuning embeddings or retrieval ranking.
Compare three boundary strategies
Boundary through the rule
Return within 14 days→Chunk B
Approved-leave exception
The deadline can rank highly while the condition needed for the question remains outside the retrieved context.
Boundary around the section
The evidence stays together, at the cost of retrieving more text for a simple deadline question.
A fixed-size baseline is useful because it is easy to reproduce. Specify the tokenizer or character-count method, target size, and overlap. Overlap can preserve nearby context, but it also duplicates text and may fill the retrieval budget with several versions of the same passage.
A section-aware strategy follows meaningful headings and splits oversized sections at paragraph or list boundaries. Include enough parent context to identify the policy and scope. Avoid assuming that every heading represents a self-contained rule; some documents put definitions in a separate introductory section.
A parent-child strategy searches smaller child passages and expands selected hits into a larger parent section. This can combine precise matching with complete evidence, but the expansion consumes context and can pull in unrelated material. Keep source identity, permissions, and version consistent between child and parent records.
Hold the rest of the retrieval experiment steady
Build a question set around the policy decisions: ordinary deadline, exception eligibility, category-specific shipping, ambiguous scope, and a question the policy does not answer. Write the required evidence for each case before seeing retrieval results. Include paraphrases that do not repeat the document's exact vocabulary.
Compare strategies using the same source versions, embedding model, query set, filtering rules, and ranking configuration. Match the total context budget as closely as practical. Comparing five huge sections with five tiny fragments confounds chunk boundaries with the amount of evidence supplied.
Measure complete-evidence retrieval: did the selected context include every passage needed for a correct answer? Also record redundant text, missing exceptions, wrong-scope passages, and irrelevant expansion. A simple “relevant chunk in the top five” score can pass while the decisive exception is absent.
Evaluate the resulting answers separately. If complete evidence arrives but the answer drops the exception, the problem is downstream of retrieval. If the answer fails because only the deadline arrived, generation tuning is unlikely to solve it. Keeping these stages separate makes the next engineering change more obvious.
Plan for edits, tables, and index growth
Chunk identifiers should support versioned replacement. When an editor inserts a paragraph near the beginning, a naive positional identifier scheme can make every later chunk appear new. That may be acceptable, but it changes reindexing cost and cache invalidation. Track the source version independently so old and new policy fragments do not mix.
Inspect tables on their own. A row without its column heading can reverse meaning, especially when headings contain dates or eligibility limits. Consider repeating header context in each searchable row or retrieving the whole relevant table segment. Test the actual parser output rather than assuming the PDF's visual layout survived extraction.
Choose the smallest configuration that consistently supplies complete evidence within your budget. Keep the experiment and representative failures for future document-format changes. Chunking is part of the knowledge product's maintenance, not a one-time indexing preference.
See citation verification for checking answer support and conflicting document handling for authority decisions. Our AI integration services can connect parsing and retrieval tests to your existing source systems.