Structured Indexing

Flat chunks ignore knowledge structure. Two cases showing the problem: counting “100 cats” is impossible via top-k search (aggregation contradicts the nature of “find a few relevant” search); Xfinity discount rule isn’t derived from three isolated cases (search returns semantically close, not the complete rule). Solution: proactively extract, abstract, and structure knowledge at indexing time—compress cases into statistical summaries and explicit rules. Attention mechanism—a soft retrieval system, not an induction engine.

Related: RAPTOR, GraphRAG, RAG