Structured Indexing

Flat chunks ignore knowledge structure. Two cases showing the problem:

  • counting “100 cats” is impossible via top-k search — aggregation contradicts the nature of “find a few relevant ones”;
  • Xfinity discount rule isn’t derived from three isolated cases — search will return semantically close, not the complete rule.

Solution: pre-extract, abstract, and structure knowledge at indexing time — compress cases into statistical summaries and explicit rules.

Attention mechanism is a soft search engine, not an induction engine.

Related: RAPTOR, GraphRAG, RAG