Dense Embedding

Dense embedding maps text into a vector space where semantically similar items are close. Similarity is measured by the cosine of the angle (direction matters, length does not). Evolution: Word2Vec – static word vectors, does not distinguish polysemy (“bank”); BERT and BGE-M3 – contextual representations, the same word in different contexts gets different vectors. ANN indexes: ANNOY (tree, fast, no incremental updates) and HNSW (graph, incremental, more accurate).

Related: [Sparse BM25 Search], [Hybrid Search], RAG