Embedding

An embedding is a numerical vector representation of content — text, images or code, for instance — in which semantic similarity is mapped to geometric proximity. Two related sentences end up close to each other in vector space even if they share no words. This makes embeddings the foundation of semantic search.

Embeddings are produced by neural networks trained for this purpose, which map inputs to vectors of fixed length, typically a few hundred to a few thousand dimensions. The similarity of two vectors is then computed via cosine similarity, for example. LLMs also work with embeddings internally to represent tokens; search applications, however, use dedicated embedding models whose vectors are stored in vector databases.

In practice, embeddings are the core building block of RAG systems, semantic search, recommender systems, duplicate detection and clustering. Two misconceptions are widespread: first, embeddings do not “understand” anything — they encode statistical similarity patterns from their training data, which is why negations are poorly distinguished and similarity does not imply factual agreement. Second, embeddings from different models are not compatible: switching models means re-indexing the entire corpus.