Retrieval-Augmented Generation (RAG)
As of:
Retrieval-augmented generation (RAG) is a technique in which an LLM receives relevant documents from an external knowledge source before generating its answer. This allows the model to answer questions about content that is not part of its training data — internal documentation, current events or fast-changing datasets, for example.
Technically, documents are first split into sections (chunks) and stored as embeddings in a vector database. At query time, the query is also converted into an embedding, the semantically most similar sections are retrieved — often combined with classical full-text search (“hybrid search”) and a subsequent reranking step — and the hits are placed, together with the question, into the model’s context window, from which it formulates the answer.
In practice, RAG is the standard way of connecting company knowledge to language models: the knowledge base can be updated without retraining, which is cheaper and more current than fine-tuning. RAG reduces hallucinations but does not prevent them — the model can misquote sources or bridge gaps on its own. And quality stands or falls with retrieval: poor chunking or an ill-suited search cannot be compensated for by even the best model.