Retrieval-augmented generation (RAG) is a technique in natural language processing in which a text-generating model is coupled to an external document collection that it consults while producing output. Rather than depending solely on facts absorbed into its weights during training, a RAG system first searches a corpus for passages relevant to the input, then conditions generation on what it finds. The design combines parametric memory, held in the generator's learned weights, with non-parametric memory, held in a searchable index that can be replaced without retraining the model [1].
The term was introduced in a 2020 paper by Patrick Lewis and collaborators, presented at NeurIPS 2020. Their starting observation was that large pre-trained language models store a substantial amount of factual knowledge but access and manipulate it imprecisely. They addressed this by pairing a pre-trained sequence-to-sequence generator with a neural retriever operating over a dense vector index of Wikipedia. So configured, the models reached state-of-the-art results on three open-domain question answering tasks, outperforming both purely parametric baselines and task-specific retrieve-and-extract pipelines, with generated language the authors described as more specific, diverse and factual [1].
Closely related work appeared in the same period. REALM, published by Guu and colleagues in 2020, introduced retrieval into language model pre-training rather than only at fine-tuning or inference time. It equips the model with a latent knowledge retriever that attends over documents drawn from a large corpus such as Wikipedia, and the retriever is learned without supervision by using masked language modelling as the training signal. On open-domain question answering the authors reported improvements of 4 to 16 percentage points absolute over previous methods, alongside qualitative benefits in interpretability and modularity [2].
The retrieval component came from Dense Passage Retrieval (DPR), presented at EMNLP 2020, which showed that retrieval could be carried out with dense learned embeddings alone, using a simple dual-encoder that embeds questions and passages separately and is trained from a comparatively small set of question-passage pairs. It improved top-20 passage retrieval accuracy over a strong Lucene-BM25 baseline by 9 to 19 percentage points absolute [3].
Most implementations follow three stages. Indexing segments the corpus into passages and stores a representation of each, typically an embedding vector in a nearest-neighbour index. At query time a retriever maps the input into that same space and returns ranked candidate passages. Those passages are placed alongside the original query in the generator's input, and the model answers conditioned on both. A survey by Gao and colleagues organises the literature around this tripartite foundation of retrieval, generation and augmentation [4].
The original paper defined two ways of conditioning the generator. In the RAG-Sequence formulation, a single retrieved document is used to condition the entire generated sequence; in the RAG-Token formulation, the model may draw on a different retrieved document for each token it emits [1].
The commonly cited motivations for RAG are that language models hallucinate, that knowledge frozen into weights becomes outdated, and that a model's reasoning is otherwise non-transparent and hard to trace. Grounding generation in retrieved documents is meant to address these by supplying current external knowledge and making the evidence behind an answer inspectable. The same survey groups systems into three successive paradigms, labelled Naive RAG, Advanced RAG and Modular RAG [4].
Retrieval has also been applied at very large scale during pre-training. The RETRO model of Borgeaud and colleagues retrieves from a database of roughly two trillion tokens and reports performance comparable to GPT-3 and Jurassic-1 while using about 25 times fewer parameters, evidence that explicit retrieval can substitute for some knowledge otherwise stored in weights [5].
RAG does not eliminate error. Retrieval may return passages that are irrelevant, contradictory or simply absent from the corpus, and the generator remains free to produce statements the retrieved text does not support. Performance also depends on how retrieved material is arranged. An empirical study by Liu and colleagues found that language models use long contexts unevenly: accuracy tends to be highest when the relevant information appears near the beginning or the end of the input and degrades noticeably when it sits in the middle, an effect observed even in models explicitly built for long contexts [6]. This bears directly on RAG, since a retrieval step that buries the decisive passage mid-prompt may fail to help even when retrieval itself succeeded.
Evaluation remains an open area; the survey notes that assessment frameworks and benchmarks for RAG are still developing and identifies outstanding challenges across all three stages [4].