Researchers examined three basic decisions in retrieval-augmented generation systems: whether to use retrieval, how many documents to provide, and how to incorporate those documents into an answer. Their experiments covered question answering and code generation across three language models and six datasets.

The results differed by task. Question answering showed a consistent range of five to 10 useful documents, while code generation had no single best amount. Prompting methods also helped code generation more than question answering, indicating that system design needs to account for both the task and the language model being used.