Year :
0 | Volume:
4 | Issue:
11 | Pages :
39-43
Retrieval-Augmented Generation (RAG): A Two-Decade Survey (2005–2020)
The rapid advancement of Artificial Intelligence (AI) and Large Language Models (LLMs) has transformed natural language processing applications across diverse domains. However, traditional language models often suffer from limitations such as hallucination, outdated knowledge, and lack of transparency in decision-making. Retrieval-Augmented Generation (RAG) has emerged as a promising paradigm that combines information retrieval techniques with generative models to enhance factual accuracy, contextual relevance, and knowledge grounding. This survey presents a comprehensive review of the evolution of RAG-related technologies over the past two decades, from early information retrieval and question-answering systems to modern neural retrieval and large language model architectures. We examine the fundamental components of RAG frameworks, including retrieval mechanisms, embedding models, vector databases, reranking strategies, and generation modules. Furthermore, we compare prominent RAG architectures, discuss their strengths and limitations, and analyze their applications in healthcare, education, enterprise knowledge management, scientific research, and conversational AI[1-2]. Finally, we identify current challenges and future research directions, including multimodal retrieval, adaptive retrieval strategies, continual learning, and trustworthy AI systems. This survey aims to provide researchers and practitioners with a structured understanding of the development, state-of-the-art methods, and future prospects of Retrieval-Augmented Generation systems.
References
[1] C. J. C. Burges, T. Shaked, E. Renshaw, A. Lazier, M. Deeds, N. Hamilton, and G. Hullender, “Learning to Rank Using Gradient Descent,” Proceedings of the 22nd International Conference on Machine Learning (ICML), pp. 89–96, 2005. DOI: 10.1145/1102351.1102363.
[2] P.-S. Huang, X. He, J. Gao, L. Deng, A. Acero, and L. Heck, “Learning Deep Structured Semantic Models for Web Search Using Clickthrough Data,” Proceedings of the 22nd ACM International Conference on Information and Knowledge Management (CIKM), 2013. DOI: 10.1145/2505515.2505665.
[3] D. Bahdanau, K. Cho, and Y. Bengio, “Neural Machine Translation by Jointly Learning to Align and Translate,” International Conference on Learning Representations (ICLR), 2015.
[4] P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang, “SQuAD: 100,000+ Questions for Machine Comprehension of Text,” Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 2383–2392, 2016. DOI: 10.18653/v1/D16-1264.
[5] D. Chen, A. Fisch, J. Weston, and A. Bordes, “Reading Wikipedia to Answer Open-Domain Questions,” Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 1870–1879, 2017. DOI: 10.18653/v1/P17-1171.
[6] M. Joshi, E. Choi, D. Weld, and L. Zettlemoyer, “TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension,” Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp. 1601–1611, 2017. DOI: 10.18653/v1/P17-1147.
[7] J. Johnson, M. Douze, and H. Jégou, “Billion-Scale Similarity Search with GPUs,” arXiv preprint arXiv:1702.08734, 2017.
[8] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” arXiv preprint arXiv:1810.04805, 2018.
[9] K. Lee, M.-W. Chang, and K. Toutanova, “Latent Retrieval for Weakly Supervised Open Domain Question Answering,” Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 6086–6096, 2019. DOI: 10.18653/v1/P19-1612.
[10] T. Kwiatkowski, J. Palomaki, O. Redfield, M. Collins, A. Parikh, C. Alberti, D. Epstein, I. Polosukhin, J. Devlin, K. Lee, K. Toutanova, L. Jones, M. Kelcey, M.-W. Chang, A. Dai, J. Uszkoreit, Q. Le, and S. Petrov, “Natural Questions: A Benchmark for Question Answering Research,” Transactions of the Association for Computational Linguistics, vol. 7, pp. 452–466, 2019. DOI: 10.1162/tacl_a_00276.
[11] K. Guu, K. Lee, Z. Tung, P. Pasupat, and M.-W. Chang, “REALM: Retrieval-Augmented Language Model Pre-Training,” Proceedings of the 37th International Conference on Machine Learning (ICML), vol. 119, pp. 3929–3938, 2020.
[12] V. Karpukhin, B. O?uz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W.-t. Yih, “Dense Passage Retrieval for Open-Domain Question Answering,” Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 6769–6781, 2020. DOI: 10.18653/v1/2020.emnlp-main.550.
[13] O. Khattab and M. Zaharia, “ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT,” arXiv preprint arXiv:2004.12832, 2020.
[14] M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer, “BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension,” Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7871–7880, 2020. DOI: 10.18653/v1/2020.acl-main.703.
[15] P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel, S. Riedel, and D. Kiela, “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” Advances in Neural Information Processing Systems (NeurIPS), 2020.
