Continual Learning of Large Language Models: A Comprehensive Survey
Published in ACM Computing Surveys, 2024
This comprehensive survey examines the landscape of continual learning approaches for large language models, providing a systematic analysis of current methodologies, identifying key challenges, and outlining future research directions.
Key Contributions
- Systematic taxonomy of continual learning approaches for LLMs and comprehensive analysis of catastrophic forgetting mitigation strategies
- In-depth discussion of evaluation protocols and benchmarks and identification of open challenges and future research directions
Status: ACM Computing Surveys
Recommended citation: Haizhou Shi, Zihao Xu, Hengyi Wang, Weiyi Qin, Wenyuan Wang, Yibin Wang, Zifeng Wang, Sayna Ebrahimi, Hao Wang. "Continual Learning of Large Language Models: A Comprehensive Survey." ACM Computing Surveys.
Download Paper