This repo aims to record survey of LLM, including instruction tuning, human alignment, LLM agent, hallucination, multi-modal, etc.
We strongly encourage the researchers that want to promote their fantastic work to the LLM survey community to make pull request to update their paper's information!
Awesome-LLM-Survey
General Survey
Training of LLM
Instruction Tuning
Human Alignment for LLM
Prompt of LLM
Chain of Thought for LLM
Prompt Engineering for LLM
Retrieval-Augmented LLM
Challenge of LLM
Hallucination in LLM
Compression for LLM
Evaluation of LLM
Reasoning with LLM
Explainability for LLM
Fairness in LLM
Graph for LLM
Long-Context for LLM
Factuality in LLM
Knowledge for LLM
Self-Correction for LLM
Attributions for LLM
Tool Using of LLM
Calibration of LLM
Agent of LLM
Role-play of LLM
Vulnerabilities of LLM
Efficiency of LLM
Data of LLM
Security and Privacy of LLM
Continual Learning of LLM
Mulitmodal of LLM
Visual LLM
Audio LLM
Code LLM
LLM for Domain Application
LLM for Health
LLM for Finance
LLM for Education
LLM for Law
LLM for Mental Health
LLM for Robotics
LLM for Downstream Tasks
LLM for Recommendation
LLM for Information Retrieval
LLM for Software Engineering
LLM for Autonomous Driving
LLM for Time Series
Detection of LLMs-Generated Content
LLM for Society
LLM for Citation
LLM for Text Watermarking
LLM for Math
LLM for Environmental Disciplines
LLM for Information Extraction
LLM for Data Annotation
LLM for Game
Star History
General Survey
A Survey of Large Language Models, 2023.11 [paper][project]
A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4, 2023.10 [paper]
Challenges and Applications of Large Language Models, 2023.07 [paper]
Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond, 2023.04 [paper][project]
A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT, 2023.02 [paper]
Large language models: a comprehensive survey of its applications, challenges, limitations, and future prospects, 2023.12 [paper] [project]
The future of gpt: A taxonomy of existing chatgpt research, current challenges, and possible future directions, 2023.04 [paper]
A Review on Large Language Models: Architectures, Applications, Taxonomies, Open Issues and Challenges, 2023.10 [paper]
Understanding LLMs: A Comprehensive Overview from Training to Inference, 2024.01 [paper]
Training of LLM
Instruction Tuning
Are Prompts All the Story? No. A Comprehensive and Broader View of Instruction Learning, 2023.03 [paper] [project]
Vision-Language Instruction Tuning: A Review and Analysis, 2023,11 [paper][project]
Instruction Tuning for Large Language Models: A Survey, 2023.08 [paper]
A Survey on Data Selection for LLM Instruction Tuning, 2024.02 [paper]
Human Alignment for LLM
Towards a Unified View of Preference Learning for Large Language Models: A Survey, 2024.09 [paper][project]
AI Alignment: A Comprehensive Survey, 2023.10 [paper][project]
Large Language Model Alignment: A Survey, 2023.09 [paper]
From Instructions to Intrinsic Human Values -- A Survey of Alignment Goals for Big Model, 2023.08 [paper][project]
Aligning Large Language Models with Human: A Survey, 2023.07 [paper][project]
Prompt of LLM
Chain of Thought for LLM
Towards Better Chain-of-Thought Prompting Strategies: A Survey, 2023.10 [paper]
A Survey of Chain of Thought Reasoning: Advances, Frontiers and Future, 2023.09 [paper][project]
Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agents, 2023.11 [paper] [project]
Prompt Engineering for LLM
Prompting Frameworks for Large Language Models: A Survey, 2023.11 [paper][project]
Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review, 2023.10 [paper]
Towards Better Chain-of-Thought Prompting Strategies: A Survey, 2023.10 [paper]
Retrieval-Augmented LLM
A Survey on Retrieval-Augmented Text Generation, 2022.02 [paper]
Retrieval-Augmented Generation for Large Language Models: A Survey, 2023.12 [paper] [project]
RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processing, 2024.04 [paper]
Challenge of LLM
Hallucination in LLM
Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Survey, 2023.11 [paper]
A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions, 2023.11 [paper][project]
A Survey of Hallucination in “Large” Foundation Models, 2023.09 [paper][project]
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models, 2023.09 [paper][project]
Cognitive Mirage: A Review of Hallucinations in Large Language Models, 2023.09 [paper][project]
Augmenting LLMs with Knowledge: A survey on hallucination prevention, 2023.09 [paper]
A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models, 2024.01 [paper]
Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment, 2023.08 [paper]
Hallucination of Multimodal Large Language Models: A Survey, 2024.04 [paper]
Compression for LLM
A Survey on Model Compression for Large Language Models, 2023.08 [paper]
A Comprehensive Survey of Compression Algorithms for Language Models, 2024.01 [paper]
Evaluation of LLM
Evaluating Large Language Models: A Comprehensive Survey, 2023.10 [paper][project]
A Survey on Evaluation of Large Language Models, 2023.07 [paper][project]
Reasoning with LLM
Reasoning with Language Model Prompting: A Survey, 2022.12 [paper][project]
A Survey of Reasoning with Foundation Models, 2023.12 [papaer][project]
Explainability for LLM
Explainability for Large Language Models: A Survey, 2023.09 [paper]
The Mystery and Fascination of LLMs: A Comprehensive Survey on the Interpretation and Analysis of Emergent Abilitie, 2023.11 [paper]
If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents, 2024.01 [paper]
From Understanding to Utilization: A Survey on Explainability for Large Language Models, 2024.01 [paper]
Awesome Attention Heads, 2024.07 [project]
Internal Consistency and Self-Feedback in Large Language Models: A Survey, 2024.07 [paper]
Attention Heads of Large Language Models: A Survey, 2024.09 [paper]
Fairness in LLM
A Survey on Fairness in Large Language Models, 2023.08 [paper]
Graph for LLM
A Survey of Graph Meets Large Language Model: Progress and Future Directions, 2023.11 [paper]
Large Language Models on Graphs: A Comprehensive Survey, 2023.12 [paper] [project]
Long-Context for LLM
Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Survey, 2023.11 [paper]
Length Extrapolation of Transformers: A Survey from the Perspective of Position Encoding, 2023.12 [paper]
Factuality in LLM
A Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity, 2023.10 [paper][project]
Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Models, 2023.10 [paper]
Knowledge for LLM
A Survey on Knowledge Distillation of Large Language Models, 2024.02 [paper]
Knowledge Unlearning for LLMs: Tasks, Methods, and Challenges, 2023.11 [paper]
Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applications, 2023.11 [paper]
Knowledge Editing for Large Language Models: A Survey, 2023.10 [paper]
Editing Large Language Models: Problems, Methods, and Opportunities, 2023.05 [paper][project]
Building trust in conversational ai: A comprehensive review and solution architecture for explainable, privacy-aware systems using llms and knowledge graph, 2023.08 [paper]
Self-Correction for LLM
Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies, 2023.08 [paper][project]
Attributions for LLM
A Survey of Large Language Models Attribution, 2023.11 [paper][project]
Tool Using of LLM
Foundation Models for Decision Making: Problems, Methods, and Opportunities, 2023.03 [paper]
Augmented Language Models: a Survey, 2023.02 [paper]
Calibration of LLM
A Survey of Language Model Confidence Estimation and Calibration, 2023.11 [paper]
Agent of LLM
A Survey on Large Language Model based Autonomous Agents, 2023.08 [paper][project]
The Rise and Potential of Large Language Model Based Agents: A Survey, 2023.09 [paper][project]
Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives, 2023.12 [paper]
Large Multimodal Agents: A Survey, 2024.02 [paper][project]
Role-play of LLM
The Oscars of AI Theater: A Survey on Role-Playing with Language Models, 2024.09 [paper][project]
Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization, 2024.06 [paper][project]
From Persona to Personalization: A Survey on Role-Playing Language Agents, 2024.04 [paper]
Vulnerabilities of LLM
Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks, 2023.10 [paper]
Efficiency of LLM
The Efficiency Spectrum of Large Language Models: An Algorithmic Survey, 2023.12 [paper][project]
Efficient Large Language Models: A Survey, 2023.12 [paper][project]
Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment, 2023.12 [paper]
A Survey on Hardware Accelerators for Large Language Models, 2024.01 [paper]
Model Compression and Efficient Inference for Large Language Models: A Survey, 2024.02 [paper]
Data of LLM
Data Management For Large Language Models: A Survey, 2023.12 [paper][project]
A Survey on Data Selection for Language Models, 2024.02 [paper]
Datasets for Large Language Models: A Comprehensive Survey, 2024.02 [paper][project]
Security and Privacy of LLM
A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly, 2023.12 [paper]
Continual Learning of LLM
Continual Learning with Pre-Trained Models: A Survey, 2024.01 [paper] [project]
Continual Learning of Large Language Models: A Comprehensive Survey, 2024.04 [paper]
Mulitmodal of LLM
Visual LLM
A Survey on Benchmarks of Multimodal Large Language Models. 2024,08 [paper][project]
The (R)Evolution of Multimodal Large Language Models: A Survey, 2024,02 [paper]
Vision-Language Instruction Tuning: A Review and Analysis, 2023,11 [paper][project]
How to Bridge the Gap between Modalities: A Comprehensive Survey on Multimodal Large Language Model, 2023.11 [paper]
A Survey on Multimodal Large Language Models, 2023.06 [paper][project]
Multimodal Large Language Models: A Survey, 2023.11 [paper]
Large Language Models Meet Computer Vision: A Brief Survey, 2023.11 [paper]
Foundational Models Defining a New Era in Vision: A Survey and Outlook, 2023.07 [paper][project]
Video Understanding with Large Language Models: A Survey, 2023.12 [paper] [project]
Audio LLM
Sparks of large audio models: A survey and outlook, 2023.08 [paper] [project]
Code LLM
A Survey of Neural Code Intelligence: Paradigms, Advances and Beyond, 2024.03 [paper][project]
A Survey on Language Models for Code, 2023.11 [paper][project]
Pitfalls in Language Models for Code Intelligence: A Taxonomy and Survey, 2023.10 [paper][project]
Large Language Models Meet NL2Code: A Survey, 2022.12 [paper]
A Prompt Learning Framework for Source Code Summarization, 2023.12 [paper]
If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents, 2024.01 [paper]
LLM for Domain Application
LLM for Health
A Survey of Large Language Models in Medicine: Progress, Application, and Challenge, 2023.11 [paper][project]
Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Review, 2023.10 [paper][project]
Large AI Models in Health Informatics: Applications, Challenges, and the Future, 2023.03 [paper][project]
A SWOT (Strengths, Weaknesses, Opportunities, and Threats) Analysis of ChatGPT in the Medical Literature: Concise Review, 2023.11 [paper]
ChatGPT in Healthcare: A Taxonomy and Systematic Review, 2023.03 [paper]
LLM for Finance
Large Language Models in Finance: A Survey, 2023.09 [paper]
LLM for Education
ChatGPT and Beyond: The Generative AI Revolution in Education, 2023.11 [paper]
LLM for Law
Large Language Models in Law: A Survey, 2023.12 [paper]
LLM for Mental Health
A review of the explainability and safety of conversational agents for mental health to identify avenues for improvement, 2023.10 [paper]
Towards a Psychological Generalist AI: A Survey of Current Applications of Large Language Models and Future Prospects, 2023.12 [paper]
Large Language Models in Mental Health Care: a Scoping Review, 2024.01 [paper]
LLM for Robotics
Large Language Models for Robotics: A Survey, 2023.11 [paper]
LLM for Downstream Tasks
LLM for Recommendation
Foundation Models for Recommender Systems: A Survey and New Perspectives, 2024.02 [paper]
User Modeling in the Era of Large Language Models: Current Research and Future Directions, 2023.12 [paper][project]
A Survey on Large Language Models for Personalized and Explainable Recommendations, 2023.11 [paper]
Large Language Models for Generative Recommendation: A Survey and Visionary Discussions, 2023.09 [paper]
A Survey on Large Language Models for Recommendation, 2023.08 [paper][project]
How Can Recommender Systems Benefit from Large Language Models: A Survey, 2023.06 [paper][project]
LLM for Information Retrieval
Large Language Models for Information Retrieval: A Survey, 2023.08 [paper][project]
LLM for Software Engineering
Large Language Models for Software Engineering: Survey and Open Problems, 2023.10 [paper]
Large Language Models for Software Engineering: A Systematic Literature Review, 2023.08 [paper]
LLM for Autonomous Driving
A Survey on Multimodal Large Language Models for Autonomous Driving, 2023.11 [paper]
A Survey of Large Language Models for Autonomous Driving, 2023.11 [paper][project]
LLM for Time Series
Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook, 2023.10 [paper][project]
Detection of LLMs-Generated Content
A Survey on Detection of LLMs-Generated Content, 2023.10 [paper][project]
A Survey on LLM-generated Text Detection: Necessity, Methods, and Future Directions, 2023.10 [paper]
[project]
Detecting ChatGPT: A Survey of the State of Detecting ChatGPT-Generated Text, 2023.09 [paper]
LLM for Society
Large Language Models as Subpopulation Representative Models: A Review, 2023.10 [paper]
LLM for Citation
When Large Language Models Meet Citation: A Survey, 2023.09 [paper]
LLM for Text Watermarking
A Survey of Text Watermarking in the Era of Large Language Models, 2023.12 [paper]
LLM for Math
Mathematical Language Models: A Survey, 2023.12 [paper]
LLM for Environmental Disciplines
Recent applications of AI to environmental disciplines: A review, 2023.10 [paper]
Opportunities and Challenges of Applying Large Language Models in Building Energy Efficiency and Decarbonization Studies: An Exploratory Overview, 2023.12 [paper]
LLM for Information Extraction
Large Language Models for Generative Information Extraction: A Survey, 2023.12 [paper] [project]
LLM for Data Annotation
Large Language Models for Data Annotation: A Survey, 2024.02 [paper] [project]
LLM for Game
Large Language Models and Games: A Survey and Roadmap, 2024.02 [paper]
A Survey on Game Playing Agents and Large Models: Methods, Applications, and Challenges, 2024.03 [paper] [project]
