Research_Summarization_Finetuning_MistralAI_LLM

Research Paper Summarization: An Integrated Approach with Abstractive Methods and RAG Technology

Project Overview

This project aims to develop a comprehensive system for summarizing research papers, combining the latest advancements in AI and NLP. It encompasses two primary methodologies:

1. Abstractive Text Summarization: Utilizing deep neural networks with an encoder-decoder and attention mechanism, this method aims to generate concise, paraphrased summaries of research papers.
2. Integration of LLMs and RAG Technology: Harnessing the capabilities of Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG), this approach focuses on producing interactive summaries and extracting valuable insights from research papers.

Methodology

Abstractive Text Summarization

• Goal: To produce paraphrased summaries encapsulating core ideas of research papers.
• Techniques: Encoder-decoder model with attention mechanism, transfer learning.

Large Language Models and RAG Technology

• Objective: To use LLMs for interactive summarization.
• Process: Fine-tuning LaMini LLM, developing knowledge graphs with neo4j, and implementing RAG methodology.

Data Source

The primary data sources are research papers from various fields, focusing on their introduction, methodology, and conclusion sections.

️ Limitations

• Dependency on the availability and quality of research paper data.
• The diverse nature of research papers could challenge the summarization process.
• Resource-intensive fine-tuning of LLMs.
• Potential unforeseen challenges with RAG methodology.

? Conclusion

Our project seeks to merge abstractive text summarization with LLMs and RAG technology to develop a unique approach for extracting insights from research papers.

? References

1. Wu Minghao et al. "Lamini-lm: A diverse herd of distilled models from large-scale instructions." arXiv preprint arXiv:2304.14402 (2023).
2. Lewis Patrick et al. "Retrieval-augmented generation for knowledge-intensive NLP tasks." Advances in Neural Information Processing Systems 33 (2020): 9459-9474.
3. Nallapati Ramesh et al. "Abstractive text summarization using sequence-to-sequence rnns and beyond." arXiv preprint arXiv:1602.06023 (2016).

---

_{Project Contributors: Smit Shah, Mayur Bhanushali, Indrajeet Roy}