RaLLe

RᴀLLᴇ

RᴀLLᴇ is an accessible framework for developing and evaluating retrieval-augmented large language models (R-LLMs).
An overview of the main uses of RᴀLLᴇ on GUI is presented in this video.

RᴀLLᴇ is developed at the Institute of Memory Technology Research and Development of Kioxia Corporation.

Key Features

• Easy development and testing: users can easily select, combine, and test various retrievers and LLMs, including open-source models, within a graphical interface.
• Objective evaluation of R-LLMs: RᴀLLᴇ provides reproducible experiments with objective benchmarks/metrics, enabling objective assessments of R-LLM performance.
• Transparent prompt engineering: all input (prompts) and output of an LLM are visible to the user, allowing for easy exploration and optimization of prompts.

Usage

Getting Started:

• Installation instruction: INSTALL.md.
• Document indexing: here.
• Using custom datasets: here.

Using RᴀLLᴇ:

• Guides on GUI: here.
• Evaluation with a Python script: here.
• Review the evaluation results with MLflow: here

Note: evaluation experiments can be performed both through the GUI and using the script.

Reference

News: Our paper has been accepted by EMNLP 2023 System Demonstrations.

Reference to cite when you use RᴀLLᴇ in a research paper:

@misc{ralle,
      title={RaLLe: A Framework for Developing and Evaluating Retrieval-Augmented Large Language Models}, 
      author={Yasuto Hoshi and Daisuke Miyashita and Youyang Ng and Kento Tatsuno and Yasuhiro Morioka and Osamu Torii and Jun Deguchi},
      url={https://arxiv.org/abs/2308.10633},
      year={2023},
      eprint={2308.10633},
      publisher={arXiv}
}

License

RᴀLLᴇ is MIT-licensed, refer to the LICENSE file for more details.