Auto_Standard_Generator

Auto Standard Generator: A Fine-Tuning Practice

This repository contains the code and data for Auto Standard Generator, a first-year project developed by a team of five undergraduate students at the Harbin Institute of Technology.

---

? News

• [2024.3.25] ??Gather Raw Data: scraped PDF technical standards.
• [2024.3.25] ??Cleaning & Preprocessing: batch processed text for dataset.
• [2024.4.23] ??Data Cleaning: removed garbled text.

TODO

• Labeling the Data: build instruction data for fine-tuning.
• Fine-tuning
• Acceleration & Compression: employ techniques of model acceleration and compression to reduce the inference latency and memory cost.
• GUI development: develop users interface.

---

Brief Introduction

Fine-tuning pre-trained general large language models (LLMs) on a smaller dataset of labeled data specific to a particular task allows the models to learn the specific features that are important for that task, thus improving its performance.

This project focuses on fine-tuning an open-source general LLM, such as Llama 2 or ChatGLM 3, to specialize it in generating technical standards documents.

---

Getting Started

1. Environment Preparation

2. Dataset Building

3. Supervised Fine-Tuning