How to use Transformers

Transformers is a Python library developed by Hugging Face, which supports loading most of the pre-trained language models. With the rise of models such as BERT and GPT, more and more users are using the Transformers library to build natural language processing applications.

This project is a code repository for the "Transformers Library Quick Start" tutorial, and the code is organized as follows:

• data : stores the used data set;
• src : Stores sample code, each task corresponds to a folder, which can be downloaded and used separately.

This tutorial is under update and is gradually adding related content to the large language model.

Quick Start of Transformers Library

• Part One: Background Knowledge

  • Chapter 1: Natural Language Processing
  • Chapter 2: Transformer Model
  • Chapter 3: Attention Mechanism

• Part 2: First meeting with Transformers

  • Chapter 4: Pipelines out of the box
  • Chapter 5: Model and Word Partitioner
  • Chapter 6: Necessary Pytorch Knowledge
  • Chapter 7: Fine-tuning the pre-trained model

• Part 3: Transformers Practical Battle

  • Chapter 8: Quick Word Partitioner
  • Chapter 9: Sequence Annotation Task
  • Chapter 10: Translation Task
  • Chapter 11: Text Summary Task
  • Chapter 12: Extracted Q&A
  • Chapter 13: Prompting Sentiment Analysis

• Part 4: The Era of Big Language Model

  • Chapter 14: Introduction to Big Language Model Technology
  • Chapter 15: Pre-training large language model
  • Chapter 16: Using Big Language Model
  • Chapter 17: Instruction Fine Tunnel FlanT5 Model
  • Chapter 18: Instruction fine-tuning Llama2 model

Sample code

• pairwise_cls_similarity_afqmc: Sentence pair classification tasks, financial synonyms judgment.
• sequence_labeling_ner_cpd: Sequence labeling task, named entity recognition.
• seq2seq_translation: seq2seq task, Chinese-English translation.
• seq2seq_summarization: seq2seq task, text summary.
• sequence_labeling_extractiveQA_cmrc: Sequence labeling task, extracted question and answer.
• text_cls_prompt_senti_chnsenticorp: text classification task, Prompt sentiment analysis.

Important updates

date	content
2024-07-06	The textual expression of the first chapter of "Natural Language Processing" was optimized, some pictures were added, and an introduction to the large language model was added.
2024-07-27	Complete the preliminary draft of the technology introduction of large language model (Chapters 14 to 16)