首頁>編程相關>其他源碼
下載

ark-nlp

ark-nlp主要是收集和復現學術與工作中常用的NLP模型

環境

  • python 3
  • torch >= 1.0.0, <1.10.0
  • tqdm >= 4.56.0
  • jieba >= 0.42.1
  • transformers >= 3.0.0
  • zhon >= 1.1.5
  • scipy >= 1.2.0
  • scikit-learn >= 0.17.0

pip安裝

 pip install --upgrade ark-nlp

項目結構

ark_nlp開源的自然語言處理庫
ark_nlp.dataset封裝數據加載、處理和轉化等功能
ark_nlp.nn封裝一些完整的神經網絡模型
ark_nlp.processor封裝分詞器、詞典和構圖器等
ark_nlp.factory封裝損失函數、優化器、訓練和預測等功能
ark_nlp.model按實際NLP任務封裝常用的模型，方便調用

實現的模型

預訓練模型

模型參考文獻
BERT BERT:Pre-training of Deep Bidirectional Transformers for Language Understanding
ERNIE1.0 ERNIE:Enhanced Representation through Knowledge Integration
NEZHA NEZHA:Neural Contextualized Representation For Chinese Language Understanding
Roformer Roformer: Enhanced Transformer with Rotary Position Embedding
ERNIE-CTM ERNIE-CTM（ERNIE for Chinese Text Mining）

文本分類(Text Classification)

模型簡介
RNN/CNN/GRU/LSTM經典的RNN, CNN, GRU, LSTM等經典文本分類結構
BERT/ERNIE常用的預訓練模型分類

文本匹配(Text Matching)

模型簡介
BERT/ERNIE常用的預訓練模型匹配分類
UnsupervisedSimcse無監督Simcse匹配算法
CoSENT CoSENT：比Sentence-BERT更有效的句向量方案

命名實體識別(Named Entity Recognition)

模型參考文獻論文源碼
CRF BERT
Biaffine BERT
Span BERT
Global Pointer BERT GlobalPointer：用統一的方式處理嵌套和非嵌套NER
Efficient Global Pointer BERT Efficient GlobalPointer：少點參數，多點效果
W2NER BERT Unified Named Entity Recognition as Word-Word Relation Classification github

關係抽取(Relation Extraction)

模型參考文獻論文源碼
Casrel A Novel Cascade Binary Tagging Framework for Relational Triple Extraction github
PRGC PRGC: Potential Relation and Global Correspondence Based Joint Relational Triple Extraction github

信息抽取(Information Extraction)

模型參考文獻論文源碼
PromptUie通用信息抽取UIE(Universal Information Extraction) github

少樣本(Few-Shot Learning)

模型參考文獻論文源碼
PromptBert Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing)

實際應用

  • CHIP2021-Task3-臨床術語標準化任務-第三名
  • CHIP2021-Task1-醫學對話臨床發現陰陽性判別任務-第一名
  • 中文醫療信息處理挑戰榜CBLUE

使用例子

完整代碼可參考test文件夾

  • 文本分類

     import torch
import pandas as pd

from ark_nlp . model . tc . bert import Bert
from ark_nlp . model . tc . bert import BertConfig
from ark_nlp . model . tc . bert import Dataset
from ark_nlp . model . tc . bert import Task
from ark_nlp . model . tc . bert import get_default_model_optimizer
from ark_nlp . model . tc . bert import Tokenizer

# 加载数据集
# train_data_df的columns必选包含"text"和"label"
# text列为文本，label列为分类标签
tc_train_dataset = Dataset ( train_data_df )
tc_dev_dataset = Dataset ( dev_data_df )

# 加载分词器
tokenizer = Tokenizer ( vocab = 'nghuyong/ernie-1.0' , max_seq_len = 30 )

# 文本切分、ID化
tc_train_dataset . convert_to_ids ( tokenizer )
tc_dev_dataset . convert_to_ids ( tokenizer )

# 加载预训练模型
config = BertConfig . from_pretrained ( 'nghuyong/ernie-1.0' ,
                                   num_labels = len ( tc_train_dataset . cat2id ))
dl_module = Bert . from_pretrained ( 'nghuyong/ernie-1.0' , 
                                 config = config )

# 任务构建
num_epoches = 10
batch_size = 32
optimizer = get_default_model_optimizer ( dl_module )
model = Task ( dl_module , optimizer , 'ce' , cuda_device = 0 )

# 训练
model . fit ( tc_train_dataset , 
          tc_dev_dataset ,
          lr = 2e-5 ,
          epochs = 5 , 
          batch_size = batch_size
         )

# 推断
from ark_nlp . model . tc . bert import Predictor

tc_predictor_instance = Predictor ( model . module , tokenizer , tc_train_dataset . cat2id )

tc_predictor_instance . predict_one_sample (待预测文本)

  • 文本匹配

     import torch
import pandas as pd

from ark_nlp . model . tm . bert import Bert
from ark_nlp . model . tm . bert import BertConfig
from ark_nlp . model . tm . bert import Dataset
from ark_nlp . model . tm . bert import Task
from ark_nlp . model . tm . bert import get_default_model_optimizer
from ark_nlp . model . tm . bert import Tokenizer

# 加载数据集
# train_data_df的columns必选包含"text_a"、"text_b"和"label"
# text_a和text_b列为文本，label列为匹配标签
tm_train_dataset = Dataset ( train_data_df )
tm_dev_dataset = Dataset ( dev_data_df )

# 加载分词器
tokenizer = Tokenizer ( vocab = 'nghuyong/ernie-1.0' , max_seq_len = 30 )

# 文本切分、ID化
tm_train_dataset . convert_to_ids ( tokenizer )
tm_dev_dataset . convert_to_ids ( tokenizer )

# 加载预训练模型
config = BertConfig . from_pretrained ( 'nghuyong/ernie-1.0' , 
                                   num_labels = len ( tm_train_dataset . cat2id ))
dl_module = Bert . from_pretrained ( 'nghuyong/ernie-1.0' , 
                                 config = config )

# 任务构建
num_epoches = 10
batch_size = 32
optimizer = get_default_model_optimizer ( dl_module )
model = Task ( dl_module , optimizer , 'ce' , cuda_device = 0 )

# 训练
model . fit ( tm_train_dataset , 
          tm_dev_dataset ,
          lr = 2e-5 ,
          epochs = 5 , 
          batch_size = batch_size
         )

# 推断
from ark_nlp . model . tm . bert import Predictor

tm_predictor_instance = Predictor ( model . module , tokenizer , tm_train_dataset . cat2id )

tm_predictor_instance . predict_one_sample ([待预测文本A , 待预测文本B ])

  • 命名實體

     import torch
import pandas as pd

from ark_nlp . model . ner . crf_bert import CRFBert
from ark_nlp . model . ner . crf_bert import CRFBertConfig
from ark_nlp . model . ner . crf_bert import Dataset
from ark_nlp . model . ner . crf_bert import Task
from ark_nlp . model . ner . crf_bert import get_default_model_optimizer
from ark_nlp . model . ner . crf_bert import Tokenizer

# 加载数据集
# train_data_df的columns必选包含"text"和"label"
# text列为文本
# label列为列表形式，列表中每个元素是如下组织的字典
# {'start_idx': 实体首字符在文本的位置, 'end_idx': 实体尾字符在文本的位置, 'type': 实体类型标签, 'entity': 实体}
ner_train_dataset = Dataset ( train_data_df )
ner_dev_dataset = Dataset ( dev_data_df )

# 加载分词器
tokenizer = Tokenizer ( vocab = 'nghuyong/ernie-1.0' , max_seq_len = 30 )

# 文本切分、ID化
ner_train_dataset . convert_to_ids ( tokenizer )
ner_dev_dataset . convert_to_ids ( tokenizer )

# 加载预训练模型
config = CRFBertConfig . from_pretrained ( 'nghuyong/ernie-1.0' , 
                                  num_labels = len ( ner_train_dataset . cat2id ))
dl_module = CRFBert . from_pretrained ( 'nghuyong/ernie-1.0' , 
                                    config = config )

# 任务构建
num_epoches = 10
batch_size = 32
optimizer = get_default_model_optimizer ( dl_module )
model = Task ( dl_module , optimizer , 'ce' , cuda_device = 0 )

# 训练
model . fit ( ner_train_dataset , 
          ner_dev_dataset ,
          lr = 2e-5 ,
          epochs = 5 , 
          batch_size = batch_size
         )

# 推断
from ark_nlp . model . ner . crf_bert import Predictor

ner_predictor_instance = Predictor ( model . module , tokenizer , ner_train_dataset . cat2id )

ner_predictor_instance . predict_one_sample (待抽取文本)

  • Casrel關係抽取

     import torch
import pandas as pd

from ark_nlp . model . re . casrel_bert import CasRelBert
from ark_nlp . model . re . casrel_bert import CasRelBertConfig
from ark_nlp . model . re . casrel_bert import Dataset
from ark_nlp . model . re . casrel_bert import Task
from ark_nlp . model . re . casrel_bert import get_default_model_optimizer
from ark_nlp . model . re . casrel_bert import Tokenizer
from ark_nlp . factory . loss_function import CasrelLoss

# 加载数据集
# train_data_df的columns必选包含"text"和"label"
# text列为文本
# label列为列表形式，列表中每个元素是如下组织的字典
# [头实体, 头实体首字符在文本的位置, 头实体尾字符在文本的位置, 关系类型, 尾实体, 尾实体首字符在文本的位置, 尾实体尾字符在文本的位置]
re_train_dataset = Dataset ( train_data_df )
re_dev_dataset = Dataset ( dev_data_df ,
                         categories = re_train_dataset . categories ,
                         is_train = False )

# 加载分词器
tokenizer = Tokenizer ( vocab = 'nghuyong/ernie-1.0' , max_seq_len = 100 )

# 文本切分、ID化
# 注意：casrel的代码这部分其实并没有进行切分、ID化，仅是将分词器赋予dataset对象
re_train_dataset . convert_to_ids ( tokenizer )
re_dev_dataset . convert_to_ids ( tokenizer )

# 加载预训练模型
config = CasRelBertConfig . from_pretrained ( 'nghuyong/ernie-1.0' ,
                                          num_labels = len ( re_train_dataset . cat2id ))
dl_module = CasRelBert . from_pretrained ( 'nghuyong/ernie-1.0' , 
                                       config = config )

# 任务构建
num_epoches = 40
batch_size = 16
optimizer = get_default_model_optimizer ( dl_module )
model = Task ( dl_module , optimizer , CasrelLoss (), cuda_device = 0 )

# 训练
model . fit ( re_train_dataset , 
          re_dev_dataset ,
          lr = 2e-5 ,
          epochs = 5 , 
          batch_size = batch_size
         )

# 推断
from ark_nlp . model . re . casrel_bert import Predictor

casrel_re_predictor_instance = Predictor ( model . module , tokenizer , re_train_dataset . cat2id )

casrel_re_predictor_instance . predict_one_sample (待抽取文本)

  • PRGC關係抽取

     import torch
import pandas as pd

from ark_nlp . model . re . prgc_bert import PRGCBert
from ark_nlp . model . re . prgc_bert import PRGCBertConfig
from ark_nlp . model . re . prgc_bert import Dataset
from ark_nlp . model . re . prgc_bert import Task
from ark_nlp . model . re . prgc_bert import get_default_model_optimizer
from ark_nlp . model . re . prgc_bert import Tokenizer

# 加载数据集
# train_data_df的columns必选包含"text"和"label"
# text列为文本
# label列为列表形式，列表中每个元素是如下组织的字典
# [头实体, 头实体首字符在文本的位置, 头实体尾字符在文本的位置, 关系类型, 尾实体, 尾实体首字符在文本的位置, 尾实体尾字符在文本的位置]
re_train_dataset = Dataset ( train_df , is_retain_dataset = True )
re_dev_dataset = Dataset ( dev_df ,
                         categories = re_train_dataset . categories ,
                         is_train = False )

# 加载分词器
tokenizer = Tokenizer ( vocab = 'nghuyong/ernie-1.0' , max_seq_len = 100 )

# 文本切分、ID化
re_train_dataset . convert_to_ids ( tokenizer )
re_dev_dataset . convert_to_ids ( tokenizer )

# 加载预训练模型
config = PRGCBertConfig . from_pretrained ( 'nghuyong/ernie-1.0' ,
                                          num_labels = len ( re_train_dataset . cat2id ))
dl_module = PRGCBert . from_pretrained ( 'nghuyong/ernie-1.0' , 
                                       config = config )

# 任务构建
num_epoches = 40
batch_size = 16
optimizer = get_default_model_optimizer ( dl_module )
model = Task ( dl_module , optimizer , None , cuda_device = 0 )

# 训练
model . fit ( re_train_dataset , 
          re_dev_dataset ,
          lr = 2e-5 ,
          epochs = 5 , 
          batch_size = batch_size
         )

# 推断
from ark_nlp . model . re . prgc_bert import Predictor

prgc_re_predictor_instance = Predictor ( model . module , tokenizer , re_train_dataset . cat2id )

prgc_re_predictor_instance . predict_one_sample (待抽取文本)

DisscussionGroup

  • 公眾號： DataArk

wechat

  • wechat ID: fk95624

Main contributors


xiangking
Jimme
Zrealshadow

Acknowledge

本項目用於收集和復現學術與工作中常用的NLP模型，整合成方便調用的形式，所以參考借鑒了網上很多開源實現，如有不當的地方，還請聯繫批評指教。 在此，感謝大佬們的開源實現。

展開
附加信息
相關應用
爲您推薦
相關資訊 全部