rasa_nlu_gq下載rasa_nlu_gq源代碼下載

rasa_nlu_gq

其他源碼

1.0.0

下載

Rasa NLU GQ

Rasa NLU (Natural Language Understanding) 是一個自然語義理解的工具，舉個官網的例子如下：

"I'm looking for a Mexican restaurant in the center of town"

And returning structured data like:

  intent: search_restaurant
  entities: 
    - cuisine : Mexican
    - location : center

Introduction

原來的項目在分支0.2.7上，可自由切換。這個版本的修改是基於最新版本的rasa，將原來rasa_nlu_gao裡面的component修改了下，並沒有做新增。並且之前做法有些累贅，並不需要在rasa源碼中修改。可以直接將原來的component當做addon加載，繼承最新版本的rasa，可實時更新。

New features

目前新增的特性如下（請下載最新的rasa-nlu-gao版本）(edit at 2019.06.24)：

新增了實體識別的模型，一個是bilstm+crf，一個是idcnn+crf膨脹卷積模型，對應的yml文件配置如下：

  language: "zh"

  pipeline:
  - name: "JiebaTokenizer"
  - name: "CountVectorsFeaturizer"
    token_pattern: "(?u)bw+b"
  - name: "EmbeddingIntentClassifier"
  - name: "rasa_nlu_gao.extractors.bilstm_crf_entity_extractor.BilstmCRFEntityExtractor"
    lr: 0.001
    char_dim: 100
    lstm_dim: 100
    batches_per_epoch: 10
    seg_dim: 20
    num_segs: 4
    batch_size: 200
    tag_schema: "iobes"
    model_type: "bilstm" # 模型支持两种idcnn膨胀卷积模型或bilstm双向lstm模型
    clip: 5
    optimizer: "adam"
    dropout_keep: 0.5
    steps_check: 100

新增了jieba詞性標註的模塊，可以方便識別名字，地名，機構名等等jieba能夠支持的詞性，對應的yml文件配置如下：

  language: "zh"

  pipeline:
  - name: "JiebaTokenizer"
  - name: "CRFEntityExtractor"
  - name: "rasa_nlu_gao.extractors.jieba_pseg_extractor.JiebaPsegExtractor"
    part_of_speech: ["nr", "ns", "nt"]
  - name: "CountVectorsFeaturizer"
    OOV_token: oov
    token_pattern: "(?u)bw+b"
  - name: "EmbeddingIntentClassifier"

新增了根據實體反向修改意圖，對應的文件配置如下：

  language: "zh"

  pipeline:
  - name: "JiebaTokenizer"
  - name: "CRFEntityExtractor"
  - name: "JiebaPsegExtractor"
  - name: "CountVectorsFeaturizer"
    OOV_token: oov
    token_pattern: '(?u)bw+b'
  - name: "EmbeddingIntentClassifier"
  - name: "rasa_nlu_gao.classifiers.entity_edit_intent.EntityEditIntent"
    entity: ["nr"]
    intent: ["enter_data"]
    min_confidence: 0

新增了bert模型提取詞向量特徵，對應的配置文件如下：

  language: "zh"

  pipeline:
  - name: "JiebaTokenizer"
  - name: "rasa_nlu_gao.featurizers.bert_vectors_featurizer.BertVectorsFeaturizer"
    ip: '127.0.0.1'
    port: 5555
    port_out: 5556
    show_server_config: True
    timeout: 10000
  - name: "EmbeddingIntentClassifier"
  - name: "CRFEntityExtractor"

新增了對CPU和GPU的利用率的配置，主要是EmbeddingIntentClassifier和ner_bilstm_crf這兩個使用到tensorflow的組件，配置如下（當然config_proto可以不配置，默認值會將資源全部利用）：

  language: "zh"

  pipeline:
  - name: "JiebaTokenizer"
  - name: "CountVectorsFeaturizer"
    token_pattern: '(?u)bw+b'
  - name: "EmbeddingIntentClassifier"
    config_proto: {
      "device_count": 4,
      "inter_op_parallelism_threads": 0,
      "intra_op_parallelism_threads": 0,
      "allow_growth": True
    }
  - name: "rasa_nlu_gao.extractors.bilstm_crf_entity_extractor.BilstmCRFEntityExtractor"
    config_proto: {
      "device_count": 4,
      "inter_op_parallelism_threads": 0,
      "intra_op_parallelism_threads": 0,
      "allow_growth": True
    }

新增了embedding_bert_intent_classifier分類器，對應的配置文件如下：

  language: "zh"

  pipeline:
  - name: "JiebaTokenizer"
  - name: "rasa_nlu_gao.featurizers.bert_vectors_featurizer.BertVectorsFeaturizer"
    ip: '127.0.0.1'
    port: 5555
    port_out: 5556
    show_server_config: True
    timeout: 10000
  - name: "rasa_nlu_gao.classifiers.embedding_bert_intent_classifier.EmbeddingBertIntentClassifier"
  - name: "CRFEntityExtractor"

在基礎詞向量使用bert的情況下，後端的分類器使用tensorflow高級api完成，tf.estimator,tf.data,tf.example,tf.saved_model intent_estimator_classifier_tensorflow_embedding_bert分類器，對應的配置文件如下：

 language: "zh"

pipeline:
- name: "JiebaTokenizer"
- name: "rasa_nlu_gao.featurizers.bert_vectors_featurizer.BertVectorsFeaturizer"
  ip: '127.0.0.1'
  port: 5555
  port_out: 5556
  show_server_config: True
  timeout: 10000
- name: "rasa_nlu_gao.classifiers.embedding_bert_intent_estimator_classifier.EmbeddingBertIntentEstimatorClassifier"
- name: "SpacyNLP"
- name: "CRFEntityExtractor"