albert zh for pytorch transformers

The original reference author has provided the model download and test results

https://github.com/lonePatient/albert_pytorch/blob/master/README_zh.md

Albert-zh for pytorch-transformers

• Stop updating
• Just convert based on reference and then step on the mine
• Albert zh for pytorch-transformers
• Test support for traditional Chinese

Available models

• albert_tiny_zh
• albert_base_zh
• albert_large_zh
• albert_xlarge_zh

API

First place albert_zh in this repo under your project

from albert_zh import ...

 AlbertConfig
AlbertTokenizer
AlbertModel
AlbertForMaskedLM
AlbertForQuestionAnswering
AlbertForSequenceClassification

https://huggingface.co/transformers/v2.3.0/model_doc/albert.html

How to use

• See usage_example.py

  Or refer to the actual usage example of p208p2002/taipei-QA-BERT

• Testing works properly in transformers 2.3.0

Frequently Asked Questions

I want to introduce it in jupyter and colab but I have problems

This repo naming does not comply with the python module naming convention, and jupyter itself does not have good support for customized modules. Please refer to the solution example below. Consider pushing pypi later

 # 此段code僅適用於jupyter、colab
!git clone https://github.com/p208p2002/albert-zh-for-pytorch-transformers.git albert
import sys 
sys.path.append('.')
from albert.albert_zh import AlbertConfig, AlbertTokenizer, AlbertForSequenceClassification

Loss cannot be reduced, training becomes garbage

Make sure model class and model config are introduced by albert_zh, not transformers

lonePatient/albert_pytorch#35

AttributeError: 'BertConfig' object has no attribute 'share_type'

config.json adds "share_type":"all"

The model prints things during training

Please use log() instead of print() , and execute blockPrint() once at the beginning of the program

 import os , sys
def log ( * logs ):
    enablePrint ()
    print ( * logs )
    blockPrint ()

# Disable
def blockPrint ():
    sys . stdout = open ( os . devnull , 'w' )

# Restore
def enablePrint ():
    sys . stdout = sys . __stdout__ 

Test environment

• python 3.6.4
• pyotrch 1.3 (with cuda 10)
• transformers 2.3.0

refer to

albert zh

• https://github.com/brightmart/albert_zh

albert tf to pytorch

• https://github.com/lonePatient/albert_pytorch