lm question generation
0.0.0
圖1：三種不同的QAG方法。
lmqg是使用語言模型（LMS）的問答生成（QAG）的Python庫。在這裡，我們考慮段落級別的QAG，用戶將提供上下文（段落或文檔），該模型將在上下文上生成問答對列表。使用lmqg ，您可以做以下操作：
2023年11月更新：中文QAG模型現在可在lmqg和AUTOQG上使用！
2023年5月更新：兩篇論文被ACL 2023接受（QAG查找，LMQG在系統演示中）。
2022年10月更新：我們的QG論文被EMNLP Main 2022接受。
我們的QAG模型可以分為三種類型：管道，多任務和End2end （見圖1）。管道由問題生成（QG）和答案提取（AE）模型獨立組成，AE將在上下文中解析所有句子以提取答案，QG將在答案上產生問題。多任務遵循與管道相同的架構，但是QG和AE模型共享模型共享。最後， End2end模型將以端到端的方式生成問答對列表。在實踐中，管道和多任務會產生更多的問答對，而end2end的產生速度卻更快，幾次，生成的問答對質量取決於語言。所有類型均以9種不同語言（en/fr/ja/ko/ru/it/es/de/zh）提供的lmqg種不同語言可用，並且模型都在HuggingFace上共享（請參閱模型卡）。要了解有關QAG的更多信息，請檢查我們的ACL 2023論文，該論文描述了QAG模型，並報告了每種語言中每個QAG模型的完整性能比較。
圖2：QAG（A）和QG（B）的示例。
所有功能也支持問題產生。我們的QG模型假設用戶除了上下文之外還可以指定答案，並且QG模型將產生一個問題，該問題是可以通過給定上下文的答案來回答的（有關QAG和QG的比較，請參見圖2）。要了解有關QG的更多信息，請查看我們的EMNLP 2022論文，該論文詳細描述了QG模型。
讓我們首先通過PIP安裝lmqg 。
pip install lmqg以幾行生成問答對。
from lmqg import TransformersQG
model = TransformersQG ( language = "en" )
context = "William Turner was an English painter who specialised in watercolour landscapes. He is often known "
"as William Turner of Oxford or just Turner of Oxford to distinguish him from his contemporary, "
"J. M. W. Turner. Many of Turner's paintings depicted the countryside around Oxford. One of his "
"best known pictures is a view of the city of Oxford from Hinksey Hill."
qa = model . generate_qa ( context )
print ( qa )
[
( 'Who was an English painter who specialised in watercolour landscapes?' , 'William Turner' ),
( 'What is William Turner often known as?' , 'William Turner of Oxford or just Turner of Oxford' ),
( "What did many of Turner's paintings depict?" , 'the countryside around Oxford' ),
( "What is one of Turner's best known pictures?" , 'a view of the city of Oxford from Hinksey Hill' )
]除了en外，我們還支持意大利語， es it ，俄羅斯ru ，韓國ko ，日本ja ，德國de ，French fr和中國zh 。您可以通過在加載模型（例如，對於西班牙語）指定語言ID（例如TransformersQG(language="es") ）來切換語言。有關更詳細的用法，請閱讀下一節。
lmqg的主要功能是用方便的API在給定上下文上生成問答對。每個QAG類的可用型號可以在模型卡上找到。
from pprint import pprint
from lmqg import TransformersQG
# initialize model
model = TransformersQG ( 'lmqg/t5-base-squad-qag' ) # or TransformersQG(model='lmqg/t5-base-squad-qg-ae')
# paragraph to generate pairs of question and answer
context = "William Turner was an English painter who specialised in watercolour landscapes. He is often known "
"as William Turner of Oxford or just Turner of Oxford to distinguish him from his contemporary, "
"J. M. W. Turner. Many of Turner's paintings depicted the countryside around Oxford. One of his "
"best known pictures is a view of the city of Oxford from Hinksey Hill."
# model prediction
question_answer = model . generate_qa ( context )
# the output is a list of tuple (question, answer)
pprint ( question_answer )
[
( 'Who was an English painter who specialised in watercolour landscapes?' , 'William Turner' ),
( 'What is William Turner often known as?' , 'William Turner of Oxford or just Turner of Oxford' ),
( "What did many of Turner's paintings depict?" , 'the countryside around Oxford' ),
( "What is one of Turner's best known pictures?" , 'a view of the city of Oxford from Hinksey Hill' )
]model和model_ae分別是QG和AE模型。 from pprint import pprint
from lmqg import TransformersQG
# initialize model
model = TransformersQG ( model = 'lmqg/t5-base-squad-qg' , model_ae = 'lmqg/t5-base-squad-ae' )
# paragraph to generate pairs of question and answer
context = "William Turner was an English painter who specialised in watercolour landscapes. He is often known "
"as William Turner of Oxford or just Turner of Oxford to distinguish him from his contemporary, "
"J. M. W. Turner. Many of Turner's paintings depicted the countryside around Oxford. One of his "
"best known pictures is a view of the city of Oxford from Hinksey Hill."
# model prediction
question_answer = model . generate_qa ( context )
# the output is a list of tuple (question, answer)
pprint ( question_answer )
[
( 'Who was an English painter who specialised in watercolour landscapes?' , 'William Turner' ),
( 'What is another name for William Turner?' , 'William Turner of Oxford' ),
( "What did many of William Turner's paintings depict around Oxford?" , 'the countryside' ),
( 'From what hill is a view of the city of Oxford taken?' , 'Hinksey Hill.' )
]model是QG模型。有關可用QG型號的列表，請參見QG-Bench，這是一種多語言QG基準。 from pprint import pprint
from lmqg import TransformersQG
# initialize model
model = TransformersQG ( model = 'lmqg/t5-base-squad-qg' )
# a list of paragraph
context = [
"William Turner was an English painter who specialised in watercolour landscapes" ,
"William Turner was an English painter who specialised in watercolour landscapes"
]
# a list of answer (same size as the context)
answer = [
"William Turner" ,
"English"
]
# model prediction
question = model . generate_q ( list_context = context , list_answer = answer )
pprint ( question )
[
'Who was an English painter who specialised in watercolour landscapes?' ,
'What nationality was William Turner?'
]model是QG模型。 from pprint import pprint
from lmqg import TransformersQG
# initialize model
model = TransformersQG ( model = 'lmqg/t5-base-squad-ae' )
# model prediction
answer = model . generate_a ( "William Turner was an English painter who specialised in watercolour landscapes" )
pprint ( answer )
[ 'William Turner' ]AutoQG（https://autoqg.net）是一個免費的Web應用程序，託管了我們的QAG模型。
lmqg還提供了一個命令行接口來微調和評估QG，AE和QAG模型。
為了微調QG（或AE，QAG）模型，我們採用了兩階段的超參數優化，如上圖所述。以下命令是通過參數優化運行微調。
lmqg-train-search -c " tmp_ckpt " -d " lmqg/qg_squad " -m " t5-small " -b 64 --epoch-partial 5 -e 15 --language " en " --n-max-config 1
-g 2 4 --lr 1e-04 5e-04 1e-03 --label-smoothing 0 0.15檢查lmqg-train-search -h以顯示所有選項。
Python中的微調模型如下。
from lmqg import GridSearcher
trainer = GridSearcher (
checkpoint_dir = 'tmp_ckpt' ,
dataset_path = 'lmqg/qg_squad' ,
model = 't5-small' ,
epoch = 15 ,
epoch_partial = 5 ,
batch = 64 ,
n_max_config = 5 ,
gradient_accumulation_steps = [ 2 , 4 ],
lr = [ 1e-04 , 5e-04 , 1e-03 ],
label_smoothing = [ 0 , 0.15 ]
)
trainer . run ()評估工具報告了QG基礎後的BLEU4 ， ROUGE-L ， METEOR ， BERTScore和MoverScore 。從命令行，運行以下命令
lmqg-eval -m " lmqg/t5-large-squad-qg " -e " ./eval_metrics " -d " lmqg/qg_squad " -l " en "其中-m是在擁抱面或通往本地檢查點的路徑上的模型別名， -e是直接導出度量文件的模型， -d是要評估的數據集， -l是測試集的語言。您可以提供一個預測文件，而不是運行模型預測，以避免每次計算。
lmqg-eval --hyp-test ' {your prediction file} ' -e " ./eval_metrics " -d " lmqg/qg_squad " -l " en "預測文件應在目標數據集（示例）中以test拆分的順序在每行中的模型生成中的文本文件。檢查lmqg-eval -h以顯示所有選項。
最後， lmqg提供了一個REST API，該API通過HuggingFace推斷API託管模型推理。您需要擁抱面API令牌才能運行自己的API並按照以下方式安裝依賴項。
pip install lmqg[api] Swagger UI可在http://127.0.0.1:8088/docs上找到，當您本地運行該應用程序時（用服務器地址替換地址）。
export API_TOKEN={Your Huggingface API Token}
uvicorn app:app --host 0.0.0.0 --port 8088docker build -t lmqg/app:latest . --build-arg api_token={Your Huggingface API Token}
docker run -p 8080:8080 lmqg/app:latestuvicorn app_local:app --host 0.0.0.0 --port 8088您必須通過環境變量API_TOKEN傳遞擁抱面API令牌。主要端點是question_generation ，它具有以下參數，
| 範圍 | 描述 |
|---|---|
| input_text | 輸入文本，段落或句子生成問題 |
| 語言 | 語言 |
| qg_model | 問題生成模型 |
| 答案_MODEL | 回答提取模型 |
並返回一個question和answer的詞典列表。
{
" qa " : [
{ " question " : " Who founded Nintendo Karuta? " , " answer " : " Fusajiro Yamauchi " },
{ " question " : " When did Nintendo distribute its first video game console, the Color TV-Game? " , " answer " : " 1977 " }
]
}如果您使用任何資源，請引用以下論文，並在需要時查看代碼以復制模型。
@inproceedings{ushio-etal-2022-generative,
title = "{G}enerative {L}anguage {M}odels for {P}aragraph-{L}evel {Q}uestion {G}eneration",
author = "Ushio, Asahi and
Alva-Manchego, Fernando and
Camacho-Collados, Jose",
booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing",
month = dec,
year = "2022",
address = "Abu Dhabi, U.A.E.",
publisher = "Association for Computational Linguistics",
}
@inproceedings{ushio-etal-2023-an-empirical,
title = "An Empirical Comparison of LM-based Question and Answer Generation Methods",
author = "Ushio, Asahi and
Alva-Manchego, Fernando and
Camacho-Collados, Jose",
booktitle = "Proceedings of the 61th Annual Meeting of the Association for Computational Linguistics: Findings",
month = Jul,
year = "2023",
address = "Toronto, Canada",
publisher = "Association for Computational Linguistics",
}
@inproceedings{ushio-etal-2023-a-practical-toolkit,
title = "A Practical Toolkit for Multilingual Question and Answer Generation",
author = "Ushio, Asahi and
Alva-Manchego, Fernando and
Camacho-Collados, Jose",
booktitle = "Proceedings of the 61th Annual Meeting of the Association for Computational Linguistics: System Demonstrations",
month = Jul,
year = "2023",
address = "Toronto, Canada",
publisher = "Association for Computational Linguistics",
}
