nlp-papernotes
Reading NLP-related papers when working, summary and thoughts. Major in Machine Reading Comprehension (MRC), Question Answering (QA), Multi-document MRC, Information Retriever (IR) and Ranking.
Basic Concepts
- LSA, references in Wikipedia, An Introduction to Latent Semantic Analysis, Indexing by Latent Semantic Analysis, CUHK Latent Semantic Indexing (LSI) An Example, CUHK PPT, Stanford NLP IR-book latent-semantic-indexing.
- LDA, references in Wikipedia, LDA Topic Modeling: An Explanation, Topic Modeling and Latent Dirichlet Allocation (LDA) in Python, Paper Latent Dirichlet Allocation, Your Guide to Latent Dirichlet Allocation, Topic modeling using Latent Dirichlet Allocation(LDA) and Gibbs Sampling explained!, Topic Modeling with Gensim (Python).
- SVD, references in Wikipedia,
Language Model
Visiting Tomas Mikolov Google Scholar to get related papers.
Other papers list:
- A Neural Probabilistic Language Model
BERT-style Models, Summary PPT.
- BERT
- XLNet
- structBERT
- spanBERT
- SemBERT
- RoBERTa
- ALBERT
MRC & QA
Summary PPT.
ACL 2019
Knowledge Distillation (KD)
KD is similar to Label Smoothing, introduction of KD from the following blogs:
- Neural Network Distiller
- Smaller, faster, cheaper, lighter: Introducing DistilBERT, a distilled version of BERT
- Tap into the dark knowledge using neural nets — Knowledge distillation
- Knowledge Distillation
MS MARCO
MS MARCO LeaderBoard, several public paper models, written on Github MSMARCO-MRC-Analysis
Passage Retrieval
- Duet: [arxiv], [PPT], [notes]
- Duet V2
Optimizer
Bookshelf
- 《Deep Learning》
- 《Deep Learning (中文版)》
- 《Mathematics for Machine Learning》
- 《Learning to rank for information retrieval》
- 《GDL-Generative Deep Learning》
- 《Introduction to Information Retrieval》
- 《word2vec math theories》
- 《TensorFlow2深度学习》 [code]
