面向深度学习研究人员的自然语言处理实例教程

nlp-tutorial是一个使用Pytorch／TensorFlow，学习自然语言处理的教程。大多数模型的代码行数少于100行。

旧的TensorFlow v1代码存档在存档文件夹中，适合初学者。

环境依赖：

• Python 3.5+
• Pytorch 1.0.0+

代码和数据集 获取方式

关注微信公众号 datayx 然后回复 NLP 即可获取。

AI项目体验地址 https://loveai.tech

1. Basic Embedding Model

• 1-1. NNLM(Neural Network Language Model) - Predict Next Word
  • Paper - A Neural Probabilistic Language Model(2003)
  • Colab - NNLM.ipynb
• 1-2. Word2Vec(Skip-gram) - Embedding Words and Show Graph
  • Paper - Distributed Representations of Words and Phrases and their Compositionality(2013)
  • Colab - Word2Vec.ipynb
• 1-3. FastText(Application Level) - Sentence Classification
  • Paper - Bag of Tricks for Efficient Text Classification(2016)
  • Colab - FastText.ipynb

2. CNN(Convolutional Neural Network)

• 2-1. TextCNN - Binary Sentiment Classification
  • Paper - Convolutional Neural Networks for Sentence Classification(2014)
  • TextCNN.ipynb

3. RNN(Recurrent Neural Network)

• 3-1. TextRNN - Predict Next Step
  • Paper - Finding Structure in Time(1990)
  • Colab - TextRNN.ipynb
• 3-2. TextLSTM - Autocomplete
  • Paper - LONG SHORT-TERM MEMORY(1997)
  • Colab - TextLSTM.ipynb
• 3-3. Bi-LSTM - Predict Next Word in Long Sentence
  • Colab - Bi_LSTM.ipynb

4. Attention Mechanism

• 4-1. Seq2Seq - Change Word
  • Paper - Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation(2014)
  • Colab - Seq2Seq.ipynb
• 4-2. Seq2Seq with Attention - Translate
  • Paper - Neural Machine Translation by Jointly Learning to Align and Translate(2014)
  • Colab - Seq2Seq(Attention).ipynb
• 4-3. Bi-LSTM with Attention - Binary Sentiment Classification
  • Colab - Bi_LSTM(Attention).ipynb

5. Model based on Transformer

• 5-1. The Transformer - Translate
  • Paper - Attention Is All You Need(2017)
  • Colab - Transformer.ipynb, Transformer(Greedy_decoder).ipynb
• 5-2. BERT - Classification Next Sentence & Predict Masked Tokens
  • Paper - BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding(2018)
  • Colab - BERT.ipynb