Dive into DL PyTorch

This project changes the MXNet code implementation in the original book "Hand-On Learning Deep Learning" to PyTorch implementation. Original book authors: Aston Zhang, Li Mu, Zachary C. Lipton, Alexander J. Smora and other community contributors, GitHub address: https://github.com/d2l-ai/d2l-zh

There are some differences in the Chinese and English versions of this book. For the PyTorch refactoring of the English version of this book, please refer to this project. There are some differences between the Chinese and English versions of this book. For the PyTorch modifying of the English version, you can refer to this repo.

Introduction

This repository mainly contains two folders: code and docs (plus some data is stored in data). The code folder is the relevant jupyter notebook code for each chapter (based on PyTorch); the docs folder is the relevant content in the "Hand-On Deep Learning" book in markdown format, and then use docsify to deploy the web document to GitHub Pages. Since the original book uses the MXNet framework, the docs content may be slightly different from the original book, but the overall content is the same. Contributions to this project or issue is welcome.

For crowds

This project is aimed at children's shoes that are interested in deep learning, especially those who want to use PyTorch for deep learning. This project does not require you to have any background knowledge of deep learning or machine learning. You only need to understand basic mathematics and programming, such as basic linear algebra, differential and probability, and basic Python programming.

How to eat

Method 1

This repository contains some latex formulas, but github's markdown native does not support formula display, and the docs folder has been deployed on GitHub Pages using docsify, so the easiest way to view the document is to directly access the web version of this project. Of course, if you still want to run the relevant code, you still have to clone the project and then run the relevant code in the code folder.

Method 2

You can also access the documentation locally and install docsify-cli tool first:

npm i docsify-cli -g

Then clone the project locally:

git clone https://github.com/ShusenTang/Dive-into-DL-PyTorch.git
cd Dive-into-DL-PyTorch

Then run a local server, so that you can easily access the document web rendering effect in real time at http://localhost:3000 .

docsify serve docs

Method 3

If you don't want to install the docsify-cli tool, and even Node.js is not installed on your computer, and for some reason you want to browse documents locally, you can run the web service in docker container.

First, clone this project to the local area:

git clone https://github.com/ShusenTang/Dive-into-DL-PyTorch.git
cd Dive-into-DL-PyTorch

Then use the following command to create a docker image named "d2dl":

docker build -t d2dl .

After the image is created, run the following command to create a new container:

docker run -dp 3000:3000 d2dl

Finally, open this address http://localhost:3000/#/ in your browser to enjoy accessing the document. Suitable for those who don't want to install too many tools on their computers.

Table of contents

• Introduction
• Reading Guide
• 1. Introduction to Deep Learning
• 2. Preparation knowledge
  • 2.1 Environment configuration
  • 2.2 Data Operation
  • 2.3 Automatically find gradients
• 3. Basics of Deep Learning
  • 3.1 Linear Regression
  • 3.2 Implementation of linear regression from scratch
  • 3.3 A concise implementation of linear regression
  • 3.4 softmax regression
  • 3.5 Image classification dataset (Fashion-MNIST)
  • 3.6 Implementation of softmax regression from scratch
  • 3.7 A simple implementation of softmax regression
  • 3.8 Multi-layer sensing machine
  • 3.9 Implementation of multi-layer perception machine from scratch
  • 3.10 Simple implementation of multi-layer perception machine
  • 3.11 Model selection, underfitting and overfitting
  • 3.12 Weight decay
  • 3.13 Discarding method
  • 3.14 Forward propagation, back propagation and calculation diagrams
  • 3.15 Numerical stability and model initialization
  • 3.16 Actual Kaggle Competition: House Price Prediction
• 4. Deep Learning Computing
  • 4.1 Model construction
  • 4.2 Access, initialization and sharing of model parameters
  • 4.3 Delay initialization of model parameters
  • 4.4 Custom layer
  • 4.5 Reading and storage
  • 4.6 GPU computing
• 5. Convolutional neural network
  • 5.1 Two-dimensional convolution layer
  • 5.2 Fill and stride
  • 5.3 Multi-input channel and multi-output channel
  • 5.4 Pooling layer
  • 5.5 Convolutional Neural Network (LeNet)
  • 5.6 Deep Convolutional Neural Network (AlexNet)
  • 5.7 Networks using repeating elements (VGG)
  • 5.8 Network in the network (NiN)
  • 5.9 Network with parallel connection (GoogLeNet)
  • 5.10 Batch Normalization
  • 5.11 Residual Network (ResNet)
  • 5.12 DenseNet
• 6. Recurrent neural network
  • 6.1 Language Model
  • 6.2 Recurrent neural network
  • 6.3 Language Model Dataset (Lyrics of Jay Chou’s Album)
  • 6.4 Implementation of recurrent neural networks from scratch
  • 6.5 Simple implementation of recurrent neural network
  • 6.6 Backpropagation through time
  • 6.7 Gated Circulation Unit (GRU)
  • 6.8 Long Short-term Memory (LSTM)
  • 6.9 Deep recurrent neural network
  • 6.10 Bidirectional recurrent neural network
• 7. Optimization algorithm
  • 7.1 Optimization and Deep Learning
  • 7.2 Gradient descent and stochastic gradient descent
  • 7.3 Small batch stochastic gradient descent
  • 7.4 Momentum method
  • 7.5 AdaGrad Algorithm
  • 7.6 RMSProp Algorithm
  • 7.7 AdaDelta Algorithm
  • 7.8 Adam Algorithm
• 8. Computational Performance
  • 8.1 Imperative and Symbol Hybrid Programming
  • 8.2 Asynchronous calculation
  • 8.3 Automatic parallel computing
  • 8.4 Multi-GPU computing
• 9. Computer Vision
  • 9.1 Image augmentation
  • 9.2 Fine adjustment
  • 9.3 Object detection and bounding box
  • 9.4 Anchor frame
  • 9.5 Multi-scale target detection
  • 9.6 Object Detection Dataset (Pikachu)
  • 9.7 Single-shot multi-frame detection (SSD)
  • 9.8 Regional Convolutional Neural Network (R-CNN) Series
  • 9.9 Semantic segmentation and datasets
  • 9.10 Full Convolutional Network (FCN)
  • 9.11 Style Migration
  • 9.12 Practical Kaggle Competition: Image Classification (CIFAR-10)
  • 9.13 Practical Kaggle Competition: Dog Breed Identification (ImageNet Dogs)
• 10. Natural Language Processing
  • 10.1 Word embedding (word2vec)
  • 10.2 Approximate training
  • 10.3 Implementation of word2vec
  • 10.4 Subword embedding (fastText)
  • 10.5 Word embedding of global vectors (GloVe)
  • 10.6 Find synonyms and analogies
  • 10.7 Text emotion classification: using recurrent neural networks
  • 10.8 Text emotion classification: Using convolutional neural network (textCNN)
  • 10.9 Encoder-Decoder (seq2seq)
  • 10.10 Bunch Search
  • 10.11 Attention Mechanism
  • 10.12 Machine Translation

Continuously updated...

Original book address

Chinese version: Learn deep learning on hands | Github repository
English Version: Dive into Deep Learning | Github Repo

Quote

If you used this project in your research please cite the original book:

 @book{zhang2019dive,
    title={Dive into Deep Learning},
    author={Aston Zhang and Zachary C. Lipton and Mu Li and Alexander J. Smola},
    note={url{http://www.d2l.ai}},
    year={2020}
}