Dive into DL TensorFlow2.0

This project changes the implementation of MXNet code in the original book "Hand-On Deep Learning" to TensorFlow2. After archersama's tutor consulted with Teacher Li Mu, the implementation of this project has been approved by Teacher Li Mu. Original book authors: Aston Zhang, Li Mu, Zachary C. Lipton, Alexander J. Smora and other community contributors, GitHub address: https://github.com/d2l-ai/d2l-zh

There are some differences in the Chinese and English versions of this book. This project mainly refactors TensorFlow2 for the Chinese version of this book. In addition, this project also refers to the project Dive-into-DL-PyTorch that refactors PyTorch in the Chinese version of this book, thank you here.

It has been updated to ten chapters and is being updated continuously. . .

The project has been reported by many public accounts such as Machine Heart and has been recognized by the original author Li Mu.

Introduction

This repository mainly contains two folders: code and docs (plus some data is stored in data). The code folder is the relevant jupyter notebook code for each chapter (based on TensorFlow2); the docs folder is the relevant content in the markdown format of "Hand-On Deep Learning" book, and then use docsify to deploy the web document to GitHub Pages. Since the original book uses the MXNet framework, the docs content may be slightly different from the original book, but the overall content is the same. Contributions to this project or issue is welcome.

For crowds

This project is aimed at children's shoes that are interested in deep learning, especially those who want to use TensorFlow2 for deep learning. This project does not require you to have any background knowledge of deep learning or machine learning. You only need to understand basic mathematics and programming, such as basic linear algebra, differential and probability, and basic Python programming.

How to eat

Method 1

This repository contains some latex formulas, but github's markdown native does not support formula display, and the docs folder has been deployed on GitHub Pages using docsify, so the easiest way to view the document is to directly access the web version of this project . Of course, if you still want to run the relevant code, you still have to clone the project and then run the relevant code in the code folder.

Method 2

You can also access the documentation locally and install docsify-cli tool first:

npm i docsify-cli -g

Then clone the project locally:

git clone https://github.com/TrickyGo/Dive-into-DL-TensorFlow2.0
cd Dive-into-DL-TensorFlow2.0

Then run a local server, so that you can easily access the document web rendering effect in real time at http://localhost:3000 .

docsify serve docs

Contributors

The sponsors and main contributors of this project are as follows

archersama (leader)

TrickyGo

SwordFaith

ShusenTang

LIANGQINGYUAN

Table of contents

• Introduction
• Reading Guide
• 1. Introduction to Deep Learning
• 2. Preparation knowledge
  • 2.1 Environment configuration
  • 2.2 Data Operation
  • 2.3 Automatically find gradients
  • 2.4 Check the document
• 3. Basics of Deep Learning
  • 3.1 Linear Regression
  • 3.2 Implementation of linear regression from scratch
  • 3.3 A concise implementation of linear regression
  • 3.4 softmax regression
  • 3.5 Image classification dataset (Fashion-MNIST)
  • 3.6 Implementation of softmax regression from scratch
  • 3.7 A simple implementation of softmax regression
  • 3.8 Multi-layer sensing machine
  • 3.9 Implementation of multi-layer perception machine from scratch
  • 3.10 Simple implementation of multi-layer perception machine
  • 3.11 Model selection, underfitting and overfitting
  • 3.12 Weight decay
  • 3.13 Discarding method
  • 3.14 Forward propagation, back propagation and calculation diagrams
  • 3.15 Numerical stability and model initialization
  • 3.16 Actual Kaggle Competition: House Price Prediction
• 4. Deep Learning Computing
  • 4.1 Model construction
  • 4.2 Access, initialization and sharing of model parameters
  • 4.3 Delay initialization of model parameters
  • 4.4 Custom layer
  • 4.5 Reading and storage
  • 4.6 GPU computing
• 5. Convolutional neural network
  • 5.1 Two-dimensional convolution layer
  • 5.2 Fill and stride
  • 5.3 Multi-input channel and multi-output channel
  • 5.4 Pooling layer
  • 5.5 Convolutional Neural Network (LeNet)
  • 5.6 Deep Convolutional Neural Network (AlexNet)
  • 5.7 Networks using repeating elements (VGG)
  • 5.8 Network in the network (NiN)
  • 5.9 Network with parallel connection (GoogLeNet)
  • 5.10 Batch Normalization
  • 5.11 Residual Network (ResNet)
  • 5.12 DenseNet
• 6. Recurrent neural network
  • 6.1 Language Model
  • 6.2 Recurrent neural network
  • 6.3 Language Model Dataset (Lyrics of Jay Chou’s Album)
  • 6.4 Implementation of recurrent neural networks from scratch
  • 6.5 Simple implementation of recurrent neural network
  • 6.6 Backpropagation through time
  • 6.7 Gated Circulation Unit (GRU)
  • 6.8 Long Short-term Memory (LSTM)
  • 6.9 Deep recurrent neural network
  • 6.10 Bidirectional recurrent neural network
• 7. Optimization algorithm
  • 7.1 Optimization and Deep Learning
  • 7.2 Gradient descent and stochastic gradient descent
  • 7.3 Small batch stochastic gradient descent
  • 7.4 Momentum method
  • 7.5 AdaGrad Algorithm
  • 7.6 RMSProp Algorithm
  • 7.7 AdaDelta Algorithm
  • 7.8 Adam Algorithm
• 8. Computational Performance
  • 8.1 Imperative and Symbol Hybrid Programming
  • 8.2 Asynchronous calculation
  • 8.3 Automatic parallel computing
  • 8.4 Multi-GPU computing
• 9. Computer Vision
  • 9.1 Image augmentation
  • 9.2 Fine adjustment
  • 9.3 Object detection and bounding box
  • 9.4 Anchor frame
  • 9.5 Multi-scale target detection
  • 9.6 Object Detection Dataset (Pikachu)
  • 9.8 Regional Convolutional Neural Network (R-CNN) Series
  • 9.9 Semantic segmentation and datasets
  • 9.10 Full Convolutional Network (FCN)
  • 9.11 Style Migration
  • 9.12 Practical Kaggle Competition: Image Classification (CIFAR-10)
  • 9.13 Practical Kaggle Competition: Dog Breed Identification (ImageNet Dogs)
• 10. Natural Language Processing
  • 10.1 Word embedding (word2vec)
  • 10.2 Approximate training
  • 10.3 Implementation of word2vec
  • 10.4 Subword embedding (fastText)
  • 10.5 Word embedding of global vectors (GloVe)
  • 10.6 Find synonyms and analogies
  • 10.7 Text emotion classification: using recurrent neural networks
  • 10.8 Text emotion classification: Using convolutional neural network (textCNN)
  • 10.9 Encoder-Decoder (seq2seq)
  • 10.10 Bunch Search
  • 10.11 Attention Mechanism
  • 10.12 Machine Translation
• 11. Appendix
  • 11.1 List of main symbols
  • 11.2 Basics of Mathematics
  • 11.3 Using Jupyter Notebook
  • 11.4 Run code using AWS
  • 11.5 GPU Purchase Guide

Continuously updated...

Original book address

Chinese version: Learn deep learning on hands | Github repository
English Version: Dive into Deep Learning | Github Repo

Quote

If you used this project in your research please cite the original book:

 @book{zhang2019dive,
    title={Dive into Deep Learning},
    author={Aston Zhang and Zachary C. Lipton and Mu Li and Alexander J. Smola},
    note={url{http://www.d2l.ai}},
    year={2019}
}