dl_note

• Project Overview
• 1. Mathematics and Programming Basics Column
• 2. Basic components of neural networks
• 3. Classic convolutional neural network model
• 4. Deep learning alchemy
• 5. Deep Learning Model Compression
• Sixth, Model Inference Deployment
• 7. Advanced courses
• References

Project Overview

This warehouse project is a computer vision and large language model learning notes summarized by personal computer vision and large language model, including basic knowledge of deep learning, detailed explanation of basic components of neural networks, deep learning alchemy strategies, deep learning model compression algorithm, deep learning inference framework code analysis and hands-on practical combat.

For column notes on LLM basics and reasoning optimization, please refer to the llm_note warehouse.

1. Mathematics and Programming Basics Column

• Basics of Deep Learning Mathematics - Probability and Information Theory
• Basics of Deep Learning-Basic Principles of Machine Learning
• The mathematical basis of stochastic gradient descent
• Python Programming Mind Navigation

2. Basic components of neural networks

1. Basic components of neural network :

1. Detailed explanation of the basic components of neural networks - Convolutional layer
2. Detailed explanation of the basic components of neural network - BN layer
3. Basic components of neural network - detailed explanation of activation function

2. Basics of deep learning :

• Detailed explanation of backpropagation and gradient descent
• Basics of Deep Learning - Detailed explanation of parameter initialization
• Deep Learning Basics - Detailed Explanation of Loss Functions
• The basics of deep learning - detailed explanation of optimization algorithm

3. Classic convolutional neural network model

1. The classic backbone of convolutional neural networks :

• ResNet network details
• Detailed explanation of DenseNet network
• ResNetv2 network details
• Classic backbone network summary

2. Detailed explanation of lightweight network :

• Detailed explanation of MobileNetv1 paper
• Detailed explanation of ShuffleNetv2 paper
• Detailed explanation of RepVGG paper
• Detailed explanation of CSPNet paper
• Interpretation of VoVNet paper
• Lightweight model design summary

4. Deep learning alchemy

1. Deep Learning Alchemy-Data Standardization
2. Deep Learning Alchemy-Data Enhancement
3. Deep Learning Alchemy - Processing of Imbalanced Samples
4. Deep Learning Alchemy-Hyperparameter Setting
5. Deep Learning Alchemy-regularization strategy

5. Deep Learning Model Compression

1. A summary of deep learning model compression algorithm
2. Model compression-Summary of lightweight network design and deployment
3. Model compression-detailed explanation of pruning algorithm
4. Model compression-detailed explanation of knowledge distillation
5. Model compression-detailed explanation of quantization algorithm

Sixth, Model Inference Deployment

1. Model inference deployment:

• Convolutional neural network complexity analysis
• Model compression deployment overview
• Detailed explanation of matrix multiplication
• Model Inference Acceleration Techniques - Fusion Convolution and BN Layers

2. ncnn framework source code analysis:

• ncnn source code analysis-sample run
• ncnn Source Code Analysis-Net Class

3. Heterogeneous calculation

1. Mobile terminal heterogeneous computing: neon programming
2. GPU terminal heterogeneous computing: cuda programming, such as analysis and optimization of gemm algorithms

7. Advanced courses

1. Recommend several better warehouses and course materials for deep learning model compression and acceleration:

1. Tutorial on basic principles of neural networks
2. AI-System: Deep learning system, mainly explains the principles, acceleration methods, matrix multiplication and addition calculations, etc. from the underlying direction.
3. pytorch-deep-learning: A good pytorch deep learning tutorial.

2. Some blog links to good notes:

• The Illustrated Transformer: Most of the best blogs in China refer to this article.
• C++ Concurrent Programming (from C++11 to C++17): A good C++ Concurrent Programming Tutorial.
• What are Diffusion Models?
• annotated_deep_learning_paper_implementations

3. Finally, it is not easy to continue to create high-quality products. If you have 5 seconds of free time, you can scan the QR code to follow my official account - Embedded Vision , record the growth path of CV algorithm engineers, and share technical summary, reading notes and personal insights.

The official account will not write title-party articles, nor will it output the anxious content it brings to everyone!

4. Star History Chart:

References

• Deep Learning
• Machine Learning
• "Learn deep learning"
• Machine Learning Systems: Design and Implementation
• "AI-EDU"
• "AI-System"
• "PyTorch_tutorial_0.0.5_Yu Tingsong"
• "Planer, a Deep Learning Inference Framework"
• Distill: Knowledge Essence and Online Visualization
• LLVM IR Getting Started Guide
• nanoPyC
• ClassifyTemplate
• pytorch-classification