Home>Programming related>Python
Download

YOLOv5, YOLOv7, YOLOv8 Model 2024/full series/latest/improvement methods tutorial (source code included)

Improved articles in the Mango Book series improvement column

Column address: Exclusive first column on the entire network "Mango YOLOv8 In-depth Improvement Tutorial"?

Column address: Exclusive first column on the entire network "Mango YOLOv5 In-depth Improvement Tutorial"?

Column address: Exclusive first column on the entire network "Mango YOLOv7 In-depth Improvement Tutorial"?

The new UltralyticsPro project is coming

Project address: https://github.com/iscyy/ultralyticsPro

Focus on YOLOv8 improvements, NEW - YOLOv8 in PyTorch >, Support to improve Backbone, Neck, Head, Loss, IoU, LA, NMS and other modules Makes YOLOv8 improvements easy again

? The new YOLOv8 improvement project ultralyticsPro is coming, the latest improvement points are updated in 2024

1. Improve YOLOv8 free content <br>This project will continue to update some free improvement points, which will be updated every week

3.22 Update: Attention mechanism: ultralytics/cfg/models/cfg2024/YOLOv8-Attention

GAM attention mechanism,
SA attention mechanism,
SimAM Attention Mechanism,
SK Attention Mechanism,
SOCA attention mechanism,
CBAM Attention Mechanism,

3.23 Update: Pyramid Structure: ultralytics/cfg/models/cfg2024/YOLOv8-Pyramid Structure Improvement

SPPELAN,
SimSPPF,
BasicRFB,
ASPP,
SPPFCSPC

For more updates, click on: https://github.com/iscyy/ultralyticsPro

? The new YOLOAir2024 version is coming, the latest improvement points are updated in 2024

YOLOv5 project update

2024.03

Added Dysample upsampling improvement points: ICCV2023 paper: Dysample upsampling improvements

YOLO Air: Makes improvements easy again



YOLOAir is a YOLO algorithm library based on PyTorch. Unify the model code framework, unified application, unified improvement, easy module combination, and build a more powerful network model.

Simplified Chinese | English

support

Features • Use? • Document? • Report a problem? • Update? • Discussion✌️
• YOLO improvement methods and references •

Introduction

Model diversification : Different detection network models are constructed based on different network modules.

Modular componentization : Helps users to customize and quickly combine Backbone, Neck, and Head, diversify network models, help scientific research improve detection algorithms, model improvements, and network arrangement combinations. Build a powerful network model.

Unify the model code framework, unified application methods, unified parameter adjustment, unified improvement, integrated multi-tasks, easy module combination, and build a more powerful network model .

Built-in integrated model network structures such as YOLOv5, YOLOv7, YOLOv6, YOLOX, YOLOR, Transformer, PP-YOLO, PP-YOLOv2, PP-YOLOE, PP-YOLOEPlus, Scaled_YOLOv4, YOLOv3, YOLOv4, YOLO-Face, TPH-YOLO, YOLOv5Lite, SPD-YOLO, SlimNeck-YOLO, PicoDet and other models... Integrate a variety of detection algorithms and related multi-task models using a unified model code framework, integrated in the YOLOAir library, and unified application methods . It is convenient for scientific researchers to improve paper algorithm models, compare models, and realize diversification of network combinations. It includes lightweight models and higher precision models, and reasonably chooses according to the scene to achieve a balance between accuracy and speed. At the same time, the library supports decoupling different structures and module components to make module components. By combining different module components, users can customize and build different detection models based on different data sets or different business scenarios.

Supports integrated multitasking, including object detection, instance segmentation, image classification, pose estimation, face detection, target tracking and other tasks

Star? and Fork will not get lost, and will update simultaneously.

Project address?: https://github.com/iscyy/yoloair

Some improvement instructions tutorial?: https://github.com/iscyy/yoloair/wiki/Improved-tutorial-presentation

? Beta Supported Updates

The following functions are supported for internal testing of YOLOAir-Beta version

  • Other features?

? To Do

  • Supports multi-tasks, including object detection, classification, segmentation, pose estimation, face detection, target tracking and other tasks

Main features

Support more improvements to the YOLO series algorithm model (continuously updated...)

The YOLOAir algorithm library summarizes a variety of mainstream YOLO series detection models, and a set of code integrates multiple models:

  • Built-in integrated YOLOv5 model network structure, YOLOv7 model network structure, YOLOv6 model network structure, PP-YOLOE model network structure, PP-YOLOEPlus model network structure, YOLOR model network structure, YOLOX model network structure, ScaledYOLOv4 model network structure, YOLOv4 model network structure, YOLOv3 model network structure, YOLO-FaceV2 model network structure, TPH-YOLOv5 model network structure, SPD-YOLO model network structure, SlimNeck-YOLO model network structure, YOLOv5-Lite model network structure, PicoDet model network structure, etc. are constantly being updated...

Support more network model components

  • As table
YOLOAir Structural Module Components
Backbones Necks Head
  • CSPDarkNet Series
  • HorNet Series
  • ResNet Series
  • RegNet Series
  • ShuffleNet Series
  • RepLKNet Series
  • MobileNet Series
  • EfficientNet Series
  • ConvNext series
  • Ghost series
  • RepVGG Series
  • transformer series
  • PANet
  • RepPAN
  • BiFPN
  • CSPPAN
  • SlimNeck
  • ELANPAN
  • GSNeck
  • YOLOv4Head
  • YOLOv5Head
  • YOLOv6 Efficient decoupled head
  • YOLOv7 IDetect Head
  • YOLOv7 IAuxDetect Head
  • YOLOX DetectX Head
  • Adaptive spatial feature fusion detection head ASFF Head
  • YOLOR Implicit Learning Head
  • EH-Head
  • Dual Weighting Head
  • FCOS Head
  • Dynamic Head
    • ...
YOLOAir Module Components
Attention mechanism Tag assignment strategy IoU Loss Function
  • Self Attention
  • Contextual Transformer
  • Bottleneck Transformer
  • S2-MLP Attention
  • AxialAttention
  • DANet
  • DoubleAttention
  • GlobalContextBlock
  • ParNetAttention
  • PolarizedSelfAttention
  • SpatialGroupEnhance copy
  • TripletAttention
  • SK Attention
  • CBAM Attention
  • SE Attention
  • Coordinate attention
  • NAM Attention
  • GAM Attention
  • ECA Attention
  • Shuffle Attention
  • CrissCrossAttention
  • SOCAttention
  • SimAM Attention
  • CrissCrossAttention
  • Multi Anchor Strategy
  • YOLOv5 Tag Allocation Policy
  • SimOTA Tag Allocation Policy
  • YOLOv7 tag allocation strategy
  • FreeAnchor Tag Allocation Policy
  • AutoAssign Tag Assignment Policy
  • Dual Weighting Tag Allocation Policy
  • Probabilistic Anchor Assignment Anchor Assignment
  • Other improved tag allocation strategies
  • CIoU
  • DIoU
  • GIoU
  • EIoU
  • SIoU
  • alpha IOU
    • ...
Module Components
Loss NMS Data Enhancement
  • ComputeLoss(v3)
  • ComputeLoss(v4)
  • ComputeLoss(v5)
  • ComputeLoss(v6)
  • ComputeLoss(X)
  • ComputeLossAuxOTA(v7)
  • ComputeLossOTA(v7)
  • NMS
  • Merge-NMS
  • Soft-NMS
  • CIoU_NMS
  • DIoU_NMS
  • GIoU_NMS
  • EIoU_NMS
  • SIoU_NMS
  • Soft-SIoUNMS
  • Soft-CIoUNMS
  • Soft-DIoUNMS
  • Soft-EIoUNMS
  • Soft-GIoUNMS
  • SimSPPF
  • SimSPPF
  • SimSPPF
  • Mosaic
  • Copy paste
  • Random affine(Rotation, Scale, Translation and Shear)
  • MixUp
  • HSV
  • Random horizontal flip
    • ...
YOLOAir Module Components
Space pyramid pooling structure Modular components AnchorBased / AnchorFree
  • SPP
  • SPPF
  • ASPP
  • RFB
  • SPPCSPC
  • SPPFCSPC
  • SimSPPF
  • Conv, GhostConv, Bottleneck, GhostBottleneck, SPP, SPPF, DWConv, MixConv2d, Focus, CrossConv, BottleneckCSP, C3, C3TR, C3SPP, C3Ghost, C3HB, C3RFEM, MultiSEAM, SEAM, C3STR, SPPCSPC, RepConv, BoT3, Air, CA, CBAM, Involution, Stem, ResCSPC, ResCSPB, ResXCSPB, ResXCSPC, BottleneckCSPB, BottleneckCSPC, ASPP, BasicRFB, SPPCSPC_group, HorBlock, CNeB, C3GC, C3C2, nn.ConvTranspose2d, DWConvblock, RepVGGBlock, CoT3, ConvNextBlock, SPPCSP, BottleneckCSP2, DownC, BottleneckCSPF, RepVGGBlock, ReOrg, DWT, MobileOne, HorNet...
  • YOLOv5, YOLOv7, YOLOv3, YOLOv4, YOLOR, ScaledYOLOv4, PPYOLO, PPYOLOv2, Improved-YOLOv5, Improved-YOLOv7
  • YOLOX, YOLOv6 (Paper), PPYOLOE, PPYOLOE+
    • ...

Support more Backbone

  • CSPDarkNet Series
  • HorNet Series
  • ResNet Series
  • RegNet Series
  • ShuffleNet Series
  • Ghost series
  • MobileNet Series
  • EfficientNet Series
  • ConvNext series
  • RepLKNet Series
  • Reparameterization series
  • RepVGG Series
  • RepMLP Series
  • ACNet Series
  • RepConv Series
  • OREPA Series
  • Mobileone series
  • Self-attention Transformer series
  • MobileViT Series
  • BoTNet-Transfomrer
  • CoTNet-Transfomrer
  • Swin-Transfomrer
  • Swin-Transfomrer(v2)
  • And other trans series are being updated continuously? Note: (YOLOAir(Beta) has completed the update of 20+ Transformer series backbone networks, multiple MLP networks, and most of the reparameterized structural model networks)

Support more Neck

  • FPN
  • PANet
  • RepPAN
  • BiFPN
  • CSPPAN
  • SlimNeck
  • ELANPAN
    Continuously updated?

Support more detection heads

  • YOLOv4, YOLOv5 Head detection head;

  • YOLOR implicitly learns the head detection head;

  • YOLOX's decoupled detection head Decoupled Head and DetectX Head;

  • Adaptive spatial feature fusion detection head ASFF Head;

  • YOLOv6-v2.0 Efficient decoupled head;

  • YOLOv7 detection head IAuxDetect Head, IDetect Head, etc.;

  • Other different detection heads

Supports more plug-and-play attention mechanisms Attention

  • Plug and play attention mechanisms in any part of the network

  • Self Attention

  • Contextual Transformer

  • Bottleneck Transformer

  • S2-MLP Attention

  • SK Attention

  • CBAM Attention

  • SE Attention

  • Coordinate attention

  • NAM Attention

  • GAM Attention

  • ECA Attention

  • Shuffle Attention

  • CrissCrossAttention

  • Coordinate attention

  • SOCAttention

  • SimAM Attention is being updated continuously?

  • Big news! ! ! As a supplement to the open source project, we recommend an attention algorithm code library External-Attention. It is summarized and organized in a comprehensive way, including various Attention and other codes. The code is simple and easy to read, and one line of code implements the Attention mechanism.

More space pyramid pooling structures

  • SPP
  • SPPF
  • ASPP
  • RFB
  • SPPCSPC
  • SPPFCSPC
  • SimSPPF is being updated continuously?

Support more Loss

  • ComputeLoss
  • ComputeLoss(v5)
  • ComputeLoss(v6)
  • ComputeLoss(X)
  • ComputeLossAuxOTA(v7)
  • ComputeLossOTA(v7)
  • ComputeNWDLoss
  • Other Loss

Supports Anchor-base and Anchor-Free

  • ? YOLOv5, YOLOv7, YOLOv3, YOLOv4, YOLOR, ScaledYOLOv4, PPYOLO, PPYOLOv2, Improved-YOLOv5, Improved-YOLOv7

  • ? YOLOX, YOLOv6 (Paper), PPYOLOE, PPYOLOE+

Supports multiple tag allocation strategies

  • Multi Anchor Strategy
  • YOLOv5 Tag Allocation Policy
  • SimOTA Tag Allocation Policy
  • YOLOv7 tag allocation strategy
  • AutoAssign Tag Assignment Policy?
  • Dual Weighting Tag Allocation Policy?
  • FreeAnchor Tag Allocation Policy?
  • Other improved tag allocation strategies are being continuously updated?

Support weighted box fusion (WBF)

Built-in multiple modular components for network model
Conv, GhostConv, Bottleneck, GhostBottleneck, SPP, SPPF, DWConv, MixConv2d, Focus, CrossConv, BottleneckCSP, C3, C3TR, C3SPP, C3Ghost, C3HB, C3RFEM, MultiSEAM, SEAM, C3STR, SPPCSPC, RepConv, BoT3, Air, CA, CBAM, Involution, Stem, ResCSPC, ResCSPB, ResXCSPB, ResXCSPC, BottleneckCSPB, BottleneckCSPC, ASPP, BasicRFB, SPPCSPC_group, HorBlock, CNeB, C3GC, C3C2, nn.ConvTranspose2d, DWConvblock, RepVGGBlock, CoT3, ConvNextBlock, SPPCSP, BottleneckCSP2, DownC, BottleneckCSPF, RepVGGBlock, ReOrg, DWT, MobileOne, HorNet... and other detailed codes./models/common.py file

Support more IoU loss functions

  • CIoU
  • DIoU
  • GIoU
  • EIoU
  • SIoU
  • alpha IOU
    Continuously updated?

Support more NMS

  • NMS
  • Merge-NMS
  • Soft-NMS
  • CIoU_NMS
  • DIoU_NMS
  • GIoU_NMS
  • EIoU_NMS
  • SIoU_NMS
  • Soft-SIoUNMS, Soft-CIoUNMS, Soft-DIoUNMS, Soft-EIoUNMS, Soft-GIoUNMS, etc.;
    Continuously updated?

Support more data enhancements

  • Mosaic, Copy paste, Random affine(Rotation, Scale, Translation and Shear), MixUp, Augment HSV(Hue, Saturation, Value, Random horizontal flip

Supports the official pre-training weights loading of YOLOv3, YOLOv4, YOLOv5, YOLOv7, YOLOR, Scaled_YOLO and other networks for transfer learning

YOLO series network model thermal diagram visualization (GardCAM, GardCAM++, etc.) supports YOLOv3, YOLOv3-SPP, YOLOv4, YOLOv5, YOLOR, YOLOv7Scaled_YOLOv4, TPH-YOLO, SPD-YOLO and custom network model models (YOLOAir (Beta version internal test) has been supported)

Summary of mainstream network model structure diagrams: Model?

The above component modules use a unified model code framework, a unified task form, and a unified application method. Module componentization can help users customize and quickly combine Backbone, Neck, and Head, diversify network models, help scientific research improve detection algorithms, and build a more powerful network model.

Note: The above main features support include Main version and Beta version. Some features are temporarily updated only in Beta. Not all updates are directly in Main. After subsequent beta versions are internally tested, they are gradually added to the Main version.

Built-in network model configuration support

Including model configuration files based on various algorithm models such as YOLOv5, YOLOv7, YOLOX, YOLOR, YOLOv3, YOLOv4, Scaled_YOLOv4, PPYOLOE, PPYOLOEPlus, Transformer, YOLO-FaceV2, PicoDet, YOLOv5-Lite, TPH-YOLOv5, SPD-YOLO, etc.

Effect preview

Target detection Target segmentation
Image classification Instance segmentation
Target segmentation Target tracking
Posture estimation Face detection
Thermal Photo 01 Thermal Photo 02

Pre-training weights

  • YOLOv5 https://github.com/ultralytics/yolov5/releases/tag/v6.1

  • YOLOv4 https://github.com/iscyy/yoloair/releases/tag/v1.0

  • YOLOv3 https://github.com/iscyy/yoloair/releases/tag/v1.0

  • YOLOR https://github.com/iscyy/yoloair/releases/tag/v1.0

  • Scaled_YOLO https://github.com/iscyy/yoloair/releases/tag/v1.0

  • YOLOv7 https://github.com/iscyy/yoloair/releases/tag/v1.0

use?

About the code. Follow the design principle of YOLOv5.
The original version was created based on YOLOv5(v6.1)

Install

Clone the version warehouse in Python>=3.7.0 and install requirements.txt, including PyTorch>=1.7 . 

$ git clone https://github.com/iscyy/yoloair.git  # 克隆
$ cd YOLOAir
$ pip install -r requirements.txt  # 安装

train 

$ python train.py --data coco128.yaml --cfg configs/yolov5/yolov5s.yaml #默认为yolo

reasoning

detect.py runs inference on various data sources and saves the detection results to the runs/detect directory. 

$ python detect.py --source 0  # 网络摄像头
                          img.jpg  # 图像
                          vid.mp4  # 视频
                          path/  # 文件夹
                          path/ * .jpg  # glob

Fusion

If you use different models to reason about datasets, you can use the wbf.py file to integrate the results through weighted box fusion. You just need to set the img path and the txt path in the wbf.py file. 

$ python wbf.py

Paper Sharing

FightingCV official account shares cutting-edge paper updates every day
(Reply to the public account to add group, add assistant to exchange group)

Performance

Model size
(pixels)		 mAP val
0.5:0.95		 mAP val
0.5		 Speed
CPU b1
(ms)		 Speed
V100 b1
(ms)		 Speed
V100 b32
(ms)		 params
(M)		 FLOPs
@640 (B)		 Weights
YOLOv5n 640 28.0 45.7 45 6.3 0.6 1.9 4.5 YOLOv5n
YOLOv5s 640 37.4 56.8 98 6.4 0.9 7.2 16.5 YOLOv5s
YOLOv5m 640 45.4 64.1 224 8.2 1.7 21.2 49.0 YOLOv5m
YOLOv5l 640 49.0 67.3 430 10.1 2.7 46.5 109.1 YOLOv5l
YOLOv5x 640 50.7 68.9 766 12.1 4.8 86.7 205.7 YOLOv5x
YOLOv5n6 1280 36.0 54.4 153 8.1 2.1 3.2 4.6 YOLOv5n6
YOLOv5s6 1280 44.8 63.7 385 8.2 3.6 12.6 16.8 YOLOv5s6
YOLOv5m6 1280 51.3 69.3 887 11.1 6.8 35.7 50.0 YOLOv5m6
YOLOv5l6 1280 53.7 71.3 1784 15.8 10.5 76.8 111.4 YOLOv5l6
YOLOv5x6
+ TTA		 1280
1536		 55.0
55.8		 72.7
72.7		 3136
-		 26.2
-		 19.4
-		 140.7
-		 209.8
-		 YOLOv5x6
Expand
  • The original version was created based on YOLOv5(6.1)
Model size
(pixels)		 mAP val
0.5:0.95		 mAP val
0.5		 deconv kernel size
 Speed
V100 b1
(ms)		 Speed
V100 b32
(ms)		 params
(M)		 FLOPs
@640 (B)
YOLOv5s 640 33.7 52.9 - 5.6 2.2 7.23 16.5
YOLOv5s-deconv-exp1 640 33.4 52.5 2 5.6 2.4 7.55 18.2
YOLOv5s-deconv-exp2 640 34.7 54.2 4 5.8 2.5 8.54 23.2
Expand
  • The training process depends on 4xV100 GPU 
 # train
python -m torch.distributed.run --nproc_per_node 4 train.py --device 0,1,2,3 --data data/coco.yaml --hyp data/hyps/hyp.scratch-low.yaml  --cfg path/to/model.yaml --batch 512 --epochs 300 --weights ''
# val
python val.py --verbose --data data/coco.yaml --conf 0.001 --iou 0.65 --batch 1 --weights path/to/model.pt
  • There is a gap between the mAP of YOLOv5s and the official one, here is just for comparison
Model size
(pixels)		 mAP val
0.5:0.95		 mAP val
0.5		 params
(M)		 FLOPs
@640 (B)
YOLOv5s 640 37.4 56.6 7.226 16.5
YOLOv5s-deconv 640 37.8 57.1 7.232 16.5
Expand
  • tested the 4x4 depthwise-separable deconv by setting the groups as input_channel
  • Their params number and FLOPS are nearly the same while the new model's mAP is about 0.4 higher than the origin.
Model size
(pixels)		 mAP val
0.5:0.95		 mAP val
0.5		 params
(M)		 FLOPs
@640 (B)
YOLOv5s 640 37.2 56.0 7.2 16.5
YOLOv5s-C3GC-backbone 640 37.7 57.3 7.5 16.8
Expand
  • The original version was created based on YOLOv5-6.0

YOLOv5 tutorial

Synchronize with YOLOv5 framework

  • Training custom data recommendations
  • Tips for getting the best training results☘️ Recommended
  • Record experiments using Weights & Biases? New
  • Roboflow: Datasets, Tags and Active Learning? New
  • Multi-GPU training
  • PyTorch Hub New
  • TFLite, ONNX, CoreML, TensorRT Export
  • Test-time data enhancement (TTA)
  • Model Integration
  • Model pruning/sparseness
  • Hyperparameter evolution
  • Transfer learning with freezing layer new
  • New architecture summary

Future enhancement

In the future, we will continue to build and improve the YOLOAir ecosystem to improve and integrate more YOLOv5 series models, continue to combine different modules, build more different network models horizontally and introduce related technologies, etc.

Citation 

 @ article { 2022 yoloair ,
  title = {{ YOLOAir }: Makes improvements easy again },
  author = { iscyy },
  repo = { github https : // github . com / iscyy / yoloair },
  year = { 2022 }
}

Statement

Expand

  • The content of this site is only for sharing notes. If some content is infringing, please sending email.

  • If you have any question, please discuss with me by sending email.

Acknowledgements

Expand

https://github.com/ultralytics/yolov5
https://github.com/ultralytics/yolov3
https://github.com/WongKinYiu/PyTorch_YOLOv4
https://github.com/WongKinYiu/ScaledYOLOv4
https://github.com/meituan/YOLOv6
https://github.com/WongKinYiu/yolov7
https://github.com/WongKinYiu/yolor
https://github.com/xmu-xiaoma666/External-Attention-pytorch
https://gitee.com/SearchSource/yolov5_yolox
https://github.com/Krasjet-Yu/YOLO-FaceV2
https://github.com/positive666/yolov5_research/
https://github.com/ppogg/YOLOv5-Lite
https://github.com/Gumpest/YOLOv5-Multibackbone

Expand
Additional Information
Related Applications
Recommended for You
Related Information All