Application of Improved YOLOv5 Model in Citrus Recognition in Natural Environment

doi:10.13304/j.nykjdb.2022.0994

Abstract

Abstract:

Green citrus in complex natural environment had different growth forms and similar color to the background color， so a detection method based on hybrid attention mechanism and improved YOLOv5 model was proposed to effectively identify green citrus. Firstly， the method improved the network structure of YOLOv5 by adding a hybrid attention mechanism in the backbone network etc.， embedding SE （squeeze and excitation） attention in layer 2 and CA （coordinate attention） attention in layer 11 of the backbone network； secondly， it improved the feature fusion structure of the network model， the lower branch was placed before the model C3 module， by combining the YOLOv5 model and concat feature fusion operation，and then the features were fused with another upper branch； finally， the classification loss function of the model was improved， and the classification loss function of the YOLOv5 model was changed to Varifocal Loss function to enhance the extraction of green citrus feature information and improve the accuracy of green citrus detection. According to the natural environment and the characteristics of the citrus itself， the self-built dataset was classified and 3 sets of comparison tests of citrus under different classification scenarios were designed to verify its effectiveness. The test results showed that the improved YOLOv5-SC model had higher precision and better robustness for the recognition of green citrus in natural environment， which accuracy was 91.74%， average accuracy was 95.09%， and F₁ was 89.56%， and it provided technical support for smart picking of green fruits.

Key words: object detection, YOLOv5, attention mechanism, loss function, green citrus

摘要：

在复杂的自然环境中绿色柑橘生长形态各异，颜色与背景色相近，为有效识别绿色柑橘，提出一种基于混合注意力机制并改进YOLOv5模型的柑橘识别方法。首先，改进YOLOv5的网络结构，在主干网络中添加混合注意力机制，即在主干网络中的第2层嵌入SE（squeeze and excitation）注意力，第11层嵌入CA（coordinate attention）注意力；其次，改进网络模型特征融合结构，将YOLOv5模型Concat特征融合操作的下层分支放在模型C3模块之前，再与另一条上层分支进行特征融合；最后，改进模型分类损失函数，将YOLOv5模型的分类损失函数改成Varifocal Loss函数，加强绿色柑橘特征信息的提取，提高绿色柑橘检测精度。根据自然环境和柑橘自身的特点，对自建数据集进行分类，设计3组不同分类场景下柑橘的对比试验以验证其有效性。试验结果表明，改进后的YOLOv5-SC模型准确率为91.74%，平均精度为95.09%，F₁为89.56%，在自然环境下对绿色柑橘的识别具有更高的准确率和更好的鲁棒性，为绿色水果智能采摘提供技术支持。

关键词: 目标检测, YOLOv5, 注意力机制, 损失函数, 绿色柑橘

CLC Number:

S126

Jun TIE, Jie ZHAO, Lu ZHENG, Lifeng WU, Bowen HONG. Application of Improved YOLOv5 Model in Citrus Recognition in Natural Environment[J]. Journal of Agricultural Science and Technology, 2024, 26(7): 111-120.

帖军, 赵捷, 郑禄, 吴立锋, 洪博文. 改进YOLOv5 模型在自然环境下柑橘识别的应用[J]. 中国农业科技导报, 2024, 26(7): 111-120.

Figures/Tables 14

References 25

1	王林惠,兰玉彬,刘志壮,等.便携式柑橘虫害实时检测系统的研制与试验[J].农业工程学报,2021,37(9):282-288.
	WANG L H, LAN Y B, LIU Z Z, et al.. Development and experiment of the portable real-time detection system for citrus pests [J]. Trans. Chin. Soc. Agric. Eng., 2021,37(9):282-288.
2	CHEN W, ZHOU Y, CHEN Y. The environmental impacts of citrus residue management in China: a case study in the three gorges reservoir region [J]. Waste Manage., 2021, 133(6):80-88.
3	TANG Y, CHEN M, WANG C, et al.. Recognition and localization methods for vision-based fruit picking robots: a review [J/OL]. Front. Plant Sci., 2020, 11:510 [2022-12-14]. .
4	YE L, DUAN J, YANG Z, et al.. Collision-free motion planning for the litchi-picking robot [J/OL]. Comput. Electron. Agric., 2021,185: 106151 [2022-12-14]. .
5	赵德安,吴任迪,刘晓洋,等.基于YOLO深度卷积神经网络的复杂背景下机器人采摘苹果定位[J].农业工程学报,2019,35(3):164-173.
	ZHAO D A, WU R D, LIU X Y, et al.. Apple positioning based on YOLO deep convolutional neural network forpicking robot in complex background [J]. Trans. Chin. Soc. Agric. Eng., 2019,35(3):164-173.
6	许德刚,王露,李凡.深度学习的典型目标检测算法研究综述[J].计算机工程与应用,2021,57(8):10-25.
	XU D G, WANG L, LI F. Review of typical object detection algorithms for deep learning [J]. Comput. Eng. Appl., 2021,57(8):10-25.
7	易诗,李俊杰,张鹏,等.基于特征递归融合YOLOv4网络模型的春见柑橘检测与计数[J].农业工程学报,2021,37(18):161-169.
	YI S, LI J J, ZHANG P, et al.. Detecting and counting of spring-see citrus using YOLOv4 network model and recursive fusion of features [J]. Trans. Chin. Soc. Agric. Eng., 2021,37(18):161-169.
8	杨婷婷,郭志勋,雷定湘,等.机器视觉技术在农业中的应用[J].安徽农学通报,2021, 27(18):110-111.
	YANG T T, GUO Z X, LEI D X, et al.. Application of machine vision technology in agriculture [J]. Anhui Agric. Sci. Bull., 2021,27(18):110-111.
9	WANG C, LEE W S, ZOU X, et al.. Detection and counting of immature green citrus fruit based on the Local Binary Patterns (LBP) feature using illumination-normalized images [J]. Precis. Agric., 2018,19(6):1062-1083.
10	LU J, LEE W S, HAO G, et al.. Immature citrus fruit detection based on local binary pattern feature and hierarchical contour analysis [J]. Biosys. Eng., 2018,171:78-90.
11	GAN H, LEE W S, ALCHANATIS V, et al.. Active thermal imaging for immature citrus fruit detection [J]. Biosys. Eng., 2020,198:291-303.
12	韩文,魏超宇,刘辉军.基于Tiny-YOLOv3的田间绿色柑橘目标检测方法[J].中国计量大学学报, 2020, 31(3):349-356, 392.
	HAN W, WEI C Y, LIU H J. Green citrus detection based on Tiny-YOLOv3 in field [J]. J. China Metrol. Univ., 2020,31(3):349-356, 392.
13	ZHENG Z, XIONG J, LIN H, et al.. A method of green citrus detection in natural environments using a deep convolutional neural network [J/OL]. Front. Plant Sci., 2021, 12:705737 [2022-12-28]. .
14	陈文康,陆声链,刘冰浩,等.基于改进YOLOv4的果园柑橘检测方法研究[J].广西师范大学学报(自然科学版),2021,39(5):134-146.
	CHEN W K, LU S L, LIU B H, et al.. Real-time citrus recognition under orchard environment by improved YOLOv4 [J]. J. Guangxi Norm. Univ. (Nat. Sci.), 2021,39(5):134-146.
15	黄彤镔,黄河清,李震,等.基于YOLOv5改进模型的柑橘果实识别方法[J].华中农业大学学报,2022,41(4):170-177.
	HUANG T B, HUANG H Q, LI Z, al.el. Citrus fruit recognition method based on the improved model of YOLOv5 [J]. J. Huazhong Agric. Univ., 2022,41(4):170-177.
16	杨长辉,刘艳平,王毅,等. 自然环境下柑橘采摘机器人识别定位系统研究[J].农业机械学报,2019,50(12):14-22.
	YANG C H, LIU Y P, WANG Y, et al.. Research and experiment on recognition and location system for citrus picking robot in natural environment [J]. Trans. Chin. Soc. Agric. Mach., 2019,50(12):14-22.
17	JIE H, LI S, GANG S, et al.. Squeeze-and-excitation networks [C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . IEEE, 2018:7132-7141.
18	HOU Q, ZHOU D, FENG J. Coordinate attention for efficient mobile network design [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2021:13713-13722.
19	ZHANG H, WANG Y, DAYOUB F, et al.. Varifocalnet:A n iou-aware dense object detector [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE,2021:8514-8523.
20	WANG C Y, LIAO H Y M, WU Y H, et al.. CSPNet:A new backbone that can enhance learning capability of CNN [J/OL]. 2019, 1911: 11929 [2022-12-28]. .
21	LIN T, DOLLAER P, GIRSHICK R, et al.. Feature pyramid networks for object detection [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2017:936-944.
22	LIU S, QI L, QIN H, et al.. Path aggregation network for instance segmentation [C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE,2018:8759-8768.
23	ZHENG Z H, WANG P, LIU W, et al.. Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression [C]// Proceedings of The Thirty-Fourth AAAI Conference on Artificial Intelligence 2017: 12993-13000..
24	宋中山,刘越,郑禄,等.基于改进YOLOV3的自然环境下绿色柑橘的识别算法[J].中国农机化学报,2021,42(11):159-165.
	SONG Z S, LIU Y, ZHENG L, al.el. Identification of green citrus based on improved YOLOV3 in natural environment [J]. J. Chin. Agric. Mechan., 2021,42(11):159-165.
25	刘芳,刘玉坤,林森,等.基于改进型YOLO的复杂环境下番茄果实快速识别方法[J].农业机械学报,2020,51(6):229-237.
	LIU F, LIU Y K, LIN S, et al.. Fast Recognition method for tomatoes under complex environments based on improved YOLO [J]. Trans. Chin. Soc. Agric. Mach., 2020,51(6):229-237.

分类参数 Classification parameter	标准Standard	天气Weather	图像数量 Image number	总计Total
果实数量 Fruit number	<5	阴天Cloudy	2 006	6 018
	<5	晴天Sunny	2 004
	5～10	阴天Cloudy	1 004
	5～10	晴天Sunny	1 004
果实大小（5个以内） Fruit size（<5）	较大Bigger	阴天Cloudy	1 002	4 010
	较大Bigger	晴天Sunny	1 000
	较小Smaller	阴天Cloudy	1 004
	较小Smaller	晴天Sunny	1 004

分类参数 Classification parameter	标准Standard	天气Weather	图像数量 Image number	总计Total
果实数量 Fruit number	<5	阴天Cloudy	2 006	6 018
	<5	晴天Sunny	2 004
	5～10	阴天Cloudy	1 004
	5～10	晴天Sunny	1 004
果实大小（5个以内） Fruit size（<5）	较大Bigger	阴天Cloudy	1 002	4 010
	较大Bigger	晴天Sunny	1 000
	较小Smaller	阴天Cloudy	1 004
	较小Smaller	晴天Sunny	1 004

序号 Number	注意力机制 Attention mechanisms	果实数量 Number of fruit	准确率 Precision/%
1	CBAM注意力 CBAM attention	5~10	93.01
2	CA注意力 CA attention	5~10	94.34
3	SE注意力 SE attention	5~10	95.14
4	SE+CA注意力 SE and CA attention	5~10	96.23

序号 Number	注意力机制 Attention mechanisms	果实数量 Number of fruit	准确率 Precision/%
1	CBAM注意力 CBAM attention	5~10	93.01
2	CA注意力 CA attention	5~10	94.34
3	SE注意力 SE attention	5~10	95.14
4	SE+CA注意力 SE and CA attention	5~10	96.23

模型 Model	准确率 Precision /%	平均精度均值 Mean average precision/%	调和平均数 F₁/%
Faster RCNN	87.61	87.55	87.21
YOLOv3-LITE	85.47	92.34	87.83
YOLOv5	89.13	94.82	88.23
YOLOv5-SC	91.74	95.09	89.56