基于深度学习的高分辨率麦穗图像检测方法

doi:10.13304/j.nykjdb.2021.0612

摘要/Abstract

摘要：

小麦是重要的粮食作物之一，针对人工田间麦穗计数及产量预测效率低的问题，基于深度学习提出了一种高分辨率的小密集麦穗实时检测方法。对麦穗图像数据集进行图像分割、标注、增强预处理，基于Tensorflow搭建YOLOv4网络模型，调整改进后对其进行迁移学习；与YOLOv3、YOLOv4-tiny、Faster R-CNN训练模型进行对比，对改进模型的实用性与局限性进行分析；重点分析影响麦穗检测模型性能的关键因素。通过图像分割的方式，证明了通过改变图像分辨率确定麦穗所占图像最优像素比，可以提高前景与背景差异，对小密集麦穗有显著效果。通过对改进模型的测试，表明该模型检测精度高，鲁棒性强。不同分辨率、不同品种、不同时期的麦穗图像均类平均精度（mAP）为93.7%，单张图片的检测速度为52帧·s^-1，满足了麦穗的高精度实时检测。该研究结果为田间麦穗计数以及产量预测提供技术支持。

关键词: 深度学习, 目标检测, 麦穗, YOLOv4, 实时检测

Abstract:

Wheat is one of the most important food crops in the world. In order to solve the problem of low efficiency of artificial field wheat ears counting and yield prediction， this paper proposed a real-time detection method based on deep learning for high resolution small and dense objects. The wheat ear images data set was preprocessed， including image segmentation， marking and enhancement， and the YOLOv4 network model was built based on TensorFlow to improve the transfer learning after adjusted； different models of YOLOv3， YOLOv4-Tiny and Faster R-CNN were trained and used compare to the practicability and limitations with the improved model； the key factors affecting the performance of the wheat ears detection model were discussed. By means of image segmentation， this paper studied the influence of images with different resolutions on model detection. The results showed that the difference between foreground and background could be improved by changing the image resolution， and the optimal pixel ratio of the target could be determined， which had a significant effect on small dense wheat ears and effectively improved the accuracy of the model. By testing the results of the improved model and evaluating the precision recall performance of different models， the model showed high detection accuracy and strong robustness. The mean average precision accuracy of wheat ear images with different resolutions， different varieties and different seasons was 93.7%， and the detection speed of single image was 52 frame·s^-1， which met the real-time detection of wheat ears. The results provided a technical guarantee for wheat spike counting and yield prediction in the field.

Key words: deep learning, object detection, grain, YOLOv4, real-time detection

中图分类号:

S126

赵越, 卫勇, 单慧勇, 穆志民, 张健欣, 吴海云, 赵辉, 胡建龙. 基于深度学习的高分辨率麦穗图像检测方法[J]. 中国农业科技导报, 2022, 24(9): 96-105.

Yue ZHAO, Yong WEI, Huiyong SHAN, Zhimin MU, ZHANGJianxin, Haiyun WU, Hui ZHAO, Jianlong HU. Wheat Ear Detection Method Based on Deep Learning[J]. Journal of Agricultural Science and Technology, 2022, 24(9): 96-105.

图/表 12

表1 不同分辨率下的模型

Table 1 Model with different resolution

模型 Model	图片尺寸 Image size	裁剪后分辨率 Cropped resolution	图片数量 Image number
#1	6 000×4 000	500×500	3 000
#2	6 000×4 000	1 000×1 000	3 000
#3	6 000×4 000	1 500×1 500	3 000
#4	6 000×4 000	2 000×2 000	3 000
#5	6 000×4 000	2 500×2 500	3 000
#6	6 000×4 000	3 000×3 000	3 000

图1 不同分辨率下的小麦麦穗图像

Fig. 1 Image of wheat ears under different resolutions

图2 标注的麦穗

Fig. 2 Annotated wheat ear

图3 YOLOv4的网络结构

Fig. 3 Network structure of YOLOv4

图4 YOLOv4主干网络

Fig. 4 Backbone network of YOLOv4

图5 PANet的网络结构

Fig. 5 Network structure of PANet

图6 训练中损失函数的变化

Fig. 6 Change of loss function in training

表3 不同分辨率下模型表现

Table 3 Model performance at different resolutions

模型Model	方法Method	分辨率Resolution	均类平均精度mAP/%
#1	YOLOv4	500×500	90.6
#2	YOLOv4	1 000×1 000	92.3
#3	YOLOv4	1 500×1 500	93.7
#4	YOLOv4	2 000×2 000	91.6
#5	YOLOv4	2 500×2 500	90.3
#6	YOLOv4	3 000×3 000	89.1

图7 不同条件下模型检测结果

Fig. 7 Model detection results at different conditions

图8 检测模型的P-R曲线

Fig. 8 P-R curves for the detection models

表4 不同网络下模型性能表现

Table 4 Model performance of different network

方法Method	损失值Loss	F₁值F₁ score	交并比IoU	类平均精度mAP/%	检测速度FPS
YOLOv3	2.4	0.786	0.842	89.4	57
YOLOv4-tiny	2.8	0.764	0.823	87.3	137
YOLOv4	0.7	0.837	0.886	93.7	52
FasterR-CNN	0.9	0.823	0.874	92.8	7

图9 不同模型检测结果注：蓝圈所示为漏检、错检麦穗。

Fig. 9 Detection results of different modelsNote： Blue circle shows missing or false wheat ears.

参考文献 24

1	CAO S, XU D, HANIF M, et al.. Genetic architecture underpinning yield component traits in wheat [J]. Theor. Appl. Genet., 2020, 133:1811-1823.
2	XU X, LI H, YIN F, et al.. Wheat ear counting using K-means clustering segmentation and convolutional neural network [J]. Plant Methods, 2020, 106(16): 1-13.
3	岑海燕,朱月明,孙大伟,等.深度学习在植物表型研究中的应用现状与展望[J].农业工程学报,2020,36(9):1-16.
	CEN H Y, ZHU Y M, SUN D W, et al.. Current status and future perspective of the application of deep learning in plant phenotype research [J]. Trans. Chin. Soc. Agric. Eng., 2020, 36(9): 1-16.
4	孙红,李松,李民赞,等.农业信息成像感知与深度学习应用研究进展[J].农业机械学报,2020,51(5):1-17.
	SUN H, LI S, LI Z M, et al.. Research Progress of Image Sensing and Deep Learning in Agriculture [J]. Trans. Chin. Soc. Agric. Mach., 2020 51(5): 1-17.
5	WU W, YANG T, LI R, et al.. Detection and enumeration of wheat grains based on a deep learning method under various scenarios and scales [J]. J. Int. Agr, 2020, 8(19): 1998-2008.
6	MA J, LI Y, DU K, et al.. Segmenting ears of winter wheat at flowering stage using digital images and deep learning [J]. Comput. Electron. Agric., 2020, 2(168): 1-16.
7	HASAN M M, CHOPIN J P, LAGA H, et al.. Detection and analysis of wheat spikes using Convolutional neural networks [J]. Plant Methods, 2018, 100(14): 1-13.
8	MADEC, JINX., LUH., et al.. Ear density estimation from high resolution RGB imagery using deep learning technique [J]. Agric. For. Met., 2019, 3(264): 225-234.
9	DU Z, YIN J, YANG J. Expanding receptive field YOLO for small object detection [J]. IOP Publ., 2019, 3(1314): 1-6.
10	XIAO J. exYOLO: A small object detector based on YOLOv3 object detector [J]. Computer Sci., 2021, 188: 18-25.
11	LabelImg is a graphical image annotation tool and label object bounding boxes in images [EB/OL]. (2017-04-03)[2018-12-03]. .
12	BOCHKOVSKIY A, WANG C, LIAO H M. YOLOv4: Optimal speed and accuracy of object detection [EB/OL]. (2020-04-23) [2021-12-23]. .
13	WANG C, LIAO H M, WU Y, et al.. CSPNet: a new backbone that can enhance learning capability of CNN [C]// Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Seattle, WA, USA: IEEE, 2020:1571-1580.
14	HE K, ZHANG X, REN S, et al.. Spatial pyramid pooling in deep convolutional networks for visual recognition [J]. IEEE Comp. Vis. Patt. Rec., 2015, 9(37): 1904-1906.
15	LIU S, QI L, QIN H, et al.. Path aggregation network for instance segmentation [C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018: 8759-8768.
16	M-TPHAM, COURTRAI L, FRIGUET C, et al.. YOLO-fine: one-stage detector of small objects under various backgrounds in remote sensing images [J/OL]. RemoteSens., 2020, 12(15): 2501 [2021-12-23]. .
17	XU D, WU Y. Improved YOLO-V3 with DenseNet for multi-scale remote sensing target detection [J/OL]. Sensors, 2020, 20(15):4276 [2021-12-23]. .
18	YANG B, FU X, SIDIROPOULOS ND, et al.. Towards K-means-friendly spaces: simultaneous deep learning and clustering [J]. Proc. Mach. Res., 2017, 70: 3861-3870.
19	LIU M, WANG X, ZHOU X, et al.. UAV-YOLO: small object detection on unmanned aerial vehicle perspective [J/OL]. Sensors, 2020, 20(8): 2238 [2021-12-23]. .
20	XIE Y, CAI J, BHOJWANI R, et al.. A locally-constrained YOLO framework for detecting small and densely-distributed building footprints [J]. Int. J. Geogr. Inf. Sci., 2019, 4(34): 1-25.
21	WU D, LV S, JIANG M, et al.. Using channel pruning-based YOLO v4 deep learning algorithm for the real-time and accurate detection of apple flowers in natural environments [J]. Comput. Electron. Agric., 2020, 178: 1-12.
22	XU Z, SHI H, LI N, et al.. Vehicle detection under UAV based on optimal dense YOLO method [C]// Proceedings of 2018 5th International Conference on Systems and Informatics. Nanjing, China: IEEE, 2018: 407-411.
23	NALLDRIN, KRASIN I, PONT-TUSET J, et al.. The open images dataset [EB/OL]. (2016-11-03) [2021-12-23]. .
24	TIAN Y, YANG G, WANG Z, et al.. Apple detection during different growth stages in orchards using the improved YOLO-V3 model [J]. Comput. Electron. Agric., 2019, 157: 417-426.

[1]	刘海涛, 韩鑫, 兰玉彬, 伊丽丽, 王宝聚, 崔立华. 基于YOLOv4网络的棉花顶芽精准识别方法[J]. 中国农业科技导报, 2022, 24(8): 99-108.
[2]	吴建伟, 黄杰, 熊晓菲, 高晗, 秦向阳. 基于AI的桃树病害智能识别方法研究与应用[J]. 中国农业科技导报, 2022, 24(5): 111-118.
[3]	高姻燕, 孙义, 李葆春. 基于无人机RGB影像估测田间小麦穗数[J]. 中国农业科技导报, 2022, 24(3): 103-110.
[4]	周惠汝, 吴波明. 深度学习在作物病害图像识别方面应用的研究进展[J]. 中国农业科技导报, 2021, 23(5): 61-68.
[5]	吕纯阳, 刘升平, 郭秀明, 肖顺夫, 刘大众, 杨菲菲, 李路华. 基于SSD模型的巢门蜜蜂检测[J]. 中国农业科技导报, 2021, 23(5): 98-107.
[6]	刘哲,袁冬根,王恩. 基于改进Bayes抠图算法的麦穗小穗自动计数方法[J]. 中国农业科技导报, 2020, 22(8): 75-82.
[7]	许贝贝1,王文生1,2*,郭雷风1,陈桂鹏3. 基于非接触式的牛只身份识别研究进展与展望[J]. 中国农业科技导报, 2020, 22(7): 79-89.
[8]	董伟1,钱蓉1,张洁2*,张立平1,陈红波2,张萌1,朱静波1,卜英乔3. 基于深度学习的蔬菜鳞翅目害虫自动识别与检测计数[J]. 中国农业科技导报, 2019, 21(12): 76-84.