基于天气数据增强和微调的苗期作物杂草识别定位模型

doi:10.13304/j.nykjdb.2023.0947

摘要/Abstract

摘要：

除草是提高作物产量的重要手段，而精准识别杂草种类是精确除草的前提。构建了SXAU杂草数据集，采用天气数据增强与在线数据增强相结合的方法提升Yolov8对杂草特征的提取能力，从而增强模型在实际应用中的效果，同时降低了计算机硬件的功耗。引入迁移学习，在公开杂草数据集上对Yolov8模型进行预训练，使用SXAU杂草数据集微调，以快速获取神经网络对杂草特征所需的相关参数。结合Yolov8算法和RealSense D435i深度相机，通过深度学习算法获取图像中作物与杂草的二维坐标，再利用深度相机得到目标三维坐标信息，实现对作物与杂草的空间定位。结果表明，基于天气数据增强和微调后的模型，平均检测精度和F1值分别达到97.43%和94.82%，平均检测时间为13.032 ms，优于Yolov7-tiny、Yolov5、Effcientdet等模型。研究结果可为轻量级杂草识别研究与智能除草机器人识别定位提供参考。

关键词: 杂草特征提取, 天气数据增强, 迁移学习, Yolov8, 轻量级, 深度相机

Abstract:

Weeding is a crucial method for boosting crop yield， and the precise identification of weed species is imperative for effective weed control. In this study， the SXAU weed dataset was established and the weather data enhancement and online data augmentation were combined to enhance the weed feature extraction capability of Yolov8. This enhancement aimed to improve the model’s practical application effectiveness while concurrently reducing the power consumption of computer hardware. Through the implementation of transfer learning， the Yolov8 model was initially pre-trained on a public weed dataset and subsequently fine-tuned using the SXAU weed dataset to swiftly acquire the relevant parameters needed by the neural network for weed feature recognition. Employing the Yolov8 algorithm in conjunction with the RealSense D435i depth camera， this approach involved utilizing a deep learning algorithm to derive the two-dimensional coordinates of crops and weeds within the image. Subsequently， the depth camera was employed to acquire three-dimensional coordinate information， facilitating spatial positioning of the crops and weeds. Experimental results demonstrate that， following the incorporation of weather data enhancement and fine-tuning， the model achieved an average detection accuracy of 97.43% and F1 value of 94.82%. The average detection time was 13.032 ms， surpassing performance metrics of Yolov7-tiny， Yolov5， Effcientdet， and other models. This research offered valuable insights for studies on lightweight weed identification， as well as the identification and spatial positioning of intelligent weeding robots.

Key words: weed feature extraction, weather data enhancement, transfer learning, Yolov8, lightweight, depth cemera

中图分类号:

S126

彭明康, 崔钰, 薛淇元, 殷允振, 尹哲, 张吴平, 李富忠. 基于天气数据增强和微调的苗期作物杂草识别定位模型[J]. 中国农业科技导报, 2024, 26(10): 125-134.

Mingkang PENG, Yu CUI, Qiyuan XUE, Yunzhen YIN, Zhe YIN, Wuping ZHANG, Fuzhong LI. Weed Identification and Location for Crop at Seedling Stage Based on Enhancing and Fine-tuning Weather Data[J]. Journal of Agricultural Science and Technology, 2024, 26(10): 125-134.

图/表 12

图1 天气数据增强效果

Fig. 1 Enhanced effect by weather data

图2 Mosaic数据增强效果

Fig. 2 Mosaic data enhancement effect

图3 基于微调的杂草检测流程

Fig. 3 Weed detection flow based on fine-tuning

表1 不同神经网络对杂草特征提取效果的评估

Table 1 Evaluation of weed feature extraction effect by different neural networks

网络 Network	类别精度均值 mAP/%	精确率Precision/%	召回率Recall/%	调和平均数F1/%	参数量Parameters/M	浮点数 Floating Point/G	平均检测时间Velocity/ms
Yolov8	97.43	94.14	95.57	94.82	11.155	25.754	13.032
Yolov7-tiny	88.99	92.46	57.93	70.10	6.149	13.611	12.547
Yolov5	96.22	93.40	91.05	92.36	7.157	16.376	15.823
Effcientdet	90.29	86.03	82.94	82.73	11.976	49.303	84.911

图4 模型在训练过程中验证集的平均精度均值曲线图

Fig. 4 Average accuracy mean curve of the validation set during the training process of the model

图5 Yolov8测试图像置信度

Fig. 5 Confidence of test image by Yolov8

表2 天气数据增强对杂草特征提取效果的评估

Table 2 Evaluation of the effect of weather data enhancement on weed feature extraction

模型

Model

平均精度均值mAP/%

准确率

Precision/%

召回率

Recall/%

调和平均数

F1/%

Yolov8-天气数据增强

Yolov8-weather augmentation

图6 天气数据增强与微调前后模型的混淆矩阵对比A：改进前；B：改进后。C1—玉米；C2—大豆；W1—龙葵；W2—藜；W3—蓟；W4—马唐；W5—打碗花；W6—反枝苋；W7—剑叶凤尾蕨；W8—马齿苋；W9—苣荬菜

Fig.6 Comparison of the confusion matrix of the model before and after weather data enhancement and fine-tuningA：Before improvement；B： After improved. C1—Corn； C2—Soybean； W1—Black nightshade； W2—Chenopodium album； W3—Cirsium； W4—Crabgrass； W5—Calystegia； W6—Pigweed； W7—Pteris； W8—Purslane； W9—Sonchus

图7 细粒度杂草图像的热区可视化

Fig. 7 Hot zone visualization of fine-grained weed images

表3 双目相机标定结果

Table 3 Binocular camera calibration results

相机 Cemera	相机参数 Camera parameters	标定结果 Evaluation result
左Left	参数矩阵 Parameter matrix	423.082 57	0		324.346 42
	参数矩阵 Parameter matrix	0	426.703 29		278.421 00
	畸变参数 Distortion parameter	0	0		1
	畸变参数 Distortion parameter	0.082 71	-0.294 30	0.000 90	0.002 77	0.310 26
右Right	参数矩阵 Parameter matrix	448.250 71	0		337.516 27
	参数矩阵 Parameter matrix	0	451.45484		256.585 24
	畸变系数 Distortion parameter	0	0		1
	畸变系数 Distortion parameter	0.078 36	-0.201 19	-0.000 58	0.001 66	0.115 51

表4 杂草定位精度试验结果

Table 4 Results of weed locating accuracy test

识别结果 Identification result	实际距离 Actual distance/m	深度相机测量距离 Depth cameras measures distance/m	深度相机误差 Depth camera error/m	双目相机测量距离 Binocular camera measures distance/m	双目相机误差 Binocular camera error/m
剑叶凤尾蕨Pteris	0.50	0.50	0.00	0.49	0.01
剑叶凤尾蕨Pteris	0.60	0.59	0.01	0.61	0.01
剑叶凤尾蕨Pteris	0.70	0.69	0.01	0.73	0.03
剑叶凤尾蕨Pteris	0.80	0.81	0.01	0.84	0.04
剑叶凤尾蕨Pteris	0.90	0.92	0.02	0.92	0.02
剑叶凤尾蕨Pteris	1.00	0.98	0.02	1.03	0.03

图8 移动端识别结果

Fig. 8 Mobile terminal identification results

参考文献 29

1	赵玉信,杨惠敏.作物格局、土壤耕作和水肥管理对农田杂草发生的影响及其调控机制[J].草业学报,2015,24(8):199-210.
	ZHAO Y X, YANG H M. Effects of crop pattern, tillage practice and water and fertilizer management on weeds and their control mechanisms [J]. Acta Prataculturae Sin., 2015, 24(8):199-210.
2	SYED AB RAHMAN S F, SINGH E, PIETERSE C M J, et al.. Emerging microbial biocontrol strategies for plant pathogens [J]. Plant Sci., 2018, 267:102-111.
3	潘俊峰,万开元,陶勇,等.基于农田养分管理的杂草生态防控策略[J].植物保护,2014,40(3):5-9, 36.
	PAN J F, WAN K Y, TAO Y, et al.. Ecological weed control strategy based on farmland nutrient management [J]. Plant Prot., 2014, 40(3):5-9, 36.
4	NIKITA G, RAYMOND A, ZEYNEP G, et al.. Deep learning-based early weed segmentation using motion blurred UAV images of sorghum fields [J/OL]. Comput. Electron. Agric., 2022,202:107388 [2024-08-08]. .
5	UTSTUMO T, URDAL F, BREVIK A, et al.. Robotic in-row weed control in vegetables [J]. Comput. Electron. Agric., 2018,154:36-45.
6	FERREIRA S D A, FREITAS M D, SILVA D G G, et al.. Weed detection in soybean crops using ConvNets [J]. Comput. Electron. Agric., 2017,143:314-324.
7	傅雷扬,李绍稳,张乐,等.田间除草机器人研究进展综述[J].机器人,2021,43(6):751-768.
	FU L Y, LI S W, ZHANG L, et al.. Research progress on field weeding robots:a review [J]. Robot, 2021,43(6):751-768.
8	SALEEM M H, POTGIETER J, ARIF K M. Automation in agriculture by machine and deep learning techniques:a review of recent developments [J]. Precis. Agric., 2021,22(6):2053-2091.
9	XU K, SHU L, XIE Q, et al.. Precision weed detection in wheat fields for agriculture 4.0:a survey of enabling technologies, methods, and research challenges [J/OL]. Comput.Electron.Agric., 2023,212:108106 [2024-08-08]. .
10	JANNEH L L, ZHANG Y J, CUI Z W, et al.. Multi-level feature re-weighted fusion for the semantic segmentation of crops and weeds [J/OL]. J. King Saud University-Comput. Inf. Sci., 2023,35(6):101545 [2024-08-08]. .
11	纪守领,杜天宇,邓水光,等.深度学习模型鲁棒性研究综述[J].计算机学报,2022,45(1):190-206.
	JI S L, DU T Y, DENG S G, et al.. Robustness certification research on deep learning models:a survey [J]. Chin. J. Comput., 2022,45(1):190-206.
12	JIANG H H, ZHANG C Y, QIAO Y L, et al.. CNN feature based graph convolutional network for weed and crop recognition in smart farming [J/OL]. Comput. Electron. Agric., 2020,174:105450 [2024-08-08]. .
13	CHEN J Q, WANG H B, ZHANG H D, et al.. Weed detection in sesame fields using a YOLO model with an enhanced attention mechanism and feature fusion [J/OL]. Comput. Electron. Agric., 2022,202:107412 [2024-08-08]. .
14	JIN X J, SUN Y X, CHE J, et al.. A novel deep learning-based method for detection of weeds in vegetables [J]. Pest Manage. Sci., 2022,78(5):1861-1869.
15	TANG J L, WANG D, ZHANG Z G, et al.. Weed identification based on K-means feature learning combined with convolutional neural network [J]. Comput. Electron. Agric., 2017,135:63-70.
16	WANG Y C, ZHANG S Q, DAI B S, et al.. Fine-grained weed recognition using Swin Transformer and two-stage transfer learning [J/OL]. Front. Plant Sci., 2023,14:1134932 [2024-08-08]. .
17	赵辉,曹宇航,岳有军,等.基于改进DenseNet的田间杂草识别[J].农业工程学报,2021,37(18):136-142.
	ZHAO H, CAO Y H, YUE Y J, et al.. Field weed recognition based on improved DenseNet [J]. Trans. Chin. Soc. Agric. Eng., 2021,37(18):136-142.
18	WANG P, TANG Y, LUO F, et al.. Weed25: a deep learning dataset for weed identification [J/OL]. Front. Plant Sci., 2022,13:1053329 [2024-08-08]. .
19	ABDALLA A, CEN H Y, WAN L, et al.. Fine-tuning convolutional neural network with transfer learning for semantic segmentation of ground-level oilseed rape images in a field with high weed pressure [J/OL].Comput.Electron.Agric.,2019,167:105091 [2024-08-08]. .
20	ESPEJO-GARCIA B, MYLONAS N, ATHANASAKOS L, et al.. Towards weeds identification assistance through transfer learning [J/OL]. Comput. Electron. Agric., 2020,171:105306 [2024-08-08]. .
21	REIS D, KUPEC J, HONG J, et al.. Real-Time flying object detection with YOLOv8 [EB/OL]. (2023-05-17) [2024-08-08]. .
22	YANG W J, WU J C, ZHANG J L, et al.. Deformable convolution and coordinate attention for fast cattle detection [J/OL]. Comput. Electron. Agric., 2023,211:108006 [2024-08-08]. .
23	TAN M X, LE Q V. Efficientnetv2: smaller models and faster training [EB/OL]. (2021-04-01) [2024-08-08]. .
24	REDMON J, DIVVALA S, GIRSHICK R, et al.. You only look once: Unified, realtime object detection. Paper presented at: CVPR 2016 [C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, 2016:779-788.
25	WANG Q F, CHENG M, HUANG S, et al.. A deep learning approach incorporating YOLOv5 and attention mechanisms for field real-time detection of the invasive weed Solanum rostratum Dunal seedlings [J/OL]. Comput. Electron. Agric., 2022,199:107194 [2024-08-08]. .
26	WANG C Y, BOCHKOVSKIY A, LIAO H Y M.YOLOv7:Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors [J/OL]. (2022-07-06) [2024-08-08]. .
27	LI S N, LI T F, SUN C, et al.. Multilayer Grad-CAM: An effective tool towards explainable deep neural networks for intelligent fault diagnosis [J]. J. Manuf. Syst., 2023,69:20-30.
28	ADADI A, BERRADA M. Peeking inside the Black-Box: a survey on explainable artificial intelligence (XAI) [J/OL]. IEEE Access, 2018,6:2870052 [2024-08-08]. .
29	张伟康,孙浩,陈鑫凯,等.基于改进YOLOv5的智能除草机器人蔬菜苗田杂草检测研究[J].图学学报,2023,44(2):346-356.
	ZHANG W K, SUN H, CHEN X K, et al.. Research on weed detection in vegetable seedling fields based on the improved YOLOv5 intelligent weeding robot [J]. J. Graph., 2023,44(2):346-356.

[1]	钱政, 杨孙哲, 张国卿, 郭紫微, 张林朋, 万家兴, 杨红云. 基于卷积神经网络的水稻氮素营养诊断[J]. 中国农业科技导报, 2023, 25(9): 113-121.
[2]	林开颜, 梅飞, 吴军辉, 郭文刚, 陈杰, 司慧萍. 基于计算机视觉的作物病害监测服务平台设计与研究[J]. 中国农业科技导报, 2023, 25(6): 89-96.
[3]	赵威, 马睿, 王佳, 郭宏杰, 许金普. 基于果穗图像的玉米品种分类识别[J]. 中国农业科技导报, 2023, 25(6): 97-106.
[4]	张彦通, 苏前敏. 基于迁移学习的玉米病害图像识别[J]. 中国农业科技导报, 2023, 25(10): 119-125.