Tomato Fruit Recognition Based on YOLOX-L-TN Model

doi:10.13304/j.nykjdb.2023.0570

Abstract

Abstract:

Aiming at the intelligent demand for tomato picking operation in plant factories， in order to overcome the problems of low recognition accuracy and low speed caused by different sizes and overlapping of tomato fruits during picking operation， an improved target recognition model of YOLOX-L-TN was proposed， in which a TN module containing residual structure was designed according to the internal structure and principle of channel and spatial attention mechanism of feature graph， and integrated into the backbone network of YOLOX-L. This model improved the speed and accuracy of model recognition while maintaining the lightweight of the network. Compared with YOLOX-L， the AP value of YOLOX-L-TN was increased by 4.81 percentage points， and the recognition time of single image is increased by 0.141 7 s， and the optimal position of TN module was between the input and the backbone network. Furthermore， TN module was compared with similar modules SENet， CAM， CBAM and CAM， and the results showed that AP value was increased by 0.53， 4.19， 6.12 and 6.34 percentage points， respectively， and the recognition time of single image is increased by 0.019 1， 0.025 0， 0.021 1， 0.018 9 s， respectively. In conclusion， the proposed YOLOX-L-TN model had the advantages of high precision， fast identification speed and high robustness， which provided technical support for the intelligent picking of tomatoes in the later stage.

Key words: tomato recognition, attention mechanism, TN module, YOLOX-L

摘要：

针对植物工厂对番茄采摘作业的智能化需求，为克服在采摘作业过程中因番茄果实大小不一、遮挡重叠造成的识别精度不高和速度不快的问题，提出了YOLOX-L的改进型目标识别模型YOLOX-L-TN。该模型依据特征图的通道和空间注意力机制的内部结构和原理，设计了含有残差结构的TN模块，并融入到YOLOX-L的主干网络中，在保持网络轻量化的同时，实现模型识别速度和精度的同时提升。与YOLOX-L相比，YOLOX-L-TN的AP值提高了4.81个百分点，单张图像的识别时间提升了0.141 7 s，TN模块的最佳位置为输入端与主干网络之间。进一步将TN模块与类似模块SENet、CAM、CBAM和CAM进行对比，AP值分别提高0.53、4.19、6.12、6.34个百分点，单张图像识别时间分别提升0.019 1、0.025 0、0.021 1、0.018 9 s。由此可见，提出的YOLOX-L-TN模型具有精度高、识别速度快、鲁棒性高等优点，可为番茄后期的智能采摘提供技术支持。

关键词: 番茄识别, 注意力机制, TN模块, YOLOX-L

CLC Number:

S225

Mingbo LI, Yule LIU, Zhimin MU, Junwang GUO, Yong WEI, Dongyue REN, Jishen JIA, Zezhong WEI, Yuhong LI. Tomato Fruit Recognition Based on YOLOX-L-TN Model[J]. Journal of Agricultural Science and Technology, 2024, 26(4): 97-105.

李名博, 刘玉乐, 穆志民, 郭俊旺, 卫勇, 任东悦, 贾济深, 卫泽中, 栗宇红. 基于YOLOX-L-TN模型的番茄果实识别[J]. 中国农业科技导报, 2024, 26(4): 97-105.

Figures/Tables 9

References 30

1	李君明,项朝阳,王孝宣,等.“十三五”我国番茄产业现状及展望[J].中国蔬菜,2021(2):13-20.
	LI J M, XIANG Z Y, WANG X X, et al.. Current situation and prospect of tomato industry in China during the 13^th Five-Year Plan [J]. Chin. Veg., 2021(2):13-20.
2	刘继展. 温室采摘机器人技术研究进展分析[J].农业机械学报,2017.48(12):1-18.
	LIU J Z. Research progress analysis of robotic harvesting technologies in greenhouse [J]. Trans. Chin. Soc. Agric. Mach., 2017,48(12):1-18.
3	KONDO N, YATA K, IIDA M, et al. Development of an end-effector for a tomato cluster harvesting robot [J]. Eng. Agric. Environ. Food., 2010,3(1):20-24.
4	XIE H, KONG D, SHAN J, et al.. Study the parametric effect of pulling pattern on cherry tomato harvesting using RSM-BBD techniques [J/OL]. Agriculture, 2021,11(9):815 [2024-02-24]. .
5	李寒,陶涵虓,崔立昊,等.基于SOM-K-means算法的番茄果实识别与定位方法[J].农业机械学报,2021,52(1):23-29.
	LI H, TAO H X, CUI L H, et al.. Tomato fruit recognition and location method based on SOM-K-means algorithm [J]. Trans. Chin. Soc. Agric. Mach., 2021,52(1):23-29.
6	王海楠,弋景刚,张秀花.番茄采摘机器人识别与定位技术研究进展[J].中国农机化学报,2020,41(5):188-196.
	WANG H N, GE J G, ZHANG X H. Research progress on identification and localization technology of tomato picking robot [J]. J. Chin. Agric. Mech., 2020,41(5):188-196.
7	刘俊明,孟卫华.基于深度学习的单阶段目标检测算法研究综述[J].航空兵器,2020,27(3):44-53.
	LIU J M, MENG W H. A review of single-stage object detection algorithms based on deep learning [J]. Aero Weap., 2020,27(3):44-53.
8	张境锋,陈伟,魏庆宇,等.基于Des-YOLO v4的复杂环境下苹果检测方法[J].农机化研究,2023,45(5):20-25.
	ZHANG J F, CHEN W, WEI Q Y, et al.. Apple detection method in complex environment based on Des-YOLOv4 [J]. J. Agric. Mech. Res., 2023,45(5):20-25.
9	张俊宁,毕泽洋,闫英,等.基于注意力机制与改进YOLO的温室番茄快速识别[J].农业机械学报,2023,54(5):236-243.
	ZHANG J N, BI Z X, YAN Y, et al.. Rapid recognition of greenhouse tomatoes based on attention mechanism and improved YOLO [J]. Trans. Chin. Soc. Agric. Mach., 2023,54(5):236-243.
10	ZENG T H, LI S Y, SONG Q M, et al.. Lightweight tomato real-time detection method based on improved YOLO and mobile deployment [J]. Comput. Electron. Agric., 2023,205:107625.
11	QI J T, LIU X N, LIU K, et al.. An improved YOLOv5 model based on visual attention mechanism: application to recognition of tomato virus disease [J]. Comput. Electron. Agric., 2022,194:106780.
12	张立杰,周舒骅,李娜,等.基于改进SSD卷积神经网络的苹果定位与分级方法[J].农业机械学报,2023,54(6):223-232.
	ZHANG L J, ZHOU S H, LI N, et al.. Apple localization and grading method based on improved SSD convolutional neural network [J]. Trans. Chin. Soc. Agric. Mach., 2023,54(6):223-232.
13	毛锐,张宇晨,王泽玺,等.利用改进Faster-RCNN识别小麦条锈病和黄矮病[J].农业工程学报,2022,38(17):176-185.
	MAO R, ZHANG Y C, WANG Z X, et al.. Identification of wheat stripe rust and yellow dwarf by improved Faster-RCNN [J]. Trans. Chin. Soc. Agric. Eng., 2022,38(17):176-185.
14	梁喜凤,章鑫宇,王永维.基于改进Mask R-CNN的番茄侧枝修剪点识别方法[J].农业工程学报,2022,38(23):112-121.
	LIANG X F, ZHANG X Y, WANG Y W. Identification method of tomato lateral branch pruning points based on improved Mask R-CNN [J]. Trans. Chin. Soc. Agric. Eng., 2022,38(23):112-121.
15	LIU G, NOUAZE J C, TOUKO MBOUEMBE P L, et al.. YOLO-tomato: a robust algorithm for tomato detection based on YOLOv3 [J/OL]. Sensors, 2020,20(7):2145 [2024-02-24]. .
16	龙洁花,赵春江,林森,等.改进Mask R-CNN的温室环境下不同成熟度番茄果实分割方法[J].农业工程学报,2021,37(18):100-108.
	LONG J H, ZHAO C J, LIN S, et al. Improved segmentation method of tomato fruit with different ripeness in greenhouse environment by Mask R-CNN [J]. Trans. Chin. Soc. Agric. Eng., 2021,37(18):100-108.
17	AFONSO M, FONTEJIN H, FIORENTIN F S, et al.. Tomato fruit detection and counting in greenhouses using deep learning [J/OL]. Front. Plant Sci., 2020,11:571299 [2024-02-24]. .
18	刘之瑜,张淑芬,刘洋,等.基于图像梯度的数据增广方法[J].应用科学学报,2021,39(2):302-311.
	LIU Z Y, ZHANG S F, LIU Y, et al.. Data augmentation method based on image gradient [J]. J. Appl. Sci., 2021,39(2):302-311.
19	赵越,卫勇,单慧勇,等.基于深度学习的高分辨率麦穗图像检测方法[J].中国农业科技导报,2022,24(9):96-105.
	ZHAO Y, WEI Y, SHAN H Y, et al.. High resolution wheat image detection method based on deep learning [J]. J. Agric. Sci. Technol., 2022,24(9):96-105.
20	杨坚,钱振,张燕军,等.采用改进YOLOv4-tiny的复杂环境下番茄实时识别[J].农业工程学报,2022,38(9):215-221.
	YANG J, QIAN Z, ZHANG Y J, et al.. Tomato real-time recognition in complex environment using improved YOLOv4-tiny [J]. Trans. Chin. Soc. Agric. Eng., 2022,38(9):215-221.
21	XU P H, FANG N, LIU N, et al.. Visual recognition of cherry tomatoes in plant factory based on improved deep instance segmentation [J]. Comput. Electron. Agric., 2022,197:106991.
22	ANANDHAKRISHNAN T, JAISAKTHI S M. Deep convolutional neural networks for image based tomato leaf disease detection [J]. Sustain. Chem. Pharm., 2022,30:100793.
23	刘芳,刘玉坤,林森,等.基于改进型YOLO的复杂环境下番茄果实快速识别方法[J].农业机械学报,2020,51(6):229-237.
	LIU F, LIU Y K, LIN S, et al.. Rapid identification method of tomato fruit in complex environment based on improved YOLO [J]. Trans. Chin. Soc. Agric. Mach. 2020,51(6):229-237.
24	成伟,张文爱,冯青春,等.基于改进YOLOv3的温室番茄果实识别估产方法[J].中国农机化学报,2021,42(4):176-182.
	CHENG W, ZHANG W A, FENG Q C, et al.. Fruit recognition and yield estimation method of greenhouse tomato based on improved YOLOv3 [J]. J. Chin. Agric. Mech., 2021,42(4):176-182.
25	LI H P, LI C Y, LI G B, et al.. A real-time table grape detection method based on improved YOLOv4-tiny network in complex background [J]. Biosyst. Eng., 2021,212:347-359.
26	ZHANG Y C, ZHANG W B, YU J Y, et al.. Complete and accurate holly fruits counting using YOLOX object detection [J/OL]. Comput. Electron. Agric., 2022,198: 107062 [2024-02-26]. .
27	ZHANG Y J, MA B X, HU Y T, et al.. Accurate cotton diseases and pests detection in complex background based on an improved YOLOX model [J/OL]. Comput. Electron. Agric., 2022,203:107484 [2024-02-26]. .
28	何斌,张亦博,龚健林,等.基于改进YOLOv5的夜间温室番茄果实快速识别[J].农业机械学报,2022,53(5):201-208.
	HE B, ZHANG Y B, GONG J L, et al.. Rapid identification of greenhouse tomato fruits at night based on improved YOLOv5 [J]. Trans. Chin. Soc. Agric. Mach., 2022,53(5):201-208.
29	李柯泉,陈燕,刘佳晨,等.基于深度学习的目标检测算法综述[J].计算机工程,2022,48(7):1-12.
	LI K Q, CHEN Y, LIU J C, et al. An overview of object detection algorithms based on deep learning [J]. Comput. Eng., 2022,48(7):1-12.
30	岳有军,孙碧玉,王红君,等.基于级联卷积神经网络的番茄果实目标检测[J].科学技术与工程,2021,21(6):2387-2391.
	YUE Y J, SUN B Y, WANG H J, et al. Tomato fruit target detection based on cascaded convolutional neural network [J]. Sci. Technol. Eng., 2021,21(6):2387-2391.

模型Model	F1得分 F1 score	平均精度 AP/%	预加载模型识别速度 Preloaded model Recognition speed/s	未预加载模型识别速度 Unpreloaded model Recognition speed /s	可学习参数 Learnable parameter /Mb	模型大小 Model size/Mb
YOLOX-L-TN	0.525 1	58.82	0.445 4	3.711 2	62.2	219.41
YOLOX-L-Upsample	0.505 6	55.49	0.573 3	3.934 2	76.7	270.44
YOLOX-L-Spp	0.490 1	54.48	0.563 3	4.060 9	122.8	433.14
YOLOX-L	0.499 5	54.01	0.587 1	3.932 8	62.0	218.66
YOLOX-L-Connect	0.499 9	53.57	0.648 7	4.441 9	123.9	437.06

模型Model	F1得分 F1 score	平均精度 AP/%	预加载模型识别速度 Preloaded model Recognition speed/s	未预加载模型识别速度 Unpreloaded model Recognition speed /s	可学习参数 Learnable parameter /Mb	模型大小 Model size/Mb
YOLOX-L-TN	0.525 1	58.82	0.445 4	3.711 2	62.2	219.41
YOLOX-L-Upsample	0.505 6	55.49	0.573 3	3.934 2	76.7	270.44
YOLOX-L-Spp	0.490 1	54.48	0.563 3	4.060 9	122.8	433.14
YOLOX-L	0.499 5	54.01	0.587 1	3.932 8	62.0	218.66
YOLOX-L-Connect	0.499 9	53.57	0.648 7	4.441 9	123.9	437.06

模型Model	F1得分 F1 score	平均精度AP /%	预加载模型识别速度 Preloaded model Recognition speed/s	未预加载模型识别速度 Unpreloaded model Recognition speed/s	可学习参数 Learnable parameter/Mb	模型大小Model size /Mb
YOLOX-L-TN	0.525 1	58.82	0.445 4	3.711 2	62.2	219.41
YOLOX-L-SENet	0.523 5	58.29	0.464 5	3.808 1	62.1	218.66
YOLOX-L-CAM	0.501 4	54.63	0.470 4	3.946 0	62.1	218.79
YOLOX-L-CBAM	0.494 6	52.70	0.466 5	3.860 9	62.1	218.82
YOLOX-L-SAM	0.488 7	52.48	0.464 3	3.907 1	62.1	218.69

模型Model	F1得分 F1 score	平均精度AP /%	预加载模型识别速度 Preloaded model Recognition speed/s	未预加载模型识别速度 Unpreloaded model Recognition speed/s	可学习参数 Learnable parameter/Mb	模型大小Model size /Mb
YOLOX-L-TN	0.525 1	58.82	0.445 4	3.711 2	62.2	219.41
YOLOX-L-SENet	0.523 5	58.29	0.464 5	3.808 1	62.1	218.66
YOLOX-L-CAM	0.501 4	54.63	0.470 4	3.946 0	62.1	218.79
YOLOX-L-CBAM	0.494 6	52.70	0.466 5	3.860 9	62.1	218.82
YOLOX-L-SAM	0.488 7	52.48	0.464 3	3.907 1	62.1	218.69

[1]	Dejun ZHANG, Xiaoming ZHANG, Di WU, Shuyun LIN, Taihua ZHANG, Weiping XU. Simulation of Pepper Cleaning Based on DEM-CFD Coupling [J]. Journal of Agricultural Science and Technology, 2023, 25(7): 87-96.
[2]	Fengguang HE, Ganran DENG, Zhende CUI, Shuang ZHENG, Guojie LI, Ling LI, Shuangmei QIN. Operating Performance of Self-propelled Silage Cassava Stem and Leaf Harvester [J]. Journal of Agricultural Science and Technology, 2022, 24(8): 109-115.
[3]	Qi LIU, Bin CHEN, Songlin SUN, Mingtao XIAO, Chaoran SUN. Research on Mechanical Characteristics of Clamping Conveyor for Leafy Vegetables [J]. Journal of Agricultural Science and Technology, 2022, 24(4): 116-125.
[4]	Yiping DUAN, Mingliang WU, Jiangnan LYU, Wei XIANG, Bo YAN, Lan MA, Jiajie LIU. Research Status and Development Suggestions of No-tillage Seeding Anti-blocking Technology [J]. Journal of Agricultural Science and Technology, 2022, 24(2): 124-135.
[5]	ZHANG Fengkui, ZHANG Hong, LAN Haipeng, YU Fufeng, LI Zhongjie, LI Ping. Simulation Analysis of Pneumatic Conveying Device of Air-suction Jujube Picker Based on CFD-EDM Coupling [J]. Journal of Agricultural Science and Technology, 2021, 23(7): 107-116.
[6]	CHEN Qing-wen, HAN Zeng-de*, CUI Jun-wei, WANG Guo-xin, QIAO Xiao-dong, ZHANG Zi. Development Status and Trend Current Situation of Self-propelled Combine Harvester [J]. , 2015, 17(1): 109-114.
[7]	CHEN Hai-feng§, ZHANG Shi-xi§, YAN Yi-ye, CHEN Hao, XIE Qiang. Studies on Harvesting Ship of Water Chestnuts Based on Soil Liquefaction Principle [J]. , 2013, 15(2): 120-124.