Diagnosis of Crop Disease Based on Multi-task Learning

doi:10.13304/j.nykjdb.2022.0650

Abstract

Abstract:

In order to judge the severity of crop diseases quickly and accurately， a novel online remote diagnosis method was proposed based on multi-task learning in this paper. The classic model MobileNetV3 was improved by introducing convolutional block attention module and feature pyramid module to boost the performance of the recognition of crops， diseases and pests， and disease levels. Besides， some data augment methods were adopted to extend the small samples. The performance of the improved model and other image recognition models in the identification of crop disease was tested， and the performance of different models with and without data enhancement processing was explored. The results showed that the mean average precision of proposed method on such 3 tasks was more than that of the original model by 1.38， 2.24 and 2.03 percentage points， respectively， and the average recall of proposed method on such 3 tasks was more than that of the original model by 2.38， 1.62 and 1.18 percentage points， respectively. The proposed method outperformed the state-of-the-art methods， such as MobileNetV3， InceptionV3 and YOLOv7.

Key words: intelligent agriculture, disease recognition, deep learning, convolutional neural network, multi-task learning

摘要：

为了快速、准确判别农作物叶片病害图像的病害类型及病害程度，提出基于多任务学习的诊断方法。引入通道和空间注意力模型，对经典的MobileNetV3网络模型进行改进，并在此基础上构建基于特征金字塔的多任务深度卷积神经网络模型，实现作物类型、病害类型和病害程度的精准识别。采用多种图像增强方法对农作物叶片病害图像进行扩展，对改进前后模型与其他图像识别模型在农作物病害叶片识别性能上进行对比试验，并探究在有无数据增强处理条件下不同模型的性能。结果表明：该模型在作物类型识别、病害类型识别与病害程度识别任务上，平均准确率比原模型分别提升1.38、2.24和2.03个百分点；召回率比原模型分别提升2.38、1.62和1.18个百分点；对比MobileNetV3，InceptionV3、YOLOv7模型，该模型在上述3个任务上平均识别准确率和召回率均达到最高。

关键词: 智慧农业, 病害识别, 深度学习, 卷积神经网络, 多任务学习

CLC Number:

S126

Guo ZHENG, Yusong JIANG. Diagnosis of Crop Disease Based on Multi-task Learning[J]. Journal of Agricultural Science and Technology, 2024, 26(1): 89-98.

郑果, 姜玉松. 基于多任务学习农作物叶片病害诊断方法[J]. 中国农业科技导报, 2024, 26(1): 89-98.

Figures/Tables 9

References 24

1	KAUR S, PANDEY S, GOEL S. Plants disease identification and classification through leaf images: a survey [J]. Arch. Comp. Methods Eng., 2018, 26(4): 507-530.
2	ESGARIO J G, KROHLING R A, VENTURA J A, et al.. Deep learning for classification and severity estimation of coffee leaf biotic stress [J]. Comp. Electron. Agric., 2020, 16(9): 155-162.
3	张帅堂,王紫烟,邹修国,等. 基于高光谱图像和遗传优化神经网络的茶叶病斑识别[J]. 农业工程学报, 2017, 33(22): 200-207.
	ZHANG S T, WANG Z Y, ZOU X G, et al.. Recognition of tea disease spot based on hyperspectral image and genetic optimization neural network [J]. Trans. Chin. Soc. Agric. Eng., 2017, 33(22): 200-207.
4	孙俊,谭文军,毛罕平,等. 基于改进卷积神经网络的多种植物叶片病害识别[J]. 农业工程学报, 2017, 33(19): 209-215.
	SUN J, TAN W J, MAO H P, et al. Recognition of multiple plant leaf diseases based on improved convolutional neural network [J]. Trans. Chin. Soc. Agric. Eng., 2017, 33(19): 209-215.
5	MOHANTY S P, HUGHES D P, SALATHE M, et al.. Using deep learning for image-based plant disease detection [J]. Front. Plant Sci., 2016, 7(1): 1-10.
6	马浚诚,杜克明,郑飞翔,等. 基于卷积神经网络的温室黄瓜病害识别系统[J]. 农业工程学报, 2018, 34(12): 186-192.
	MA J C, DU K M, ZHENG F X, et al.. Disease recognition system for greenhouse cucumbers based on deep convolutional neural network [J]. Trans. Chin. Soc. Agric. Eng., 2018, 34(12):186-192
7	张建华,孔繁涛,吴建寨,等. 基于改进VGG卷积神经网络的棉花病害识别模型[J].中国农业大学学报, 2018, 23(11): 167-177.
	ZHANG J H, KONG F T, WU J Z, et al.. Image recognition of cotton leaf diseases based on improved VGG convolutional neural network [J]. J. Chin. Agric. Univ., 2018,23(11):167-177.
8	刘瑾蓉,林剑辉,李婷婷. 基于卷积神经网络的银杏叶片患病程度识别[J]. 中国农业科技导报, 2018, 20(6):55-61.
	LIU J R, LIN J H, LI T T. Identification of leaf disease grade based on convolutional neural network [J]. J. Agric. Sci. Tech., 2018, 20(6):55-61.
9	曾伟辉,李淼,张健,等.面向农作物病害识别的高阶残差卷积神经网络研究[J].中国科学技术大学学报,2019,49(10):781-791.
	ZENG W H, LI M, HUANG J, et al.. Research on high-order residual convolution neural network for crop disease recognition application [J]. J. Univ. Sci. Tech. Chin. 2019, 49(10):781-791.
10	李淼,王敬贤,李华龙,等.基于CNN 和迁移学习的农作物病害识别方法研究[J].智慧农业, 2019, 1(3): 46-55.
	LI M, WANG J X, LI H L, et al.. Method for identifying crop disease based on CNN and transfer learning [J]. Smart Agric., 2019, 1(3):46-55.
11	魏超,范自柱,张泓,等. 基于深度学习的农作物病害检测[J]. 江苏大学学报(自然科学版), 2019, 40(2): 71-77.
	WEI C, WEI C, FAN Z Z, ZHANG H, et al.. Crop disease detection based on deep learning [J]. J. Jiangsu Univ. (Nat. Sci.), 2019, 40(2):71-77.
12	刘洋,冯全,王书志. 基于轻量级CNN的植物病害识别方法及移动端应用[J]. 农业工程学报, 2019, 35(17): 194-204.
	LIU Y, FENG Q, WANG S Z. Plant disease identification method based on lightweight CNN and mobile application [J]. Trans. Chin. Soc. Agric. Eng., 2019, 35(17):194-204.
13	JIANG P, CHEN Y, LIU B, et al.. Real-time detection of apple leaf diseases using deep learning approach based on improved convolutional neural networks [J]. IEEE Access, 2019, 7: 59069-59080.
14	许景辉,邵明烨,王一琛,等.基于迁移学习的卷积神经网络玉米病害图像识别[J].农业机械学报,2020,51(2):230-236.
	XU J H, SHAO M Y, WANG Y C, et al.. Recognition of corn leaf spot and rust based on transfer learning with convolutional neural network [J]. Tans. Chin. Soc. Agric. Mach., 2020, 51(2):230-236.
15	CHEN J, ZHANG D, NANEHKARAN Y A. Identifying plant diseases using deep transfer learning and enhanced lightweight network [J]. Multimedia Tools App., 2020, 79(7): 31497-31515.
16	赵立新,侯发东,吕正超,等.基于迁移学习的棉花叶部病虫害图像识别[J]. 农业工程学报,2020,36(7):184-191.
	ZHAO L X, HOU F D, LYU Z C, et al.. Image recognition of cotton leaf diseases and pests based on transfer learning [J]. Trans. Chin. Soc. Agric. Eng., 2020, 36(7):184-191.
17	王春山,周冀,吴华瑞,.改进Multi-scale ResNet的蔬菜叶部病害识别[J].农业工程学报,2020, 36(20):217-225.
	WANG C S, ZHOU J, WU H R, et al.. Identification of vegetable leaf diseases based on improved multi-scale ResNet [J]. Trans. Chin. Soc. Agric. Eng., 2020, 36(20):217-225.
18	刘阳,高国琴. 采用改进的 SqueezeNet 模型识别多类叶片病害[J]. 农业工程学报,2021,37(2) :187-195.
	LIU Y, GAO G Q. Identification of multiple leaf diseases using improved SqueezeNet model [J]. Trans. Chin. Soc. Agric. Eng., 2021,37(2) :187-195.
19	HOWARD A., SANDLER M., CHEN B., et al.. Searching for MobileNetV3 [C]// Proceedigns of IEEE/CVF International Conference on Computer Vision (ICCV). South Korea, 2019: 1314-1324.
20	SANGHYUN W, JONGCHAN P, JOON-YOUNG L, et al.. CBAM: convolutional block attention module [C]// Proceedings of European Conference on Computer Vision. Munich, Germany: 2018: 3-19.
21	SENER O, KOLTUN V. Multi-task learning as multi-objective optimization [J/OL]. arXiv:1810.04650v2 [cs.LG]. [2023-08-30] . .
22	LIN T Y, DOLLAR P, GIRSHICK R., et al.. Feature pyramid networks for object detection [C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, 2017:936-944.
23	SZEGEDY C, VANHOUCKE V, IOFFE S, et al.. Rethinking the inception architecture for computer vision [C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2016: 2818-2826.
24	WANG C Y, ALEXEY B, LIAO H Y., YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors [J/OL]. arXiv:2207.02696v1 [cs.CV] [2023-08-30]. .

植物 Plant	类别编号 Class code	类别名称 Class name	图像数量 Number of images	植物 Plant	类别编号 Class code	类别名称 Class name	图像数量 Number of images
苹果Apple	C1	疮痂病Corky scab	630		C20	健康叶health	1 478
	C2	黑腐病Black rot	622	马铃薯 Potato	C21	早疫病Early blignt	1 000
	C3	锈病Rust	275		C22	健康叶health	152
	C4	健康Health	1 645		C23	晚疫病Late blight	1 000
蓝莓 Blueberry	C5	健康Health	1 502		C24	干腐病 Mummy disease	371
樱桃 Cherry	C6	健康Health	854	大豆Soybean	C25	健康Health	5 090
樱桃 Cherry	C7	白粉病 Powdery mildew	1 052	南瓜Pumpkin	C26	白粉病 Powdery mildew	1 835
玉米 Corn	C8	灰斑病 Gray leaf spot	513	草莓Strayberry	C27	健康Health	456
	C9	锈病Rust	1 192	草莓Strayberry	C28	叶焦病Tipburn	1 109
	C10	健康Health	1 162	番茄 Tomato	C29	斑点病Scab	2 127
	C11	枯叶病Leaf blight	985		C30	早疫病Early blignt	1 000
葡萄Grape	C12	黑腐病Black rot	1 180		C31	健康Health	1 591
	C13	黑豆病Black bean	1 383		C32	晚疫病Late blight	1 909
	C14	健康Health	423		C33	叶霉病Leaf mold	952
	C15	枯叶病Leaf blight	1 076		C34	斑枯病Septoria	1 771
桔子Orange	C16	黄龙病 Yellow shoot	5 507		C35	叶螨病Acariasis	1 667
桃Peach	C17	斑点病Scab	2 297		C36	轮斑病Zonate spot	1 404
桃Peach	C18	健康Health	360		C37	黄叶病Yellowtop	373
辣椒Pepper	C19	斑点病Scab	997		C38	黄曲病 Yellow aspergillosis	5 357

植物 Plant	类别编号 Class code	类别名称 Class name	图像数量 Number of images	植物 Plant	类别编号 Class code	类别名称 Class name	图像数量 Number of images
苹果Apple	C1	疮痂病Corky scab	630		C20	健康叶health	1 478
	C2	黑腐病Black rot	622	马铃薯 Potato	C21	早疫病Early blignt	1 000
	C3	锈病Rust	275		C22	健康叶health	152
	C4	健康Health	1 645		C23	晚疫病Late blight	1 000
蓝莓 Blueberry	C5	健康Health	1 502		C24	干腐病 Mummy disease	371
樱桃 Cherry	C6	健康Health	854	大豆Soybean	C25	健康Health	5 090
樱桃 Cherry	C7	白粉病 Powdery mildew	1 052	南瓜Pumpkin	C26	白粉病 Powdery mildew	1 835
玉米 Corn	C8	灰斑病 Gray leaf spot	513	草莓Strayberry	C27	健康Health	456
	C9	锈病Rust	1 192	草莓Strayberry	C28	叶焦病Tipburn	1 109
	C10	健康Health	1 162	番茄 Tomato	C29	斑点病Scab	2 127
	C11	枯叶病Leaf blight	985		C30	早疫病Early blignt	1 000
葡萄Grape	C12	黑腐病Black rot	1 180		C31	健康Health	1 591
	C13	黑豆病Black bean	1 383		C32	晚疫病Late blight	1 909
	C14	健康Health	423		C33	叶霉病Leaf mold	952
	C15	枯叶病Leaf blight	1 076		C34	斑枯病Septoria	1 771
桔子Orange	C16	黄龙病 Yellow shoot	5 507		C35	叶螨病Acariasis	1 667
桃Peach	C17	斑点病Scab	2 297		C36	轮斑病Zonate spot	1 404
桃Peach	C18	健康Health	360		C37	黄叶病Yellowtop	373
辣椒Pepper	C19	斑点病Scab	997		C38	黄曲病 Yellow aspergillosis	5 357

类别编号 Class code	准确率 Accuracy/%				召回率 Recall/%
类别编号 Class code	MobileNetV3	InceptionV3	YOLOV7	本文Proposed	MobileNetV3	InceptionV3	YOLOV7	本文Proposed
C1	92.86	93.65	93.78	93.89	94.32	95.68	95.23	95.26
C2	98.40	97.58	98.82	99.12	96.53	95.61	96.79	97.32
C3	100.00	100.00	99.98	100.00	98.65	99.22	99.38	99.62
C4	99.39	96.96	98.96	100.00	92.43	93.95	94.62	96.25
C5	100.00	100.00	100.00	100.00	95.42	96.98	95.54	95.89
C6	97.08	98.83	98.89	98.65	98.26	97.88	98.32	98.15
C7	99.00	99.00	99.13	99.20	95.24	96.32	96.85	97.24
C8	83.50	92.23	90.85	91.84	86.48	92.23	92.27	90.62
C9	98.74	100.00	99.78	99.69	96.64	97.04	96.93	96.89
C10	99.57	99.98	99.92	100.00	97.88	98.06	98.17	98.56
C11	93.88	90.36	93.81	94.23	94.56	93.86	94.32	94.72
C12	98.73	99.58	99.18	99.35	97.92	98.88	98.30	97.45
C13	97.84	98.56	98.49	98.52	98.26	98.89	98.56	98.43
C14	96.47	100.0	98.93	99.88	95.39	96.03	97.35	98.48
C15	99.07	99.07	99.18	99.28	97.35	96.98	97.46	97.88
C16	99.30	99.40	99.47	99.50	96.26	97.87	97.92	98.12
C17	98.03	98.26	99.03	98.54	97.34	97.25	97.17	97.96
C18	95.83	97.22	97.59	98.12	96.46	96.29	97.92	97.82
C19	100.0	96.48	98.94	100.0	98.13	97.68	98.37	98.80
C20	93.58	97.97	96.42	96.80	96.36	98.34	98.41	97.87
C21	93.50	96.50	96.52	96.32	96.34	98.56	97.78	97.93
C22	83.33	93.33	93.90	94.86	92.42	94.13	94.87	95.46
C23	91.00	97.50	97.83	98.21	93.32	96.62	96.07	97.31
C24	100.0	100.00	99.95	100.0	98.34	97.89	99.18	99.12
C25	98.72	97.45	97.98	98.45	98.72	97.92	97.52	97.86
C26	100.0	99.18	99.93	100.0	98.12	97.48	98.52	99.04
C27	97.80	97.80	98.15	98.24	96.47	96.93	97.16	97.72
C28	100.0	97.30	98.95	100.0	98.42	97.94	98.37	98.60
C29	99.53	93.41	98.86	99.80	99.13	94.76	97.20	96.81
C30	70.00	68.00	75.26	76.21	86.34	88.67	89.50	89.45
C31	95.91	89.31	95.97	96.42	96.23	92.39	96.91	96.82
C32	84.25	90.58	91.62	90.68	89.34	91.50	90.82	91.88
C33	96.84	90.53	97.28	97.65	95.89	93.59	96.17	96.25
C34	83.90	90.40	91.59	92.78	90.42	91.47	92.09	92.18
C35	97.31	90.15	98.10	98.34	96.34	95.89	96.39	96.82
C36	81.14	91.81	92.19	92.60	90.23	91.90	90.45	91.96
C37	97.33	100.0	99.39	99.78	98.46	99.00	98.28	98.38
C38	98.79	95.14	98.38	98.90	96.34	96.89	97.14	97.27
均值Average	95.02	95.62	96.92	97.26	95.99	95.98	96.43	97.17

类别编号 Class code	准确率 Accuracy/%				召回率 Recall/%
类别编号 Class code	MobileNetV3	InceptionV3	YOLOV7	本文Proposed	MobileNetV3	InceptionV3	YOLOV7	本文Proposed
C1	92.86	93.65	93.78	93.89	94.32	95.68	95.23	95.26
C2	98.40	97.58	98.82	99.12	96.53	95.61	96.79	97.32
C3	100.00	100.00	99.98	100.00	98.65	99.22	99.38	99.62
C4	99.39	96.96	98.96	100.00	92.43	93.95	94.62	96.25
C5	100.00	100.00	100.00	100.00	95.42	96.98	95.54	95.89
C6	97.08	98.83	98.89	98.65	98.26	97.88	98.32	98.15
C7	99.00	99.00	99.13	99.20	95.24	96.32	96.85	97.24
C8	83.50	92.23	90.85	91.84	86.48	92.23	92.27	90.62
C9	98.74	100.00	99.78	99.69	96.64	97.04	96.93	96.89
C10	99.57	99.98	99.92	100.00	97.88	98.06	98.17	98.56
C11	93.88	90.36	93.81	94.23	94.56	93.86	94.32	94.72
C12	98.73	99.58	99.18	99.35	97.92	98.88	98.30	97.45
C13	97.84	98.56	98.49	98.52	98.26	98.89	98.56	98.43
C14	96.47	100.0	98.93	99.88	95.39	96.03	97.35	98.48
C15	99.07	99.07	99.18	99.28	97.35	96.98	97.46	97.88
C16	99.30	99.40	99.47	99.50	96.26	97.87	97.92	98.12
C17	98.03	98.26	99.03	98.54	97.34	97.25	97.17	97.96
C18	95.83	97.22	97.59	98.12	96.46	96.29	97.92	97.82
C19	100.0	96.48	98.94	100.0	98.13	97.68	98.37	98.80
C20	93.58	97.97	96.42	96.80	96.36	98.34	98.41	97.87
C21	93.50	96.50	96.52	96.32	96.34	98.56	97.78	97.93
C22	83.33	93.33	93.90	94.86	92.42	94.13	94.87	95.46
C23	91.00	97.50	97.83	98.21	93.32	96.62	96.07	97.31
C24	100.0	100.00	99.95	100.0	98.34	97.89	99.18	99.12
C25	98.72	97.45	97.98	98.45	98.72	97.92	97.52	97.86
C26	100.0	99.18	99.93	100.0	98.12	97.48	98.52	99.04
C27	97.80	97.80	98.15	98.24	96.47	96.93	97.16	97.72
C28	100.0	97.30	98.95	100.0	98.42	97.94	98.37	98.60
C29	99.53	93.41	98.86	99.80	99.13	94.76	97.20	96.81
C30	70.00	68.00	75.26	76.21	86.34	88.67	89.50	89.45
C31	95.91	89.31	95.97	96.42	96.23	92.39	96.91	96.82
C32	84.25	90.58	91.62	90.68	89.34	91.50	90.82	91.88
C33	96.84	90.53	97.28	97.65	95.89	93.59	96.17	96.25
C34	83.90	90.40	91.59	92.78	90.42	91.47	92.09	92.18
C35	97.31	90.15	98.10	98.34	96.34	95.89	96.39	96.82
C36	81.14	91.81	92.19	92.60	90.23	91.90	90.45	91.96
C37	97.33	100.0	99.39	99.78	98.46	99.00	98.28	98.38
C38	98.79	95.14	98.38	98.90	96.34	96.89	97.14	97.27
均值Average	95.02	95.62	96.92	97.26	95.99	95.98	96.43	97.17

编号 Code	植物 Plant	准确率 Accuracy/%				召回率 Recall/%
编号 Code	植物 Plant	MobileNetV3	InceptionV3	YOLOV7	本文 Proposed	MobileNetV3	InceptionV3	YOLOV7	本文Proposed
P1	苹果Apple	95.56	96.25	96.58	97.82	92.25	91.86	92.08	93.62
P2	蓝莓Cherry	93.40	92.68	94.87	95.42	94.46	93.89	95.13	96.82
P3	樱桃Cherry	99.20	99.31	99.53	100.0	98.80	98.82	99.05	99.86
P4	玉米Corn	98.67	97.99	98.92	100.0	96.58	96.82	98.13	97.56
P5	葡萄Grape	100.0	100.0	99.95	100.0	95.82	94.67	95.19	95.25
P6	橘子Orange	95.88	97.89	97.68	98.65	96.78	95.96	97.32	99.45
P7	桃Peach	98.90	99.10	99.09	99.20	96.86	98.65	97.94	98.84
P8	辣椒Pepper	93.50	98.23	98.15	96.35	92.59	96.73	96.82	96.42
P9	土豆Potato	99.89	99.92	99.96	99.88	96.88	97.80	96.51	96.98
P10	大豆Soybean	98.45	99.96	98.87	99.68	96.42	98.98	96.82	97.82
P11	南瓜Pumpkin	97.87	96.84	97.56	98.60	96.82	97.56	98.23	98.62
P12	草莓Strayberry	95.39	98.93	99.53	98.89	94.32	98.26	96.95	97.88
P13	番茄Tomato	98.78	95.14	96.68	98.96	97.72	96.84	97.84	98.26
平均Average		97.35	97.86	98.26	98.73	95.87	96.68	96.77	97.49