Vegetable Recognition Based on Unmanned Aerial Vehicle （UAV） Multispectral Imagery and Random Forest Algorithm

doi:10.13304/j.nykjdb.2022.0027

Abstract

Abstract:

Real-time and accurate information about vegetable distribution is critical for improving the management of water/fertilizer and accurate estimation of the yield. Based on spectrum features of the segmented UAV multispectral images objects， other features including index， textural and geometric features were added to build eight classification schemes （S1～S8）， and then the random forest algorithm was used to classify and analyze the classification effect. The results showed that the scheme S5 （SPEC+GLCM+INDE） performed better than the others， with overall accuracy and Kappa of 92.75% and 0.92， respectively. We discovered that adding geometric features reduced the classification accuracy， while adding index and texture features had the opposite effect. It was difficult to distinguish the Chinese cabbage and cabbage effectively only by spectrum， index and texture features， so it was necessary to introduce features such as height to improve the accuracy in the follow-up research. The feature importance ranking results showed that spectrum features， followed by the index features， had the greatest impact on classification accuracy. To summarize， the method proposed in this study， which combined object-oriented and RF algorithm， could achieve higher classification accuracy based on multi-spectral images from UAV， and the important features that affect vegetable classification could be effectively identified， providing a reference for the accurate identification of other crops.

Key words: unmanned aerial vehicle, random forests, vegetable, multi-spectral image, object-oriented method

摘要：

实时准确的蔬菜种植信息是实现水肥精准管理和产量准确估算的重要基础。对无人机多光谱影像进行分割，以光谱特征（spectrum features，SPEC）为基础，分别引入指数特征（index features，INDE）、纹理特征（grey-level co-occurrence matrix features，GLCM）和几何特征（geometric features，GEOM）构建8个分类方案（S1～S8），使用随机森林算法进行分类并分析分类效果。结果表明，方案S5（SPEC+GLCM+INDE）的分类效果最好，总体精度和Kappa系数分别为92.75%和0.92。几何特征的引入降低了分类精度，而纹理和指数特征则与其相反；仅依靠光谱、指数和纹理特征仍难以有效区分白菜和包菜，为提高精度后续研究有必要引入植株高度等特征；在4大类特征中，重要性排在首位的是光谱特征，其次为指数特征。基于无人机多光谱影像和随机森林算法能获得较高的蔬菜分类精度，并能确认影响精度的重要特征，可为其他作物的精准识别提供借鉴。

关键词: 无人机, 随机森林, 蔬菜, 多光谱影像, 面向对象

CLC Number:

S127

Qian GUO, Jiahao WEI, Jian ZHANG, Zhangxi YE, Houxi ZHANG, Zhengqing LAI, Hui DENG. Vegetable Recognition Based on Unmanned Aerial Vehicle （UAV） Multispectral Imagery and Random Forest Algorithm[J]. Journal of Agricultural Science and Technology, 2023, 25(2): 99-110.

郭倩, 魏嘉豪, 张健, 叶章熙, 张厚喜, 赖正清, 邓辉. 基于无人机多光谱影像和随机森林的蔬菜识别[J]. 中国农业科技导报, 2023, 25(2): 99-110.

Figures/Tables 12

Fig. 1 RGB image of research area

Fig. 2 Technical flow

Table 1 Different classification schemes

方案 Scheme	子特征集 Feature subset	特征总数 Number of total features
S1	SPEC	12
S2	SPEC+GLCM	52
S3	SPEC+INDE	38
S4	SPEC+GEOM	30
S5	SPEC+GLCM+INDE	78
S6	SPEC+GLCM+GEOM	70
S7	SPEC+INDE+GEOM	56
S8	SPEC+GLCM+INDE+GEOM	96

Table 2 Samples for different categories

地物类型 Type of object	总样本数 Total samples
番薯 Sweet potato	150
葱 Shallot	60
油麦菜 Leaf lettuce	100
生菜 Lettuce	110
包菜 Cabbage	100
花椰菜 Cauliflower	100
芥菜 Mustard	70
白菜 Chinese cabbage	80
水 Water	60
土壤 Soil	150
地膜 Mulch film	70

Fig. 3 Object spectrum image

Fig. 4 Segmentation effect of different segmentation scales

Fig. 5 Out-of-bag error rate of different numbers of features and different numbers of decision trees for various schemes

Fig. 6 Classification accuracy of different schemesNote： × in the figure shows the location of the mean.

Fig. 7 Producer accuracy of different schemes for different vegetable categories

Fig. 8 User accuracy of different schemes for different vegetable categories

Fig. 9 The twenty most important features for different models

Fig. 10 Classification result based on the best scheme

References 24

1	赵建鹏,杨秀峰,李国洪,等.基于面向对象的设施蔬菜高分遥感影像提取[J].江苏农业学报, 2019, 35(4): 911-918.
	ZHAO J P, YANG X F, LI G H, et al.. Object oriented extraction of high resolution remote sensing images of facility vegetables [J]. Jiangsu J. Agric. Sci., 2019, 35(4): 911-918.
2	樊鸿叶,李姚姚,卢宪菊,等.基于无人机多光谱遥感的春玉米叶面积指数和地上部生物量估算模型比较研究[J].中国农业科技导报, 2021, 23(9): 112-120.
	FAN H Y, LI, Y Y, LU X J, et al.. Comparative analysis of LAI and above-ground biomass estimation models based UAV multispectral remote sensing [J]. J. Agric. Sci. Technol., 2021, 23(9): 112-120.
3	VARELA S, DHODDA P R, HSU W H, et al.. Early-season stand count determination in corn via integration of imagery from unmanned aerial systems (UAS) and supervised learning techniques [J/OL]. Remote Sens., 2018, 10(2): 343 [2021-12-10]. .
4	WANG W J, ZHAO Z M, ZHU H Q. Object-oriented change detection method based on multi-scale and multi-feature fusion [C]//2009 Joint Urban Remote Sensing Event, IEEE, 2009: 1-5.
5	叶章熙,郭倩,张健,等.基于无人机可见光影像与OBIA-RF算法的城市不透水面提取[J].农业工程学报,2022,38(4):225-234.
	YE Z X, GUO Q, ZHANG J, et al.. Extraction of urban impervious surface based on the visible images of UAV and OBIA-RF algorithm [J]. Trans. Chin. Soc. Agric. Eng., 2022, 38(4): 225-234.
6	LEE R Y, CHANG K C, OU D Y, et al.. Evaluation of crop mapping on fragmented and complex slope farmlands through random forest and object-oriented analysis using unmanned aerial vehicles [J]. Geocar. Int., 2020, 35(12): 1293-1310.
7	WANG X X, GAO X W, ZHANG Y Z, et al.. Land-cover classification of coastal wetlands using the RF algorithm for Worldview-2 and Landsat 8 images [J/OL]. Remote Sens., 2019, 11(16): 1927 [2021-12-10]. .
8	刘舒,姜琦刚,马玥,等.基于多目标遗传随机森林特征选择的面向对象湿地分类[J].农业机械学报, 2017, 48(1): 119-127.
	LIU S, JIANG Q G, MA Y, et al.. Object-oriented wetland classification based on hybrid feature selection method combining with Relief F, multi-objective genetic algorithm and random forest [J]. Trans. Chin. Soc. Agric. Machin., 2017, 48(1): 119-127.
9	刘舒,朱航.基于超高空间分辩率无人机影像的面向对象土地利用分类方法[J].农业工程学报, 2020, 36(2): 87-94.
	LIU S, ZHU H. Object-oriented land use classification based on ultra-high resolution images taken by unmanned aerial vehicle [J]. Trans. Chin. Soc. Agric. Eng., 2020, 36(2): 87-94.
10	ROUSE J W, HAAS R H, SCHELL J A, et al.. Monitoring the vernal advancement of retrogradation (green wave effect) of natural vegetation [J]. NASA, 1974:1-12.
11	BARNES E M, CLARKE T R, RICHARDS S E, et al.. Coincident detection of crop water stress, nitrogen status, and canopy density using ground based multispectral data [C]//Proceedings of the Fifth International Conference on Precision Agriculture, Bloomington, MN, USA, 2000: 1619.
12	GITELSON A A, KAUFMAN Y J, MERZLYAK M N. Use of a green channel in remote sensing of global vegetation from EOS-MODIS [J]. Remote Sens. Environ., 1996, 58(3): 289-298.
13	JORDAN C F. Derivation of leaf‐area index from quality of light on the forest floor [J]. Ecology, 1969, 50(4): 663-666.
14	陈鹏,冯海宽,李长春,等.无人机影像光谱和纹理融合信息估算马铃薯叶片叶绿素含量[J].农业工程学报, 2019, 35(11): 63-74.
	CHEN P, FENG H K, LI C C, et al.. Estimation of chlorophyll content in potato using fusion of texture and spectral features derived from UAV multispectral image [J]. Trans. Chin. Soc. Agric. Eng., 2019, 35(11): 63-74.
15	HUETE A R. A soil-adjusted vegetation index (SAVI) [J]. Remote Sens. Environ., 1988, 25(3): 295-309.
16	GAO B C. NDWI-a normalized difference water index for remote sensing of vegetation liquid water from space [J]. Remote Sens. Environ., 1996, 58(3): 257-266.
17	尹慧新,樊彦国.基于"珠海一号"高光谱卫星的洱海水体提取方法研究[J].航天返回与遥感, 2020, 41(4): 118-127.
	YIN H X, FAN Y G. Research on the extraction method of water in Erhai Lake based on OHS-2A [J]. Spacecraft Recov. Remote Sens., 2020, 41(4): 118-127.
18	杜军,宁晓刚,刘纪平,等.基于遥感监测的北京市城市空间扩展格局与形态特征分析[J].地域研究与开发, 2019, 38(2): 73-78.
	DU J, NING X G, LIU J P, et al.. Analysis of urban expansion pattern and spatial morphologic characteristics of Beijing based on remote sensing [J]. Areal Res. Dev., 2019, 38(2): 73-78.
19	冯志立,肖锋,卢小平,等.基于随机森林特征优选的冬小麦分类方法[J].测绘通报, 2022(3):70-75.
	FENG Z L, XIAO F, LU X P, et al.. Winter wheat classification method based on festure optimization of random forest [J]. Bull. Surv. Map., 2022(3):70-75.
20	夏盈,厉恩华,王学雷,等.基于特征优选的随机森林算法在湿地信息提取中的应用——以湖北洪湖湿地自然保护区为例[J].华中师范大学学报(自然科学版), 2021, 55(4): 639-648, 660.
	XIA Y, LI E H, WANG X L, et al.. Application of random forest algorithm based on feature optimization in wetland information extraction: a case study of Honghu wetland nature reserve in Hubei province [J]. J. Central Chin. Normal Univ. (Nat. Sci.), 2021, 55(4):639-648, 660.
21	NGUYEN L H, JOSHI D R, CLAY D E, et al.. Characterizing land cover/land use from multiple years of Landsat and MODIS time series: a novel approach using land surface phenology modeling and random forest classifier [J/OL]. Remote Sens. Environ., 2020, 238: 111017 [2021-12-10]. .
22	HAO P, ZHAN Y, WANG L, et al.. Feature selection of time series MODIS data for early crop classification using random forest: a case study in Kansas, USA [J]. Remote Sens., 2015, 7(5): 5347-5369.
23	RODRIGUEZ-GALIANO V F, CHICA-OLMO M, ABARCA-HERNANDEZ F, et al.. Random forest classification of Mediterranean land cover using multi-seasonal imagery and multi-seasonal texture [J]. Remote Sens. Environ., 2012, 121: 93-107.
24	CHAN J C W, PAELINCKX D. Evaluation of random forest and adaboost tree-based ensemble classification and spectral band selection for ecotope mapping using airborne hyperspectral imagery [J]. Remote Sens. Environ., 2008, 112(6): 2999-3011.

[1]	Yan MENG, Wei WANG, Quancai XI, Yi LI, Laisheng CHEN, Zhongping DU, Rui HAN. Effect of Biogas Slurry Pretreatment on Anaerobic Digestion of Vegetable Straws [J]. Journal of Agricultural Science and Technology, 2022, 24(9): 188-196.
[2]	Qi LIU, Bin CHEN, Songlin SUN, Mingtao XIAO, Chaoran SUN. Research on Mechanical Characteristics of Clamping Conveyor for Leafy Vegetables [J]. Journal of Agricultural Science and Technology, 2022, 24(4): 116-125.
[3]	Zihong YAN, Dongsheng FAN, Yanliang ZHAO, Zhizhong PENG, Limin ZHENG, Tian XIE, Zhiwei MIAO. Development of Fully Water⁃soluble Ammonium Polyphosphate⁃rare Earth Slow⁃release Fertilizer and Its Application Effect to Vegetables [J]. Journal of Agricultural Science and Technology, 2022, 24(1): 157-163.
[4]	SHI Yangjie, XI Xiaobo, YUAN Rican, SHAN Xiang, ZHANG Qi, ZHAO Qidong, ZHANG Ruihong. Design and Experiment of Microwave Vacuum Drying Equipment with CO₂ Cold Trap [J]. Journal of Agricultural Science and Technology, 2021, 23(6): 97-104.
[5]	HUANG Li1,2, DENG Yishu1, PU Entang2, DAI Xuefang2, LI Wenxi2. Simultanous Determination of Three Fungicides in Vegetables by UPLC-MS/MS#br# [J]. Journal of Agricultural Science and Technology, 2021, 23(3): 99-104.
[6]	CHEN Huiying, LIU Xiaoqing, ZHOU Yanping. Current Situation and Development Suggestions on Vegetable Commercial Breeding in China [J]. Journal of Agricultural Science and Technology, 2021, 23(12): 1-6.
[7]	FAN Xin1, SHANG Delin1, LAN Yubin1, BAI Jingbo2, HAN Xin1*. Application of Ozone Technology and Ozone Plant Protection Devices in Agriculture [J]. Journal of Agricultural Science and Technology, 2020, 22(5): 71-77.
[8]	JIN Yue, XIAO Hongru*, XIAO Suwei, XU Meng, DING Wenqin, LIU Dong. Research Statue and Development Trendency on Leaf Vegetable Harvesting Technology and Equipment [J]. Journal of Agricultural Science and Technology, 2018, 20(9): 72-78.
[9]	XIANG Yanyan1, HUANG Yunxiang1*, HUANG Chuyu2, LIU Lishan1, LONG Quan1, LONG Huaiyu3. Influence of Different Water Supply Negative Pressure on Growth and Main Physiological Indexes of Chinese Cabbage [J]. Journal of Agricultural Science and Technology, 2018, 20(8): 16-22.
[10]	TIAN Kunpeng, ZHANG Bin*, LI Xianwang, HUANG Jicheng, SHEN Cheng. Study on Die Hole Structure Design and Simulation Optimization of Fruit and Vegetable Skin Residue Granular Molding Machine [J]. Journal of Agricultural Science and Technology, 2018, 20(6): 62-68.
[11]	XU Weicheng1, CHEN Kai1, ZHANG Ming1, LI Zhengfeng2, ZHANG Tianshun1*. Life Lifting Scheme for Plant Protection UAV Based on Solar [J]. Journal of Agricultural Science and Technology, 2018, 20(11): 62-68.
[12]	QIU Rongzhou 1, CHI Meixiang1, CHEN Hong2, WENG Qiyong3, ZHAO Jian2*. Development and Application of the Safe Production and Quality Traceability System for Vegetable [J]. Journal of Agricultural Science and Technology, 2017, 19(4): 51-58.
[13]	HE Wei, YANG Zhen-chao*, CAI Hua, WANG Da-fei, WANG Xiao-xu. Effects of Light Quality on Growth and Morphologenesis Regulation of Vegetables: A Review [J]. Journal of Agricultural Science and Technology, 2016, 18(2): 9-18.
[14]	WANG Ya-kun1, WANG Hui-jun1,2*. Studies on Protected Vegetable Production Efficiency in China [J]. , 2015, 17(2): 159-166.
[15]	HUANG Baoyong, SUN Jiang, XIAO Zhiyong. Risk Assessment of Chronic Intaking Chlorpyrifos in Vegetable in Beijing [J]. , 2013, 15(4): 55-60.