基于三维点云的大豆植株器官分割及表型分析

doi:10.13304/j.nykjdb.2022.0861

摘要/Abstract

摘要：

为了解决大豆等多分枝作物叶片成簇、叶片相互遮挡带来的高通量表型测量困难问题，提出了基于植株三维点云的器官分割及表型参数测量方法。以分枝期大豆植株为研究对象，采集植株多视角图像，利用三维重建技术得到植株稠密点云、过滤点云噪声并还原实际尺度；以法线微分差异算法、改进的区域生长算法以及点云曲率特征实现植株各器官的分割；最后采用有向包围盒、改进的三角剖分法以及最邻近算法提取植株叶面积、叶宽、叶长、叶倾角和茎粗等表型参数。试验结果表明，器官分割后冠层叶片点云平均分割率为84.24%，单叶点云分割率均高于95.29%，表型参数测量值与人工实测值具有较强相关性，叶面积、叶宽、叶长、叶倾角和茎粗测量值与人工实测值的决定系数分别为0.987 9、0.961 3、0.962 6、0.931 1和0.963 4，均方根误差分别为0.541 7 cm²、0.141 2 cm、0.175 5 cm、3.279 6°和0.047 5 cm。提出的方法对叶片相互粘连的植株具有较好的分割效果，为多分枝作物的器官分割及表型参数测量提供了有效的解决方案。

关键词: 三维点云, 器官分割, 叶片分割, 表型参数测量

Abstract:

In order to solve the problem of high-throughput phenotypic measurements caused by clustering and mutual shading of leaves of multi-branched crops such as soybean， a method of organ segmentation and phenotypic parameter measurement based on three-dimensional（3D）point clouds of the plant was proposed. Firstly， multi-view images of soybean plants were collected at the branching stage and used 3D reconstruction technology to obtain dense point clouds of plants， filtered point cloud noise and restored the actual scale. Further， the difference of normals algorithm， the improved regional growth algorithm and the point clouds curvature feature were used to segment the organs of the plant. Finally， the leaf area， leaf width， leaf length， leaf inclination angle and stem diameter were extracted by using the oriented bounding box， improved triangulation and the nearest neighbor algorithm. The results showed that the average segmentation rate of canopy leaf point clouds after organ segmentation was 84.24%， and the segmentation rate of single leaf point clouds was higher than 95.29%， and the measured values of phenotypic parameters had strong correlation with the manually measured values. The coefficients of determination of leaf area， leaf width， leaf length， leaf inclination and stem diameter measurements and manual measurements were 0.987 9， 0.961 3， 0.962 6， 0.931 1 and 0.963 4， respectively， with root mean square errors of 0.541 7 cm²， 0.141 2 cm， 0.175 5 cm， 3.279 6° and 0.047 5 cm. The proposed method had a good segmentation effect on plants with leaves adhering to each other and provided an effective solution for organ segmentation and phenotypic parameter measurement of multi-branched crops.

Key words: three-dimensional point clouds, organ segmentation, leaf segmentation, measurement of phenotypic parameters

中图分类号:

S565.1

肖奕同, 刘帅, 侯晨连, 刘琦, 李富忠, 张吴平. 基于三维点云的大豆植株器官分割及表型分析[J]. 中国农业科技导报, 2023, 25(8): 115-125.

Yitong XIAO, Shuai LIU, Chenlian HOU, Qi LIU, Fuzhong LI, Wuping ZHANG. Organ Segmentation and Phenotypic Analysis of Soybean Plants Based on Three-dimensional Point Clouds[J]. Journal of Agricultural Science and Technology, 2023, 25(8): 115-125.

图/表 15

图1 相机位置参照

Fig. 1 Camera position reference

图2 重建的大豆植株三维点云A：稀疏点云； B：稠密点云

Fig. 2 Reconstructed 3D point clouds of soybean plantA： Sparse point clouds； B：Dense point clouds

图3 滤波处理后的大豆植株点云A：前视图； B：俯视图

Fig. 3 Point clouds of soybean plant after filteringA：Front view； B：Top view

图4 大豆植株茎叶分割A：原始点云；B：初步冠层点云；C：初步茎秆点云；D：过滤茎秆点云；E：冠层点云回填；F：完整的冠层点云

Fig. 4 Stem and leaf segmentation of soybean plantA： Initial point clouds； B： Preliminary canopy point clouds；C： Preliminary stem point clouds； D： Filter the stem point clouds； E： Backfill canopy point clouds； F：Full canopy point clouds

图5 大豆植株冠层叶片分割A：叶片粘连下的区域生长分割；B：叶片去粘连后区域生长分割；C：改进的区域生长分割

Fig. 5 Leaves segmentation of soybean canopyA：Region growing segmentation of adherent leaves；B：Region growing segmentation of leaves with adhesions removed；C：Improved region growing segmentation

图6 大豆表型参数测量A：植株坐标系；B：叶片有向包围盒；C：提取叶宽及叶倾角；D：提取茎粗；E：叶片网格；F：叶脉拟合

Fig. 6 Measurement of soybean phenotypic parametersA： Plant coordinate system; B: Leaf oriented bounding box; C: Extracted leaf width and leaf inclination angle; D: Extracted stem diameter; E: Leaf grids; F: Fitting of leaf midrib

图7 不同参数下茎叶分割结果

Fig. 7 Results of stem and leaf segmentation under different parameters

表1 植株茎叶分割结果统计

Table 1 Statistics of stem and leaf segmentation results of plants

植株编号 No.	分割前点云总数N	冠层叶片点云数量N_c		冠层叶片点云分割率R_c/%
植株编号 No.	分割前点云总数N	DoN	RANSAC	DoN	RANSAC
1	230 775	195 719	172 714	84.81	74.84
2	146 930	123 517	115 810	84.07	78.82
3	172 506	144 472	129 035	83.75	74.80
4	198 863	165 371	162 251	83.16	81.59
5	221 826	189 764	166 273	85.55	74.96
6	215 876	185 681	170 564	86.01	79.01
7	173 328	143 937	137 762	83.04	79.48
8	166 324	140 344	134 996	84.38	81.16
9	201 988	168 796	162 237	83.57	80.32
10	185 933	156 402	145 894	84.12	78.47
平均 Average	191 435	161 400	149 754	84.24	78.34

图8 不同参数下叶片分割结果A：较大参数下区域生长分割；B：较小参数下区域生长分割；C：较小参数下改进的区域生长分割

Fig. 8 Results of leaf segmentation under different parametersA：Region growth segmentation；B：Region growth segmentation with small parameters；C：Improved region growth segmentation with small parameters

表2 植株叶片分割结果统计

Table 2 Statistics of leaf segmentation results of plants

植株编号 No.	分割前点云总数N_c	叶片聚类点云总数 N_l	单叶点云分割率R_l/%
1	189 764	183 691	96.80
2	165 371	160 138	96.84
3	144 472	138 853	96.11
4	195 719	186 988	95.54
5	123 517	120 173	97.29
6	185 681	181 085	97.52
7	143 937	138 100	95.94
8	140 344	135 017	96.20
9	168 796	160 844	95.29
10	156 402	150 702	96.36
平均 Average	161 400	155 559	96.39

图9 大豆叶面积不同方法测量值与真实值比较

Fig. 9 Comparison of soybean leaf area by different method

表3 不同方法测量叶面积结果评价

Table 3 Result evaluation of leaf area by different methods

测量方法

Measurement methods

平均相对误差

MRE/%

决定系数

R²

均方根误差

RMSE/cm²

表4 叶宽、叶长及茎粗测量结果评价

Table 4 Result evaluation of leaf width， leaf length and stem diameter

表型参数 Phenotypic parameter	平均相对误差 MRE/%	决定系数 R²	均方根误差 RMSE
叶宽 Leaf width	2.68	0.961 3	0.141 2 cm
叶长 Leaf length	2.87	0.962 6	0.175 5 cm
茎粗 Stem diameter	3.99	0.963 4	0.047 5 cm
叶倾角 Leaf inclination angle	7.22	0.931 1	3.279 6 °

图10 大豆表型参数测量值与真实值比较A：叶宽；B：叶长；C：茎粗；D：叶倾角

Fig. 10 Comparison of soybean phenotypic parameters extracted with actual valuesA：Leaf width； B：Leaf length； C：Stem diameter； D： Leaf inclination angle

表5 时效分析结果

Table 5 Results of processing time analysis

处理阶段 Processing stage	点云重建 Point clouds reconstruction/min	滤波 Filtering/s	器官分割 Organ segmentation/s		表型参数提取 Phenotypic parameter extraction/s
处理阶段 Processing stage	点云重建 Point clouds reconstruction/min	滤波 Filtering/s	茎叶分割 Stem and leaf segmentation	单叶分割 Single leaf segmentation	表型参数提取 Phenotypic parameter extraction/s
最小处理时间 Minimum processing time	19.73	7.11	13.79	6.53	34.65
最大处理时间 Maximum processing time	26.58	8.91	18.44	10.90	53.17
平均处理时间 Average processing time	24.17	7.35	14.63	8.42	46.72

参考文献 22

1	翁杨,曾睿,吴陈铭,等.基于深度学习的农业植物表型研究综述[J].中国科学:生命科学, 2019, 49(6):698-716.
	WENG Y, ZENG R, WU C M,et al.. A survery on deep-learning-based plant phenotype research in agriculture [J]. Sci. Sin. Vitae, 2019, 49(6):698-716.
2	潘映红.论植物表型组和植物表型组学的概念与范畴[J].作物学报, 2015, 41(2):175-186.
	PAN Y H. Analysis of concepts and categories of plant phenome and phenomics [J]. Acta Agron. Sin., 2015, 41(2):175-186.
3	YANG W N, FENG H, ZHANG X H,et al..Crop phenomics and high-throughput phenotyping: past decades, current challenges, and future perspectives [J]. Mol. Plant, 2020, 13(2):187-214.
4	MEYER G E, NETO J C, JONES D D, et al.. Intensified fuzzy clusters for classifying plant, soil, and residue regions of interest from color images [J]. Comp. Electron. Agric., 2004, 42(3):161-180.
5	李晓斌,王玉顺,付丽红. 用K-means图像法和主成分分析法监测生菜生长势[J].农业工程学报, 2016, 32(12):179-186.
	LI X B, WANG Y S, FU L H. Monitoring lettuce growth using K-means color image segmentation and principal component analysis method [J]. Trans. Chin. Soc. Agric. Eng., 2016, 32(12):179-186.
6	周静静,郭新宇,吴升,等. 基于多视角图像的植物三维重建研究进展[J].中国农业科技导报, 2019, 21(2):9-18.
	ZHOU J J, GUO X Y, WU S, et al.. Research progress on plant three-dimensional reconstruction based on multi-view images [J]. J. Agric. Sci. Technol., 2019, 21(2):9-18.
7	LI L, ZHANG Q, HANG D F. A review of imaging techniques for plant phenotyping [J]. Sensors, 2014, 14:20078-20111.
8	史蒲娟,翟瑞芳,常婷婷,等.基于单目视觉和激光扫描技术的油菜植株模型重建及株型参数测量[J].华中农业大学学报, 2017, 36(3):63-68.
	SHI P J, ZHAI R F, CHANG T T, et al.. 3D model generation and phenotypic measurement of rapeseed plant based on monocular visio and laser scanning technology [J]. J. Huazhong Agric. Univ., 2017, 36(3):63-68.
9	李杨先,张慧春,杨旸.一种基于图像处理技术的植物形态表型参数获取方法[J].林业工程学报, 2020, 5(6):128-136.
	LI Y X, ZHANG H C, YANG Y. A method for obtaining plant morphological phenotypic parameters using image processing technology [J]. J. Forestry Eng., 2020, 5(6):128-136.
10	高宇,高军萍,李寒,等.植物表型监测技术研究进展及发展对策[J].江苏农业科学, 2017, 45(11):5-10.
11	WU S, WEN W L, XIAO B X, et al.. An accurate skeleton extraction approach from 3D point clouds of maize plants [J]. Front. Plant Sci., 2019, 10:248-262.
12	BOOGAARD F P, HENTEN E V, KOOTSTRA G. Boosting plant-part segmentation of cucumber plants by enriching incomplete 3D point clouds with spectral data [J]. Biosyst. Eng., 2021, 211:167-182.
13	彭程,李帅,苗艳龙,等.基于三维点云的番茄植株茎叶分割与表型特征提取[J].农业工程学报, 2022, 38(9):187-194.
	PENG C, LI S, MIAO Y L, et al.. Stem-leaf segmentation and phenotypic trait extraction of tomatoes using three-dimensional poind cloud [J]. Trans. Chin. Soc. Agric. Eng., 2022, 38(9):187-194.
14	朱超,苗腾,许童羽,等.基于骨架和最优传输距离的玉米点云茎叶分割和表型提取[J].农业工程学报, 2021, 37(4):188-198.
	ZHU C, MIAO T, XU T Y, et al.. Segmentation and phenotypic trait extraction of maize point cloud stem-leaf based on skeleton and optimal transportation distances [J]. Trans. Chin. Soc. Agric. Eng., 2021, 37(4):188-198.
15	阳旭,胡松涛,王应华,等.利用多时序激光点云数据提取棉花表型参数方法[J].智慧农业, 2021, 3(1):51-62.
	YANG X, HU S T, WANG Y H, et al..Cotton phenotypic trait extraction using multi-temporal laser point clouds [J]. Smart Agric., 2021, 3(1):51-62.
16	LI D W, CAO Y, SHI G L, et al.. An overlapping-free leaf segmentation method for plant point clouds [J]. IEEE Access, 2019,7( 9): 129054-129070.
17	赖亦斌,陆声链,钱婷婷,等.植物三维点云分割[J].应用科学学报, 2021, 39(4):660-671.
	LAI Y B, LU S L, QIAN T T, et al.. Three-dimensional point cloud segmentation for plants [J]. J. Appl. Sci., 2021, 39(4):660-671.
18	RUSU R B, COUSINS S. 3D is here: point cloud library (PCL)[C]// 2011 IEEE International Conference on Robotics & Automation. USA: NewYork, IEEE, 2011:1-4.
19	IOANNOU Y, TAATI B, HARRAP R, et al.. Difference of normals as a multi-scale operator in unorganized point clouds [C]// 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission. Zurich. Switzerland:IEEE, 2012:501-508.
20	孟祥丽,刘一鸣,刘瑶,等.基于摄影测量技术的植物叶面积精确测量方法研究[J].西北林学院学报, 2019, 34(2):222-226.
	MENG X L, LIU Y M, LIU Y, et al.. Precise measurement of plant leaf area based on photogrammetry technology [J]. J. Northwest For. Univ., 2019, 34(2):222-226.
21	彭宝江,钟若飞,孙海丽,等.面向 DEM 构建的点云四叉树和R树混合索引研究[J].应用科学学报, 2018, 36(4):644-654.
	PENG B J, ZHONG R F, SUN H L, et al.. Hybrid index method based on quad tree and R-tree for DEM reconstruction of airborne point cloud [J]. J. Appl. Sci., 2018, 36(4):644-654.
22	孙智慧,陆声链,郭新宇,等.基于点云数据的植物叶片曲面重构方法[J].农业工程学报, 2012, 28(3):184-190.
	SUN Z H, LU S L, GUO X Y, et al.. Surfaces reconstruction of plant leaves based on point cloud data [J]. Trans. Chin. Soc.Agric. Eng., 2012, 28(3):184-190.