Nondestructive Segmentation and Extraction of Stem and Leaf Phenotypes During Tomato Plant Growth

doi:10.13304/j.nykjdb.2024.0080

Abstract

Abstract:

Aiming at the current problem that it is difficult to extract the phenotypic parameters of tomato plants without any loss， the stem and leaf segmentation and phenotypic extraction method of tomato plants based on 3D reconstruction was proposed. Firstly， the multi-view image sequences of tomato plants were collected to construct a 3D model of the plant， and a combination of multiple filtering algorithms was used to complete the pre-processing. The skeleton of the pre-processed point cloud was extracted using the Laplace-based skeleton extraction algorithm， and the segmentation of the stem and leaves was completed based on the skeleton of the plant， and the segmentation of the single leaves was completed using the method based on the clustering of hyperbolomers. 6 phenotypic parameters including the height of the plant， the thickness of the stem， the angle of the inclination of the leaf， the length of the leaf， the width of the leaf， and the area of the leaf were also extracted. The results showed that the average accuracy， average recall and average F1 score of stem and leaf segmentation were 0.88， 0.80 and 0.84， respectively， and the segmentation indexes were better than the other 4 segmentation algorithms. The coefficients of determination between the calculated and measured values of plant height， stem thickness， leaf inclination， leaf length， leaf width and leaf area were 0.97， 0.84， 0.88， 0.94， 0.92 and 0.93， respectively， and the root mean square errors were 2.17 cm， 0.346 cm， 5.65°， 3.18 cm， 2.99 cm and 8.79 cm². The measured values of the proposed method had a strong correlation with the calculated values， which provided technical support for high-throughput phenotypic parameter extraction in tomato plants.

Key words: tomato, three-dimensional reconstruction, stem and leaf segmentation, phenotype extraction

摘要：

针对当前番茄植株表型参数难以无损提取的问题，提出基于三维重建的番茄植株茎叶分割与表型提取方法。首先采集番茄植株的多视角图像序列构建植株三维模型，采用多种滤波算法结合的方式完成预处理；对预处理后的点云使用基于拉普拉斯的骨架提取算法提取骨架，以植株骨架为基础完成茎秆与叶片的分割，使用基于超体素聚类的方法分割单叶；提取株高、茎粗、叶倾角、叶长、叶宽和叶面积共6个表型参数。结果表明，茎叶分割的平均准确率、平均召回率和平均F1分数分别为0.88、0.80和0.84，分割指标均优于其他4种分割算法；株高、茎粗、叶倾角、叶长、叶宽和叶面积计算值与实测值之间的决定系数分别为0.97、0.84、0.88、0.94、0.92和0.93，均方根误差分别为2.17 cm、0.346 cm、5.65°、3.18 cm、2.99 cm和8.79 cm²，该方法的计算值与实测值具有较强的相关性，研究结果为番茄植株的高通量表型参数提取提供了技术支持。

关键词: 番茄, 三维重建, 茎叶分割, 表型提取

CLC Number:

S126

Yaxin WANG, Yangcheng LYU, Wenqi WANG, Qi LIU, Jie YANG, Guihong REN, Wuping ZHANG, Fuzhong LI. Nondestructive Segmentation and Extraction of Stem and Leaf Phenotypes During Tomato Plant Growth[J]. Journal of Agricultural Science and Technology, 2025, 27(7): 90-100.

王亚鑫, 吕洋成, 王文琦, 刘琦, 杨杰, 任桂鸿, 张吴平, 李富忠. 番茄植株生长过程中茎叶表型的无损分割与提取[J]. 中国农业科技导报, 2025, 27(7): 90-100.

Figures/Tables 14

Fig. 1 Image acquisition method

Fig. 2 Reconstruction and preprocessing of tomato plant point cloudA： Dense point cloud； B：Point cloud denoising； C：Point cloud reduction； D：Dynamic growth and reconstruction of tomato plants

Fig. 3 Tomato skeleton extractionA： Skeleton points after contraction； B：Undirected graph point cloud skeleton

Fig. 4 Schematic diagram of stem extractionA： Coordinate system conversion； B：Extract the highest point path of the skeleton； C：Height constraint； D：Radius constraint； E：Separated stem and leaves

Fig. 5 Single leaf segmentationA： Leaf point cloud； B：Leaf supervoxels； C：Leaf segmentation situation

Fig. 6 Phenotype parameter measurementA： Plant height； B：Stem thickness； C：Leaf inclination； D：Leaf point cloud； E：Leaf length and leaf width； F：Leaf area

Table 1 Statistics on the effect of different parameters on blade segmentation

参数 Parameter			超体素块数量 Number of superstitial blocks	运行时间 Running time/s
种子分辨率 $R s e e d$	体素分辨率 $R v o x e l$	最小分割尺寸 $S m i n$	超体素块数量 Number of superstitial blocks	运行时间 Running time/s
0.05	0.01	0.10	378	21.23
0.05	0.05	0.20	386	20.79
0.10	0.01	0.10	291	18.44
0.10	0.05	0.20	274	18.03
0.15	0.05	0.20	191	15.97
0.25	0.10	0.30	123	14.22
0.25	0.15	0.40	108	13.76
0.35	0.10	0.35	56	10.17
0.45	0.10	0.30	19	7.66
0.45	0.15	0.40	10	7.31

Table 1 Statistics on the effect of different parameters on blade segmentation

参数 Parameter			超体素块数量 Number of superstitial blocks	运行时间 Running time/s
种子分辨率 $R s e e d$	体素分辨率 $R v o x e l$	最小分割尺寸 $S m i n$	超体素块数量 Number of superstitial blocks	运行时间 Running time/s
0.05	0.01	0.10	378	21.23
0.05	0.05	0.20	386	20.79
0.10	0.01	0.10	291	18.44
0.10	0.05	0.20	274	18.03
0.15	0.05	0.20	191	15.97
0.25	0.10	0.30	123	14.22
0.25	0.15	0.40	108	13.76
0.35	0.10	0.35	56	10.17
0.45	0.10	0.30	19	7.66
0.45	0.15	0.40	10	7.31

Fig. 7 Leaf segmentation results with different algorithm parametersA：Parameter setting too small； B：Parameter setting too large； C：Moderate parameter setting

Fig. 8 Tomato stem and leaf segmentation results at different growth daysA：True value of manual segmentation； B：Algprithm segmentation results. The black circle represents the part of the algorithm that is segmented incorrectly

Table 2 Precision of stem and leaf segmentation in tomato on different growth days

定植天数 Growth days/d	准确率 Precision	召回率 Recall	F₁分数 F₁-score
5	0.88	0.80	0.84
14	0.91	0.84	0.87
21	0.92	0.85	0.88
30	0.88	0.82	0.85
45	0.86	0.77	0.81
60	0.84	0.74	0.79
平均 Average	0.88	0.80	0.84

Fig. 9 Segmentation effects of different algorithmsA： This research method； B：A method based on skeleton extraction； C：Normal differential method； D：Regional growth segmentation method；E：Segmentation method based on concavity and convexity

Table 3 Comparison of stem and leaf segmentation accuracy among five methods

方法 Method	平均准确率 Average of precision	平均召回率 Average of recall	平均F₁分数 Average of F₁-score
本研究方法 This research method	0.88	0.80	0.84
基于骨架提取的分割方法 Segmentation method based on skeleton extraction	0.77	0.63	0.76
法线微分差异法 Normal differential method	0.64	0.59	0.57
区域生长分割法 Regional growth segmentation method	0.58	0.53	0.59
基于凹凸性的分割方法 Segmentation method based on concavity and convexity	0.54	0.57	0.62

Fig. 10 Phenotypic parameter measurement resultsA： Plant height； B：Stem thickness； C：Leaf inclination； D：Leaf length； E：Leaf width； F：Leaf area

Table 4 Algorithm efficiency statistics

指标Index

点云重建

Point cloud reconstruction

茎叶分割

Stem and leaf segmentation

表型提取

Phenotypic extraction

总计

Total

最小处理时间Minimum processing time/min

最大处理时间Maximum processing time/min

平均处理时间Average processing time/min

References 28

[1]	CHITWOOD-BROWN J, VALLAD G E, LEE T G,et al..Breeding for resistance to Fusarium wilt of tomato:a review [J/OL].Genes, 2021,12(11): 1673 [2023-12-30]. .
[2]	ROOHANITAZIANI R, DE MAAGD R A, LAMMERS M, et al.. Exploration of a resequenced tomato core collection for phenotypic and genotypic variation in plant growth and fruit quality traits [J/OL]. Genes,2020,11(11):1278 [2023-12-30]. .
[3]	SHAKOOR N, LEE S, MOCKLER T C.High throughput phenotyping to accelerate crop breeding and monitoring of diseases in the field [J]. Curr. Opin. Plant Biol., 2017, 38: 184-192.
[4]	张慧春, 张萌, 边黎明, 等. 基于YOLO v5的植物叶绿素含量估测与可视化技术[J].农业机械学报, 2022,53(4): 313-321.
	ZHANG H C, ZHANG M, BIAN L M,et al.. Estimation and visualization of chlorophyll content in plant based on YOLO v5 [J].Trans. Chin. Soc. Agric. Mach., 2022, 53(4): 313-321.
[5]	LU C P, LIAW J J, WU T C, et al.. Development of a mushroom growth measurement system applying deep learning for image recognition [J/OL]. Agronomy, 2019, 9(1):32 [2023-12-30]. .
[6]	WANG D D, LI C Y, SONG H B, et al.. Deep learning approach for apple edge detection to remotely monitor apple growth in orchards [J]. IEEE Access, 2020, 8: 26911-26925.
[7]	陆健强, 兰玉彬, 毋志云, 等. 植物三维建模ICP点云配准优化[J]. 农业工程学报, 2022,38(2): 183-191.
	LU J Q, LAN Y B, WU Z Y,et al.. Optimization of ICP point cloud registration in plants 3D modeling [J]. Trans. Chin. Soc.Agric. Eng., 2022, 38(2): 183-191.
[8]	杨征鹤, 喻晨, 杨会民, 等. 基于LiDAR的温室番茄冠层几何参数提取[J]. 新疆农业科学, 2021, 58(10): 1909-1917.
	YANG Z H, YU C, YANG H M,et al.. Geometric parameters extraction of tomato canopy in greenhouse based on LiDAR [J].Xinjiang Agric. Sci., 2021, 58(10): 1909-1917.
[9]	陈海波, 刘圣搏, 王乐乐, 等. 基于Kinect V3的单株作物自动化三维重建与验证[J]. 农业工程学报, 2022, 38(16): 215-223.
	CHEN H B, LIU S B, WANG L L, et al.. Automatic 3D reconstruction and verification of an individual crop using Kinect V3 [J]. Trans. Chin. Soc. Agric. Eng., 2022, 38(16): 215-223.
[10]	JIN S C, SU Y J, WU F F, et al.. Stem-leaf segmentation and phenotypic trait extraction of individual maize using terrestrial LiDAR data [J]. IEEE Trans. Geosci. Remote. Sens., 2019,57(3): 1336-1346.
[11]	李玉超. 基于多视角图像的苗期玉米植株三维重建和表型测量研究[D]. 保定: 河北农业大学, 2022.
	LI Y C. Three-dimensional reconstruction and phenotype measurement of maize plants at seedling stage based on multi-view images [D]. Baoding: Hebei Agricultural University, 2022.
[12]	ROSE J C, PAULUS S, KUHLMANN H.Accuracy analysis of a multi-view stereo approach for phenotyping of tomato plants at the organ level [J]. Sensors, 2015, 15(5): 9651-9665.
[13]	贾奥博, 董天浩, 张彦, 等. 基于三维点云和集成学习的大田烟草株型特征识别[J]. 浙江大学学报(农业与生命科学版), 2022, 48(3): 393-402.
	JIA A B, DONG T H, ZHANG Y, et al.. Recognition of field-grown tobacco plant type characteristics based on three-dimensional point cloud and ensemble learning [J]. J. Zhejiang Univ. (Agric. Life Sci.), 2022, 48(3): 393-402.
[14]	肖奕同, 刘帅, 侯晨连, 等. 基于三维点云的大豆植株器官分割及表型分析[J]. 中国农业科技导报, 2023, 25(8): 115-125.
	XIAO Y T, LIU S, HOU C L, et al.. Organ segmentation and phenotypic analysis of soybean plants based on three-dimensional point clouds [J]. J. Agric. Sci. Technol., 2023, 25(8): 115-125.
[15]	阳旭. 基于多时相点云数据的作物表型参数获取及动态量化方法研究[D]. 武汉: 华中农业大学, 2019.
	YANG X. Research on crop phenotypic parameter acquisition and dynamic quantification method based on multi-temporal point cloud data [D]. Wuhan: Huazhong Agricultural University, 2019.
[16]	李少辰, 张爱武, 张希珍, 等. 叶片尺度的玉米幼苗三维表型信息提取方法[J]. 激光与光电子学进展, 2023, 60(2): 61-69.
	LI S C, ZHANG A W, ZHANG X Z, et al.. 3D phenotypic information extraction method of maize seedlings at leaf scale [J]. Laser Optoelectron. Prog., 2023, 60(2): 61-69.
[17]	WANG Y H, HU S T, REN H, et al.. 3DPhenoMVS: a low-cost 3D tomato phenotyping pipeline using 3D reconstruction point cloud based on multiview images [J/OL]. Agronomy, 2022,12(8):1865 [2023-12-30]. .
[18]	彭程, 李帅, 苗艳龙, 等. 基于三维点云的番茄植株茎叶分割与表型特征提取[J]. 农业工程学报, 2022, 38(9): 187-194.
	PENG C, LI S, MIAO Y L, et al.. Stem-leaf segmentation and phenotypic trait extraction of tomatoes using three-dimensional point cloud [J]. Trans. Chin. Soc. Agric. Eng., 2022, 38(9): 187-194.
[19]	李百明, 吴茜, 吴劼, 等. 基于多视角自动成像系统的作物三维点云重建策略优化[J]. 农业工程学报, 2023, 39(9): 161-171.
	LI B M, WU Q, WU J, et al.. Optimization of crop 3D point cloud reconstruction strategy based on the multi-view automatic imaging system [J]. Trans. Chin. Soc. Agric. Eng., 2023, 39(9): 161-171.
[20]	CAO J J, TAGLIASACCHI A, OLSON M, et al.. Point cloud skeletons via Laplacian based contraction[C]//2010 Shape Modeling International Conference. IEEE, 2010: 187-197.
[21]	WANG Y W, CHEN Y F, ZHANG X N, et al.. Research on measurement method of leaf length and width based on point cloud [J/OL]. Agriculture, 2021, 11(1): 63 [2024-12-30]. .
[22]	DIJKSTRA E W. A note on two problems in connexion with graphs [J]. Numer. Math., 1959, 1(1): 269-271.
[23]	任栋宇, 李晓娟, 林涛, 等. 基于Kinect v2传感器的果树枝干三维重建方法[J]. 农业机械学报, 2022, 53(): 197-203.
	REN D Y, LI X J, LIN T, et al.. 3D reconstruction method for fruit tree branches based on Kinect v2 sensor [J]. Trans. Chin. Soc. Agric. Mach., 2022, 53(S2): 197-203.
[24]	WU Q H, LIU J C, GAO C, et al.. Improved RANSAC point cloud spherical target detection and parameter estimation method based on principal curvature constraint [J/OL].Sensors, 2022, 22(15): 5850 [2023-12-30]. .
[25]	LIAO L F, TANG S J, LIAO J H, et al.. A supervoxel-based random forest method for robust and effective airborne LiDAR point cloud classification [J/OL]. Remote. Sens., 2022,14(6):1516 [2024-12-30]. .
[26]	ZHENG L, WANG L, WANG M, et al.. Automatic 3D point cloud reconstruction of Leaf lettuce based on Kinect camera [J]. Trans. Chin. Soc. Agric. Mach., 2021, 52(7): 159-168.
[27]	赖亦斌, 陆声链, 钱婷婷, 等. 植物三维点云分割[J]. 应用科学学报, 2021, 39(4): 660-671.
	LAI Y B, LU S L, QIAN T T, et al.. Three-dimensional point cloud segmentation for plants [J]. J. Appl. Sci., 2021, 39(4):660-671.
[28]	ZHANG Z C, MA X D, GUAN H O, et al.. A method for calculating the leaf inclination of soybean canopy based on 3D point clouds [J]. Int. J. Remote. Sens., 2021, 42(15): 5719-5740.