Evaluation of Plant Self-rotation Multi-view Reconstruction Technique in 3D Phenotype Acquisition of Wheat Plants

doi:10.13304/j.nykjdb.2022.0100

Abstract

Abstract:

The high-throughput acquisition system of crop phenotypes based on multi-view 3D reconstruction technology with low cost and high acquisition efficiency has attracted more and more attentions. The plant self-rotation capture platform is easy to build and low cost， but the jitter generated during plant rotation has an impact on the point cloud 3D reconstruction and phenotype resolution accuracy. To evaluate the applicability of plant rotational multi-view imaging in 3D phenotype resolution of wheat plants， designed a portable 3D phenotype high-throughput acquisition system for wheat plants based on plant self-rotation， and wheat plants of different varieties at the spike stage were selected as experimental samples for point cloud reconstruction， and the accuracy error of the reconstructed point cloud was evaluated based on Hausdorff distance； the accuracy of the extracted phenotypic indicators was evaluated based on manual measurement data. The experimental results showed that the point clouds reconstructed by plant self-rotation had high consistency with those reconstructed by camera rotation， and the difference of point cloud accuracy was basically controlled below 0.4 cm； the root mean square errors （RMSE） of obtained leaf length， leaf width and plant height were 0.79， 0.13 and 0.53 cm， and the mean absolute percent errors （MAPE） were 3.26%， 7.63% and 0.74%， respectively， indicating that this approach was suitable for phenotypic reconstruction of wheat plants at the spike stage and had high accuracy of point cloud reconstruction and phenotype extraction， which provided a low-cost solution for wheat plant phenotyping.

Key words: multi-view 3D reconstruction, wheat plant, point cloud, phenotype, low cost

摘要：

基于多视角重建技术的作物三维表型高通量获取系统成本低、获取效率高，引起越来越多的关注。植物自旋转式拍摄平台易于搭建，但植物旋转过程中产生的抖动对点云三维重建和表型解析精度有一定影响。为评估旋转式多视角成像在小麦植株三维表型解析中的适用性，基于植物旋转设计了便携式小麦植株三维表型高通量采集系统，选取穗期不同品种的小麦植株作为实验样本进行点云重建，基于Hausdorff距离评价了重建点云的精度误差；并基于人工测量数据，对所提取的表型指标精度进行评价。结果表明，植物旋转式重建的点云与相机旋转式重建的点云有较高的一致性，点云精度差距基本控制在0.4 cm以下；获取的叶长、叶宽和株高的均根方误差分别为0.79、0.13和0.53 cm，平均绝对百分比误差分别为3.26%、7.63%和0.74%，表明该方式适合穗期的小麦植株表型重建，具有较高的点云重建和表型提取精度，并为小麦植株表型评价提供了一种低成本的解决方案。

关键词: 多视角三维重建, 小麦植株, 点云, 表型, 低成本

CLC Number:

S126

Wenqi ZHANG, Sheng WU, Xinyu GUO, Weiliang WEN, Xianju LU, Chunjiang ZHAO. Evaluation of Plant Self-rotation Multi-view Reconstruction Technique in 3D Phenotype Acquisition of Wheat Plants[J]. Journal of Agricultural Science and Technology, 2022, 24(8): 87-98.

张文麒, 吴升, 郭新宇, 温维亮, 卢宪菊, 赵春江. 植株自旋转多视角重建技术在小麦植株三维表型获取中的应用评估[J]. 中国农业科技导报, 2022, 24(8): 87-98.

Figures/Tables 11

Fig. 1 Plant rotating multi-view acquisition deviceNote：1—Camera； 2—Rotation stand； 3—Calibration unit； 4—Backdrop； 5—Computer.

Fig. 2 Pipeline wheat phenotype processing system

Fig. 3 Wheat plant sample image black background removal

Fig. 4 Wheat plant point cloud extraction and denoising

Fig. 5 Reconstructed field point cloud and its calibration circle radius extraction

Fig. 6 Visualization of point clouds reconstructed by plant self-rotation and camera rotationA， B and C are Jimai 38 samples； D， E and F are Xinnong 979 samples. G， H and I are Xinmai 26 samples. The left point cloud is obtained by plant selt-rotation， and the right by camera rotation.

Fig. 7 Point cloud Hausdorff distance visualizationA： Plant point cloud； B： Leaf point cloud； C： Spike point cloud. Each row from left to right is the experimental subject of Jimai 38， Xinnong 979 and Xinmai 26， respectively

Fig. 8 Point cloud distancevdistribution and percentage

Fig. 9 Wheat reconstruction point cloud distance statistical resultsA and B are Xinnong 979 samples； C and D are Xinmai 26 samples； E and F are Jimai 38 samples

Fig. 10 Regression analysis and RSME and MAPE results

Table 1 Comparison of cost and reconstruction efficiency of plant self-rotation shooting platform and camera rotation shooting platform

项目 Item		植物旋转式 Plant self-rotation	相机旋转式 Camera rotation
相机数量 Camera number		2	3
组件成本 Component cost/yuan	转台装置 Turntable unit	443	34 500
	相机 Camera	12 800	19 200
	计算机 Computer	5 000	5 000
图像数量 Image number		60	90
图像大小 Image size/Mb		1.54	6.00
单个样本耗时 Reconstruction time per sample/s		932	2 552
重建效率/（s·幅^-1） Reconstruction efficiency/（s·frame^-1）		15.53	28.36

References 29

1	LI L, ZHANG Q, HUANG D. A review of imaging techniques for plant phenotyping [J]. Sensors (Basel), 2014, 14(11): 20078-20111.
2	PIERUSCHKA R, SCHURR U. Plant phenotyping: past, present, and future [J]. Plant Phenomics, 2019(3): 1-6.
3	彭永彬,谢先芝.表型组学在水稻研究中的应用[J].中国水稻科学,2020,34(4):300-306.
	PENG Y B, XIE X Z. Application of phenomics in rice research [J]. Chin. J. Rice Sci., 2020, 34(4): 300-306.
4	程超华,唐蜻,邓灿辉,等.表型组学及多组学联合分析在植物种质资源精准鉴定中的应用 [J].分子植物育种,2020,18(8):2747-2753.
	CHENG C H, TANG Q, DENG C H, et al.. Application of phenomics and multiomics joint analysis in accurate identification of plant germplasm resources [J]. Mol. Plant Breeding, 2020, 18(8): 2747-2753
5	穆金虎,陈玉泽,冯慧,等.作物育种学领域新的革命:高通量的表型组学时代[J].植物科学学报,2016,34(6):962-971.
	MU J H, CHEN Y Z, FENG H, et al.. A new revolution in crop breeding : the era of high-throughput phenomics [J]. Plant Sci. J., 2016, 34(6): 962-971.
6	卢少志,杨蒙,杨万能,等.田间作物表型检测平台设计与试验[J].华中农业大学学报,2021,40(4):209-218.
	LU S Z, YANG M, TANG W N, et al.. Design and experiment of a platform for detecting phenotype of field crop [J]. J. Huazhong Agric. Univ., 2021, 40(4): 209-218.
7	郭庆华,吴芳芳,庞树鑫,等.Crop 3D—基于激光雷达技术的作物高通量三维表型测量平台[J].中国科学:生命科学,2016,46(10):1210-1221.
	GUO Q H, WU F F, PANG S X, et al.. Crop 3D: a platform based on LiDAR for 3D high-throughput crop phenotyping [J]. Sci. Sin.: Vitae, 2016, 46(10): 1210-1221.
8	SHIFERAW B, SMALE M, BRAUM H J, et al.. Crops that feed the world 10. past successes and future challenges to the role played by wheat in global food security [J]. Food Security, 2013, 5(3): 291-317.
9	刘守阳,金时超,郭庆华,等.基于数字化植物表型平台(D3P)的田间小麦冠层光截获算法开发[J].智慧农业,2020,2(1):87-98.
	LIU S Y, JIN S C, GUO Q H, et al.. An algorithm for estimating field wheat canopy lightinterception based on digital plant phenotyping platform [J]. Smart Agric., 2020, 2(1): 87-98.
10	WATANABE K, GUO W, ARAI K, et al.. High-throughput phenotyping of sorghum plant height using an unmanned aerial vehicle and its application to genomic prediction modeling [J/OL]. Front. Plant Sci., 2017, 8: 421 [2022-06-22]. .
11	王传宇,郭新宇,杜建军,等.基于时间序列图像的玉米植株干旱胁迫表型检测方法[J].农业工程学报,2016,32(21):189-195.
	WANG C Y, GUO X Y, DU J J, et al.. Maize plant drought stress phenotype testing method based on time-series images [J]. Trans. Chin. Soc. Agric. Eng., 2016, 32(21): 189-195.
12	黄林生,邵松,卢宪菊,等.基于卷积神经网络的生菜多光谱图像分割与配准[J].农业机械学报,2021,52(09):186-194.
	HUANG L S, SHAO S, LU X J, et al.. Segmentation and registration of lettuce multispectral image based on convolutional neural network [J]. Trans. Chin. Soc. Agric. Mach., 2021, 52(09): 186-194.
13	CHOUDHURY S, BASHYAM S, QIU Y, et al.. Holistic and component plant phenotyping using temporal image sequence [J]. Plant Methods, 2018, 14(1): 1-21.
14	PAULUS S. Measuring crops in 3D: using geometry for plant phenotyping [J]. Plant Methods, 2019, 15(1): 1-13.
15	劳彩莲,杨瀚,李鹏,等.基于消费级深度相机的玉米植株三维重建[J].农业机械学报,2019,50(7):222-228.
	LAO C L, YANG H, LI P, et al.. 3D Reconstruction of maize plants based on consumer depth camera [J]. Trans. Chin. Soc. Agric. Mach., 2019, 50(7): 222-228.
16	翟晓晓,邵杰,张吴明,等.基于移动LiDAR点云的树木三维重建[J].中国农业信息,2019,31(5):84-89.
	ZHAI X X, SHAO J, ZHANG W M, et al. Three-dimensional reconstruction of trees using mobile laser scanning point clouds [J]. China Agric. Inform., 2019, 31(5): 84-89.
17	周静静,郭新宇,吴升,等.基于多视角图像的植物三维重建研究进展[J].中国农业科技导报,2019,21(2):9-18.
	ZHOU J J, GUO X Y, WU S, et al. Research progress on plant three-dimensional reconstruction based on multi-view images [J]. J. Agric. Sci. Technol., 2019, 21(2): 9-18.
18	WESTOBY M J, BRASINGTON J, GLASSER N F, et al.. ‘Structure-from-Motion’ photogrammetry: a low-cost, effective tool for geoscience applications [J]. Geomorphology, 2012, 179: 300-314.
19	GOESELE M, SNAVELY N, CURLESS B, et al.. Multi-view stereo for community photo collections [C]// Proceedings of the 2007 IEEE 11th International Conference on Computer Vision, October 14-21, 2007. Rio de Janeiro, Brazil: IEEE, 2007: 1-8.
20	LOWE D G. Distinctive image features from scale-invariant keypoints [J]. Int J. Computer Vision, 2004, 60(2): 91-110.
21	SCHONBERGER J L, FRAHM J M. Structure-from-motion revisited [C]// Proceedings of the IEEE conference on computer vision and pattern recognition. Las Vegas, NV, USA: IEEE 2016: 4104-4113.
22	WU S, WEN W, WANG Y, et al.. MVS-Pheno: a portable and low-cost phenotyping platform for maize shoots using multiview stereo 3D reconstruction [J/OL]. Plant Phenomics, 2020, 2020:1848437 [2022-06-22]. .
23	LIU S, ACOSTA-GAMBOA L M, HUANG X, et al.. Novel low cost 3D surface model reconstruction system for plant phenotyping [J/OL]. J. Imaging, 2017, 3(3): 39 [2022-06-22]..
24	ROSSI R, LEOLINI C, COSTAFREDA-AUMEDES S, et al.. Performances evaluation of a low-cost platform for high-resolution plant phenotyping [J/OL]. Sensors, 2020, 20(11): 3150 [2022-06-22]. .
25	WANG Y, WEN W, WU S, et al.. Maize plant phenotyping: comparing 3D laser scanning, multi-view stereo reconstruction, and 3D digitizing estimates [J/OL]. Remote Sens., 2019, 11(1): 63 [2022-06-22]. .
26	MOULON P, MONASSE P, PERROT R, et al.. Openmvg: Open multiple view geometry [C]// Proceedings of the International Workshop on Reproducible Research in Pattern Recognition. Cham: Springer, 2016: 60-74.
27	ANDREW V, GOLDBERG, SAGI H, et al. OpenMVS: multi-view stereo reconstruction library[EB/OL]. [2021-06-17]. .
28	ZHANG Y, TENG P, SHIMIZU Y, et al.. Estimating 3D leaf and stem shape of nursery paprika plants by a novel multi-camera photography system [J/OL]. Sensors, 2016, 16(6): 874 [2022-06-22]..
29	LU X, ONO E, LU S, et al.. Reconstruction method and optimum range of camera-shooting angle for 3D plant modeling using a multi-camera photography system [J]. Plant Methods, 2020, 16(1): 1-16.

[1]	LI Shuxin, ZHANG Hao, ZHENG Housheng, ZHENG Peihe, PANG Shifeng, XU Shiquan. Transcriptome Analysis of Phenotypic Differences Between Ermaya and Changbo of Forest Cultivated Panax ginseng [J]. Journal of Agricultural Science and Technology, 2021, 23(9): 56-68.
[2]	LIU Zhongxiang, YANG Yanzhong, WANG Xiaojuan, LIAN Xiaorong, ZHOU Wenqi, HE Haijun, ZHOU Yuqian, KOU Sirong. Phenotypic Identification and Combining Ability Effect Analysis of Mutants by Fast Neutron Irradiation [J]. Journal of Agricultural Science and Technology, 2021, 23(6): 184-194.
[3]	WU Dan1, YE Junli1, WANG Kang1, GENG Zedong1, FU Jingbo1, LIU Lingbo2, YANG Wanneng1*. Three-dimension Reconstruction Method Based on Silhouette for Pot Rice [J]. Journal of Agricultural Science and Technology, 2020, 22(9): 87-95.
[4]	ZHOU Jingjing1,2,3, GUO Xinyu2,3, WU Sheng2,3, DU Jianjun2,3，ZHAO Chunjiang2,3*. Research Progress on Plant Three-dimensional Reconstruction Based on Multi-view Images [J]. Journal of Agricultural Science and Technology, 2019, 21(2): 9-18.
[5]	WANG Kaiyue, CHEN Fangquan, HUANG Wuxing*. Research Advance on Drought Stress Response Mechanism in Plants [J]. Journal of Agricultural Science and Technology, 2019, 21(2): 19-25.
[6]	YU Zetao1,2, WEN Weiliang1,2, GUO Xinyu1,2*, WANG Yongjian1,2, QIAN Tingting3. Morphological Changes of Maize Plants at Late Grain Filling Stages Based on 3D Point Cloud [J]. Journal of Agricultural Science and Technology, 2018, 20(12): 59-66.
[7]	PAN Xiaodi1,2, ZHANG Ying1,2, SHAO Meng1,2, MA Liming1,2, GUO Xinyu1,2*. Research Progress on Adaptive Responses of Crop Root Structure to Drought Stress [J]. Journal of Agricultural Science and Technology, 2017, 19(2): 51-58.
[8]	YUAN Jie1,2, DU Jian-jun2, GUO Xin-yu2*. Research Progress on Maize Plant Phenotype Detection Based on Visible Image [J]. Journal of Agricultural Science and Technology, 2016, 18(6): 90-96.
[9]	WEN Wei-liang1,2, WANG Yong-jian1,2, XU Tong-yu3*, YANG Tao3, GUO Xin-yu1,2, ZHU Hong-yu4, DONG Cheng-yu4. Geometric Modeling of Maize Ear based on Three-dimensional Point Cloud [J]. Journal of Agricultural Science and Technology, 2016, 18(5): 88-93.
[10]	SHAO Meng1,2, ZHANG Ying1,2, GUO Xin-yu1,2*. Research Progress on Phenotype of Transport Tissue of Ear and Kernel in Maize [J]. Journal of Agricultural Science and Technology, 2016, 18(3): 38-45.
[11]	WANG Yong\\|jian1,2, WEN Wei\\|liang2, GUO Xin\\|yu2, ZHAO Chun\\|jiang2*. Three\\|dimensional Reconstruction of Plant Leaf Blade Based on Point Cloud Data [J]. , 2014, 16(5): 83-89.