Study on Fish Body Length Measurement Method Based on Segmentation Mask and Binocular Vision

doi:10.13304/j.nykjdb.2023.0581

Abstract

Abstract:

To achieve higher precision underwater fish body length measurement， a new method based on segmentation mask and binocular vision was proposed. The underwater image was obtained after underwater calibration by the binocular camera， and then U2-Net was used to segment the mask of the fish target. After stereoscopic correction and the SGBM （semi-global block matching） stereoscopic matching algorithm， the point cloud data on the surface of the adult fish body was generated. Finally， the noise filtering was added in the normalization and point cloud fitting process， which was for reducing the error of body length data. The results showed that the accuracy of this method was improved by about 2% after underwater calibration. After noise filtering， the error of measurement results was about 3%， which decreased more than 2% compared with the existing other methods. Moreover， when the angle between the fish body and the camera was less than 60°， a high measurement accuracy could be achieved， which was better than the existing other methods. Above results provided a feasible idea for non-contact size measurement of fish in aquaculture.

Key words: binocular vision, U²-Net, three-dimensional reconstruction, ichthyometry

摘要：

为了实现更高精度的水下鱼类体长测量，提出基于分割掩膜和双目视觉的鱼类体长测量方法。通过双目相机进行水下标定后获得水下图像；再使用U²-Net分割出鱼类目标的掩膜，经过立体校正以及SGBM（semi-global block matching）立体匹配算法后生成鱼体表面的点云数据；最后通过在归一化以及点云拟合过程中增加噪声过滤，降低体长数据误差。结果表明，该方法经过水下标定后，精度提升2%左右；增加噪声过滤后，测量结果的误差在4％以内，相较于现有方法降低2％以上。并且，鱼体与相机夹角在60°以内时都能达到较高的测量精度，明显优于现有方法。以上研究结果为水产养殖中的鱼类无接触尺寸测量提供了可行的思路。

关键词: 双目视觉, U²-Net, 三维重建, 鱼体测量

CLC Number:

TP391

Mingjie ZHU, Wei LIAO, Zhen XU, Yilin TIAN. Study on Fish Body Length Measurement Method Based on Segmentation Mask and Binocular Vision[J]. Journal of Agricultural Science and Technology, 2025, 27(2): 116-124.

朱铭劼, 廖薇, 徐震, 田伊林. 基于分割掩膜和双目视觉的鱼类体长测量方法研究[J]. 中国农业科技导报, 2025, 27(2): 116-124.

Figures/Tables 10

Fig. 1 Simplified underwater camera imaging model

Fig. 2 Binocular images of fish objects

Fig. 3 Point cloud data before and after normalization

Fig. 4 Overall design frame diagram

Table 1 Underwater calibration parameters of binocular camera

参数 Parameter	左摄像头 Left camera	右摄像头 Right camera
内参矩阵Internal parameter matrix	$464.637 2 0 331.273 9 0 464.198 1 247.363 8 001$	$464.658 8 0 335.391 3 0 464.926 0 253.333 5 001$
径向畸变Radial distortion	$- 0.016 0.087 - 0.058$	$0.007 - 0.009 - 0.009$
切向畸变Tangential distortion	$00$	$00$
R旋转矩阵 Rotation matrix	$0.999 9 - 0.002 3 0.010 2 0.002 3 1 - 0.002 2 - 0.010 2 0.002 2 0.999 9$
T平移向量 Translation vector	$- 59.680 0 - 0.006 9 - 0.218 1$
f₁：真实焦距 True focal length/cm	46.42
f₂：虚拟焦距 Virtual focal length/cm	61.73

Table 1 Underwater calibration parameters of binocular camera

参数 Parameter	左摄像头 Left camera	右摄像头 Right camera
内参矩阵Internal parameter matrix	$464.637 2 0 331.273 9 0 464.198 1 247.363 8 001$	$464.658 8 0 335.391 3 0 464.926 0 253.333 5 001$
径向畸变Radial distortion	$- 0.016 0.087 - 0.058$	$0.007 - 0.009 - 0.009$
切向畸变Tangential distortion	$00$	$00$
R旋转矩阵 Rotation matrix	$0.999 9 - 0.002 3 0.010 2 0.002 3 1 - 0.002 2 - 0.010 2 0.002 2 0.999 9$
T平移向量 Translation vector	$- 59.680 0 - 0.006 9 - 0.218 1$
f₁：真实焦距 True focal length/cm	46.42
f₂：虚拟焦距 Virtual focal length/cm	61.73

Fig. 5 Underwater image and segmentation result

Fig. 6 Comparison before and after noise filtering

Fig. 7 Error box diagram of fish body length measurement results

Table 2 Comparison of the measurement error with other methods

测量方法 Measurement method		鱼类体长范围 Range of fish body length/mm
测量方法 Measurement method		110~140	230~250
方法1^［17］ Method 1		0.055	0.051
方法2^［22］ Method 2		0.053	0.046
本研究方法 Method of this study	空气标定 Air calibration	0.054	0.048
本研究方法 Method of this study	水下标定 Underwater calibration	0.032	0.027

Fig. 8 Measurement error of fish body length at different angles and distances

References 22

1	张志强,牛智有,赵思明,等.基于机器视觉技术的淡水鱼质量分级[J].农业工程学报,2011,27(2):350-354.
	ZHANG Z Q, NIU Z Y, ZHAO S M, et al.. Weight grading of freshwater fish based on computer vision [J]. Trans. Chin. Soc. Agric. Eng., 2011, 27(2): 350-354.
2	RATHI D, JAIN S, INDU S. Underwater fish species classification using convolutional neural network and deep learning [C]// Proceedings of Ninth International Conference on Advances in Pattern Recognition (ICAPR). Bangalore, India, 2017:1-6.
3	ZHANG S, YANG X T, WANG Y Z, et al.. Automatic fish population counting by machine vision and a hybrid deep neural network model [J]. Animals, 2020, 10(2): 364-381.
4	张佳林,徐立鸿,刘世晶.基于水下机器视觉的大西洋鲑摄食行为分类[J].农业工程学报,2020,36(13):158-164.
	ZHANG J L, XU L H, LIU S J. Classification of Atlantic salmon feeding behavior based on underwater machine vision [J]. Trans. Chin. Soc. Agric. Eng., 2020, 36(13): 158-164.
5	WILLIAMS K, LAUFFENBURGER N, CHUANG M C, et al.. Automated measurements of fish within a trawl using stereo images from a camera-trawl device (CamTrawl) [J]. Methods Oceanography, 2016, 17:138-152.
6	HAO M, YU H, LI D. The measurement of fish size by machine vision-a review [C]// Proceedings of Computer and Computing Technologies in Agriculture IX: 9th IFIP WG 5.14 International Conference, CCTA 2015, Beijing, China, September 27-30, 2015, Revised Selected Papers, Part Ⅱ 9. Springer International Publishing, 2016:15-32.
7	SABERIOON M, GHOLIZADEH A, CISAR P, et al.. Application of machine vision systems in aquaculture with emphasis on fish: state-of-the-art and key issues [J]. Rev. Aquac., 2017, 9(4):369-387.
8	段延娥,李道亮,李振波,等.基于计算机视觉的水产动物视觉特征测量研究综述[J].农业工程学报,2015,31(15):1-11.
	DUAN Y E, LI D L, LI Z B, et al.. Review on visual characteristic measurement research of aquatic animals based on computer vision [J]. Trans. Chin. Soc. Agric. Eng., 2015, 31(15): 1-11.
9	HSIEH C L, CHANG H Y, CHEN F H, et al.. A simple and effective digital imaging approach for tuna fish length measurement compatible with fishing operations [J]. Comp. Electr. Agric., 2011, 75(1):44-51.
10	CHUANG M C, HWANG J N, WILLIAMS K, et al.. Tracking live fish from low-contrast and low-frame-rate stereo videos [J]. IEEE Trans. Circuits Syst. Video Technol., 2014, 25(1):167-179.
11	QUSSAY A J, WALEED A H, MAJID A T, et al.. An automated vision system for measurement of zebrafish length using low-cost orthogonal web cameras [J]. Aquac. Eng., 2017, 78:155-162.
12	SHAFAIT F, HARVEY E S, SHORTIS M R, et al.. Towards automating underwater measurement of fish length: a comparison of semi-automatic and manual stereo-video measurements [J]. ICES J. Marine Sci., 2017, 74(6):1690-1701.
13	COSTA C, SCARDI M, VITALINI V, et al.. A dual camera system for counting and sizing Northern Bluefin Tuna (Thunnus thynnus; Linnaeus, 1758) stock, during transfer to aquaculture cages, with a semi automatic artificial neural network tool [J]. Aquaculture, 2009, 291(3-4):161-167.
14	ZHAO Y, SUN Z Y, DU H, et al.. A novel centerline extraction method for overlapping fish body length measurement in aquaculture images [J/OL]. Aquacult. Eng., 2022, 99:102302 [2024-03-06]. .
15	MUÑOZ-BENAVENT P, ANDREU-GARCÍA G, VALIENTE-GONZÁLEZ J M, et al.. Enhanced fish bending model for automatic tuna sizing using computer vision [J]. Comp. Electr. Agric., 2018, 150: 52-61.
16	周佳龙,季柏民,倪伟强,等.基于三维姿态拟合的非接触式红鳍东方鲀全长精准估算方法[J].农业工程学报,2023,39(4): 154-161.
	ZHOU W L, JI B M, NI W Q, et al.. Non-contact method for the accurate estimation of the full-length of Takifugu rubripes based on 3D pose fitting [J]. Trans. Chin. Soc. Agric. Eng., 2023, 39(4): 154-161.
17	李艳君,黄康为,项基.基于立体视觉的动态鱼体尺寸测量[J].农业工程学报,2020,36(21):220-226.
	LI Y J, HUANG K W, XIANG J. Measurement of dynamic fish dimension based on stereoscopic vision [J]. Trans. Chin. Soc. Agric. Eng., 2020, 36(21):220-226.
18	HUANG K W, LI Y J, SUO F, et al.. Stereo vison and mask-RCNN segmentation based 3D points cloud matching for fish dimension measurement [C]// Proceedings of 2020 39th Chinese Control Conference (CCC). IEEE, 2020:6345-6350.
19	QIN X, ZHANG Z, HUANG C, et al.. U² -Net: going deeper with nested U-structure for salient object detection [J/OL]. Pattern Recognition, 2020, 106:107404 [2024-3-06]. .
20	ZHANG Z. A flexible new technique for camera calibration [J]. IEEE Transactions Pattern Analysis Machine Intelligence, 2000, 22(11):1330-1334.
21	ZHANG W M, DENG X X, ZHANG Q, et al.. Non-parallel system underwater image transformation model [J/OL]. Acta Photonica Sin., 2015, 44(2):211002 [2024-03-06]. .
22	董鹏,周烽,赵悰悰,等.基于双目视觉的水下海参尺寸自动测量方法[J].计算机工程与应用,2021,57(8):271-278.
	DONG P, ZHOU F, ZHAO C C, et al.. Automatic measurement of underwater sea cucumber size based on binocular vision [J]. Comp. Eng. App., 2021, 57(8):271-278.

[1]	Kui FANG, Cheng LI, Xiao HE, Yineng CHEN. Research on Multi-angle Identification of Grape Leaf Disease Based on 3D Reconstruction [J]. Journal of Agricultural Science and Technology, 2022, 24(7): 86-96.
[2]	ZHOU Jingjing1,2,3, GUO Xinyu2,3, WU Sheng2,3, DU Jianjun2,3，ZHAO Chunjiang2,3*. Research Progress on Plant Three-dimensional Reconstruction Based on Multi-view Images [J]. Journal of Agricultural Science and Technology, 2019, 21(2): 9-18.
[3]	WEN Wei-liang1,2, WANG Yong-jian1,2, XU Tong-yu3*, YANG Tao3, GUO Xin-yu1,2, ZHU Hong-yu4, DONG Cheng-yu4. Geometric Modeling of Maize Ear based on Three-dimensional Point Cloud [J]. Journal of Agricultural Science and Technology, 2016, 18(5): 88-93.