CN-122024275-A - Fish and shrimp body length measurement method and system based on deep measurement learning and large model

CN122024275ACN 122024275 ACN122024275 ACN 122024275ACN-122024275-A

Abstract

The invention relates to the technical field of image processing and discloses a method and a system for measuring the length of fish and shrimp bodies based on depth measurement learning and a large model, wherein the method comprises the steps of determining a regional image of each fish and shrimp object in one or more fish and shrimp objects in an aquaculture environment, wherein the regional image is determined based on a visual large model; and inputting the regional image of each fish and shrimp object into a pre-trained body length regression model for measurement to obtain the absolute body length value of the fish and shrimp object, wherein the body length regression model is trained based on a depth measurement learning frame, and learns the mapping relation between the fish and shrimp visual characteristics and the absolute body length through a measurement learning loss function constraint model. Therefore, the method can improve the measurement accuracy of the fish and shrimp body length, and provides a good data basis for measuring the growth condition, adjusting the feeding strategy, determining the grouping and harvesting time and the like.

Inventors

CAI JIA
LI WENZE
JIN XIAO
HUANG JUNJIE
LI ZHONGXIN
YU DAPENG

Assignees

广东海洋大学深圳研究院
深圳市天衍智擎科技有限公司

Dates

Publication Date: 20260512
Application Date: 20251208

Claims (10)

1. A method for measuring the body length of fish and shrimp based on deep metric learning and a large model, which is characterized by comprising the following steps: determining a regional image of each of one or more fish and shrimp objects in an aquaculture environment, wherein the regional image is determined based on a visual large model; inputting the regional image of each fish and shrimp object into a pre-trained body length regression model for measurement to obtain the absolute body length value of the fish and shrimp object; The body length regression model is trained based on a depth measurement learning framework, and learns the mapping relation between the visual characteristics of the fishes and the shrimps and the absolute body length through a measurement learning loss function constraint model.
2. The method of claim 1, wherein determining an image of a region of each of one or more fish-shrimp objects in an aquaculture environment comprises: acquiring an initial image containing one or more fish and shrimp objects in an aquaculture environment, and preprocessing the initial image to obtain an enhanced image; Inputting the enhanced image into a pre-trained instance segmentation model for segmentation to obtain an instance segmentation mask of each fish and shrimp object, wherein the instance segmentation model is obtained by fine adjustment on a pre-acquired fish and shrimp image data set based on the visual large model; based on the instance segmentation mask, a region image of each of the fish and shrimp objects is cropped from the enhanced image.
3. The method for measuring the body length of fish and shrimp based on depth metric learning and large model according to claim 2, wherein the body length regression model is trained by the following ways: Constructing a training data set and an encoder network, wherein the training data set comprises a plurality of fish and shrimp images marked with absolute body length true values, and the encoder network is used for outputting characteristic embedded vectors according to the fish and shrimp images; mapping the characteristic embedded vector into a body length predicted value through a regression header network connected with the encoder network; Defining a composite loss function, the composite loss function comprising a metric learning loss term and a regression loss term; and performing joint training on the encoder network and the regression head network according to the training data set and the composite loss function to obtain the body length regression model.
4. The method for measuring the fish and shrimp body length based on the depth metric learning and the large model according to claim 3, wherein the metric learning loss term is a triplet loss or a contrast loss, and the regression loss term is a smooth L1 loss.
5. The method for measuring the body length of fish and shrimp based on depth metric learning and large model of claim 4 wherein the training dataset further comprises a category of each of the fish and shrimp images and/or a size of each of the fish and shrimp images; And performing joint training on the encoder network and the regression head network according to the training data set and the composite loss function to obtain the body length regression model, wherein the method comprises the following steps: Grouping the training data set according to the categories of all the fish and shrimp images and the sizes of all the fish and shrimp images contained in the training data set to obtain one or more fish and shrimp image groups, wherein all the fish and shrimp images in each fish and shrimp image group correspond to the same fish and shrimp sample; One of the fish and shrimp images of the fish and shrimp image group corresponding to the anchor point is determined to be a positive sample, and one of the rest fish and shrimp samples except the anchor point in all the fish and shrimp samples is determined to be a negative sample; And according to the training data set and the composite loss function, and taking the fact that the first distance between the anchor point and the positive sample in a preset feature space is smaller than the second distance between the anchor point and the negative sample in the feature space and the distance difference between the first distance and the second distance is larger than the preset distance difference as a training target, carrying out joint training on the encoder network and the regression head network to obtain the body length regression model.
6. The method for measuring the body length of a fish and a shrimp based on a depth metric learning and a large model according to any one of claims 1-5, wherein, for each of the fish and shrimp objects, inputting the regional image of the fish and shrimp object into a pre-trained body length regression model for measurement, obtaining the absolute body length value of the fish and shrimp object, comprising: inputting the regional image of each fish and shrimp object into the metric learning loss function constraint model for extraction to obtain the visual characteristics of the fish and shrimp object; Fusing the measured depth information of the fish and shrimp objects and the visual characteristics of the fish and shrimp objects to obtain an information fusion result of the fish and shrimp objects; And inputting the information fusion result of the fish and shrimp object into a regression head network contained in the body length regression model to predict the body length, so as to obtain the absolute body length value of the fish and shrimp object.
7. The method for measuring the body length of a fish and a shrimp based on depth metric learning and large model according to claim 6, wherein after the regional image of the fish and shrimp object is input into a pre-trained body length regression model for measurement, the method further comprises: acquiring a video sequence for determining the region image; Utilizing a preset multi-target tracking algorithm to carry out cross-frame correlation on all the fish and shrimp objects in the video sequence so as to determine a plurality of body length measurement results of each fish and shrimp object, wherein each body length measurement result is a result of inputting each frame area image of the fish and shrimp object in the video sequence into the body length regression model for measurement so as to obtain an absolute body length value; And fusing all body length measurement results of each fish and shrimp object to obtain a body length fusion result, and updating the absolute body length value of the fish and shrimp object according to the body length fusion result.
8. A fish and shrimp body length measurement system based on depth metric learning and a large model, the system comprising: The determining module is used for determining a regional image of each of one or more fish and shrimp objects in the aquaculture environment, wherein the regional image is determined based on a visual large model; The measurement module is used for inputting the regional image of each fish and shrimp object into a pre-trained body length regression model for measurement to obtain the absolute body length value of the fish and shrimp object; The body length regression model is trained based on a depth measurement learning framework, and learns the mapping relation between the visual characteristics of the fishes and the shrimps and the absolute body length through a measurement learning loss function constraint model.
9. A fish and shrimp body length measurement system based on depth metric learning and a large model, the system comprising: a memory storing executable program code; a processor coupled to the memory; The processor invokes the executable program code stored in the memory to perform the depth metric learning and large model based fish-shrimp body length measurement method of any one of claims 1-7.
10. A computer storage medium storing computer instructions which, when invoked, are operable to perform depth metric learning and large model based fish and shrimp body length measurements as claimed in any one of claims 1 to 7.

Description

Fish and shrimp body length measurement method and system based on deep measurement learning and large model Technical Field The invention relates to the technical field of image processing, in particular to a method and a system for measuring the length of fish and shrimp bodies based on depth measurement learning and a large model. Background The global aquaculture industry is facing a profound revolution from "empirically driven" to "data driven". The body length and the body weight of the fish and the shrimp are core key performance indexes for measuring the growth condition, adjusting the feeding strategy and determining the grouping and harvesting time. The main current measurement mode of the body length of the fish and the shrimp is usually based on the recognition of a depth detection model, the marked fish and shrimp bounding box data is utilized to train the model, the automatic detection and the positioning of the target are realized, and the body length is calculated by combining a reference object. However, the conventional measurement mode of the body length of the fish and the shrimp is essentially 'detection' instead of 'direct measurement', which detects the target first and then converts the target through another module (reference object), and factors such as errors of detection of the reference object, perspective deformation of a camera, the fact that the target and the reference object are not on the same plane and the like easily cause errors of the final body length measurement result, especially larger measurement errors of the body length of the fish and the shrimp in a complex underwater environment (such as changeable light, turbid water body and fish shoal shielding). It is seen that how to improve the measurement accuracy of the fish and shrimp body length is important. Disclosure of Invention The invention provides a method and a system for measuring the length of a fish and shrimp body based on depth measurement learning and a large model, which can improve the measurement accuracy of the length of the fish and shrimp body. In order to solve the technical problems, the first aspect of the invention discloses a method for measuring the length of fish and shrimp bodies based on depth measurement learning and a large model, which comprises the following steps: determining a regional image of each of one or more fish and shrimp objects in an aquaculture environment, wherein the regional image is determined based on a visual large model; inputting the regional image of each fish and shrimp object into a pre-trained body length regression model for measurement to obtain the absolute body length value of the fish and shrimp object; The body length regression model is trained based on a depth measurement learning framework, and learns the mapping relation between the visual characteristics of the fishes and the shrimps and the absolute body length through a measurement learning loss function constraint model. As an optional implementation manner, in the first aspect of the present invention, the determining an area image of each of one or more fish and shrimp objects in an aquaculture environment includes: acquiring an initial image containing one or more fish and shrimp objects in an aquaculture environment, and preprocessing the initial image to obtain an enhanced image; Inputting the enhanced image into a pre-trained instance segmentation model for segmentation to obtain an instance segmentation mask of each fish and shrimp object, wherein the instance segmentation model is obtained by fine adjustment on a pre-acquired fish and shrimp image data set based on the visual large model; based on the instance segmentation mask, a region image of each of the fish and shrimp objects is cropped from the enhanced image. As an alternative embodiment, in the first aspect of the present invention, the length regression model is trained by: Constructing a training data set and an encoder network, wherein the training data set comprises a plurality of fish and shrimp images marked with absolute body length true values, and the encoder network is used for outputting characteristic embedded vectors according to the fish and shrimp images; mapping the characteristic embedded vector into a body length predicted value through a regression header network connected with the encoder network; Defining a composite loss function, the composite loss function comprising a metric learning loss term and a regression loss term; and performing joint training on the encoder network and the regression head network according to the training data set and the composite loss function to obtain the body length regression model. In a first aspect of the present invention, as an alternative implementation manner, the metric learning loss term is a triplet loss or a contrast loss, and the regression loss term is a smooth L1 loss. As an optional implementation manner, in the first aspect of the present invention, the trai