CN-122023951-A - Multimode fish phenotype plasticity classification method integrating two-dimensional key point detection and three-dimensional structured light

CN122023951ACN 122023951 ACN122023951 ACN 122023951ACN-122023951-A

Abstract

The invention discloses a multimode fish phenotype plasticity classification method integrating two-dimensional key point detection and three-dimensional structured light, which constructs a two-dimensional and three-dimensional point cloud data set of large yellow croaker, designs a characteristic engineering flow, selects a key point detection and geometric characteristic extraction algorithm, lays a foundation for subsequent modeling, compares the performances of various machine learning models through systematic experiments, screens key discrimination features with statistical significance, deduces a simplified formula suitable for on-site quick discrimination, deeply discusses the biological connotation of fish curvature as a phenotype plasticity sensitivity index according to experimental results, analyzes the improvement effect of the multimode characteristic fusion on precision, and explains the application value of results in resource protection and industry optimization. The invention realizes the penetration from microscopic curvature characteristics to macroscopic industrial application, and researches the difference of fish surface plasticity of sea fishing and large yellow croaker cultivation for the first time, and extracts key morphological distinguishing characteristics with high discrimination.

Inventors

ZHANG SHENGMAO
SONG YIFAN
DAI YANG
YANG SHENGLONG
WU ZULI
CUI XUESEN

Assignees

中国水产科学研究院东海水产研究所
海南省种业实验室

Dates

Publication Date: 20260512
Application Date: 20260413

Claims (10)

1. The multimode fish phenotype plasticity classification method integrating two-dimensional key point detection and three-dimensional structured light is characterized by comprising the following steps of: (1) Selecting acquisition equipment to acquire data of the large yellow croaker; (2) Collecting two-dimensional and three-dimensional fish body geometric characteristics by using collecting equipment, and defining and marking key points; (3) Comparing the performances of a plurality of machine learning models through a systematic experiment, and screening out key distinguishing features of statistical significance; (4) Deriving a simplified formula suitable for on-site rapid identification, constructing a wild fish and farmed fish discrimination formula suitable for on-site rapid application based on a logistic regression modeling framework of 5-fold cross validation, wherein the formula is based on 5 morphological characteristics which are easy to measure and statistically significant, ensures high discrimination accuracy and simultaneously gives consideration to calculation convenience, the adopted 5 characteristic variables are all screened for p values < 0.01 through three correlation analysis, cover body height and body length subsection measurement values and standardized ratios thereof, and specifically comprise body_high, body_length_1, body_length_2, body_length_3 and body_length_total, wherein 5 characteristics are obviously and negatively correlated with the farmed state, and the numerical value increase of the 5 characteristics is indicated to point to the farmed fish, on the basis, the training set is normalized through Z-score in each-fold cross validation, and a logistic regression model is fitted, and finally, the 5-fold coefficient average is summarized to obtain the following discrimination function: , Decision boundary setting of the discriminant rule according to the output probability of logistic regression When it is determined that the fish is wild When the fish is judged to be cultivated, the rule is equivalent to the prediction probability Pwild >0.5, the exponential operation is avoided, and the manual operation feasibility is obviously improved; (5) The model performance is verified by 5-fold cross, which shows that the formula has excellent generalization capability and stability.
2. The method for classifying the phenotype plasticity of the multi-modal fish by combining two-dimensional key point detection with three-dimensional structured light according to claim 1, wherein the collecting equipment in the step (1) adopts a Zivid Two M high-precision structured light 3D camera manufactured by the company Zivid Norway to collect 1408 pieces of large yellow croaker data, wherein 416 pieces of large yellow croaker data are cultured, 992 pieces of large yellow croaker data are sea-fished, and the data comprise large yellow croaker point cloud data and color RGB data.
3. The method for classifying the phenotype plasticity of the multi-modal fish by combining two-dimensional key point detection and three-dimensional structured light according to claim 1, wherein the collected points in the step (1) are stored in a cloud PLY format, each file comprises about 2,332,800 vertexes, and each vertex accurately records the X, Y, Z space coordinates of a floating point type and the RGB color components of an unsigned character type.
4. The method for classifying the phenotype plasticity of the multi-modal fish by fusing two-dimensional key point detection and three-dimensional structured light according to claim 1, wherein in the step (2), the centerline of the fish body is divided into a plurality of line segments, the body states of different types of fish are described by using the segmented arc length, a labeling scheme of the key points of the fish body is constructed, and the fish body and pectoral fins are labeled.
5. The method for classifying the multi-modal fish phenotypes by fusing two-dimensional key point detection and three-dimensional structured light according to claim 1, wherein in the step (3), geometric features of the three-dimensional fish body are extracted, a mapping relation between images and point cloud data is established, a structured light three-dimensional scanning system and a synchronous triggered two-dimensional RGB camera are adopted for multi-modal data acquisition of a data set, the structured light camera generates point cloud data comprising space coordinates X, Y, Z and RGB color information by projecting a coded grating and capturing a deformation pattern, the point cloud data is stored in PLY format, and meanwhile, the two-dimensional camera records a high-resolution PNG projection image corresponding to a visual angle.
6. The method for classifying the phenotype plasticity of the multi-modal fish by fusing two-dimensional key point detection and three-dimensional structured light as claimed in claim 1, wherein in the step (3), in order to realize multi-modal data fusion, an accurate geometric mapping relation is established, three-dimensional point cloud is back projected back to a two-dimensional image plane according to an imaging model, and the alignment of the point cloud data and an original image at a pixel level is ensured, and the specific technology is realized by the following steps that the origin of a structured light camera coordinate system coincides with the center of the imaging plane, and the optical axis is along The axis is right in the half-axis direction, so we refer to the principle of a structured light camera according to any point in a given point cloud Which is back projected to the target plane Is achieved by ray parameterization, wherein, For the preset target depth, 50% of the maximum Z value of the point cloud is taken to balance perspective distortion, and a calculation formula is as follows: , Rear projection rear point The coordinates of (2) are: , Subsequently, it will Is linearly normalized to the original image resolution, wide W, high H, , , The representation is rounded down and up, And (3) for pixel coordinates, finally, directly writing the point cloud color values into corresponding pixel positions to generate a reconstructed image.
7. The multi-modal fish phenotype plasticity classification method integrating two-dimensional key point detection and three-dimensional structured light according to claim 1, wherein in the step (3), performance of six main stream machine learning algorithms on classification tasks is evaluated, accuracy, precision, recall, F1 score and AUC values are adopted as core evaluation indexes, training is carried out by adopting 5-fold real values in the training process, testing is carried out by using a testing set after training, and all models are trained and verified by using features calculated by real value labels.
8. The method for classifying the phenotype plasticity of the multi-modal fish by combining two-dimensional key point detection and three-dimensional structured light according to claim 1, wherein in the step (3), the accuracy accuracy, precision accuracy, recall recall, F1-score and AUC of the evaluation index of the traditional machine learning model in the machine learning and classifying task are commonly used evaluation indexes, and the following are mathematical definitions thereof, and are applicable to the evaluation index part in the paper: 1. Accuracy represents the correct proportion of predictions in all predictions: , TP is true example, TN is true negative example, FP is false positive example, FN is false negative example; 2. Precision indicates the proportion of samples predicted to be positive, actually positive: , 3. recall represents the proportion of samples actually of positive class that are correctly predicted to be of positive class: , 4. F1-score is the harmonic mean of Precision and Recall, used to balance both: , 5. AUC is the area under ROC curve, which measures the ability of a model to distinguish between positive and negative samples at different classification thresholds, whose mathematical expression is defined by the integral or ranking statistic: , Wherein: , , in practice, AUC can also be interpreted as the probability that a model scores positive samples higher than negative samples by randomly choosing a positive sample and a negative sample.
9. The method for classifying the phenotype plasticity of the multi-modal fish by combining two-dimensional key point detection and three-dimensional structured light according to claim 1, wherein in the step (3), the integrated model LightGBM based on the tree structure stands out from the comprehensive optimal performance, lightGBM is the preferred model for the task in terms of the comprehensive performance, the classification threshold robustness and the calculation efficiency, and when the service scene is extremely sensitive to the omission cost, the random forest can be used as an alternative scheme for high recall rate.
10. The method for classifying the phenotype plasticity of the multi-modal fish by combining two-dimensional keypoint detection with three-dimensional structured light according to claim 1, wherein in the step (5), the cross-validation operation comprises the steps of: (a) Measuring 9 morphological indexes by using a digital caliper with the precision not lower than 0.1 mm; (b) Substituting the values into the formula to calculate the discrimination function value; (c) And making a classification decision according to the symbol, and suggesting retesting key ratio characteristics or combining auxiliary means for the boundary sample.

Description

Multimode fish phenotype plasticity classification method integrating two-dimensional key point detection and three-dimensional structured light Technical Field The invention relates to application of a computer vision technology in fishery, in particular to a multi-modal fish phenotype plasticity classification method integrating two-dimensional key point detection and three-dimensional structured light. Background Sea fishing large yellow croaker individuals are highly similar to mainstream breeding individuals (engineering ships and deepwater net cages) in appearance, so that market supervision difficulty and wild resource protection face technical bottlenecks, and accurate identification basis needs to be established from the aspect of phenotype plasticity. The fish phenotype plasticity, namely the capability of individuals with the same genotype to generate reversible phenotype characteristics under different environmental conditions, is a key mechanism for adapting to environmental changes and maintaining survival in a short period. Under the framework of ecology and evolutionary biology, this mechanism is considered as a bridge linking environmental stress, individual response and long-term adaptive evolution. The traditional measurement means are difficult to comprehensively capture the fine geometric differences, so that the accurate identification of sea fishing and breeding groups faces technical bottlenecks. Therefore, development of a precise quantitative method based on three-dimensional reconstruction is needed, and the system analyzes the surface geometric feature difference, so that support is provided for breaking through the technical problems of market supervision and resource protection. Computer vision technology has become the dominant means for achieving automated, non-invasive acquisition of phenotypic parameters of fish. The existing passive three-dimensional method is still insufficient in obtaining high-resolution surface details, and key local curvature characteristics required by distinguishing source groups are difficult to capture. Therefore, a more accurate and robust active three-dimensional phenotype acquisition technology is needed to be introduced, so that the influence of different living environments on the body surface geometry of the fish is deeply analyzed, and a new technical path is provided for the precise identification of sea fishing and large yellow croaker cultivation. The three-dimensional visual technology such as structured light effectively makes up the defect of the traditional two-dimensional method in characteristic dimension by acquiring high-precision surface geometric information. Although the two-dimensional image method based on deep learning is widely applied to fish identification and size measurement, the method relies on projection images, and is difficult to capture fine geometric features such as three-dimensional fluctuation, curvature and the like of the surface of a fish body, which are key indexes reflecting physiological states such as fish fullness, body surface texture and the like and plastic phenotypes. Therefore, the adoption of three-dimensional visual technologies such as structured light and the like to acquire high-precision geometric data becomes an advanced and necessary means for accurately quantifying the body surface morphology plasticity of economic fishes such as large yellow croakers and the like and comparing the subtle phenotype differences between sea fishing wild individuals and cultured individuals by a system. As important economic fish in China, the large yellow croaker has increasingly prominent contradiction between the vigorous development of the cultivation industry and the continuous decline of wild resources. The fish phenotype plasticity is taken as a key mechanism of environment adaptation, and the external expression (such as body type, body surface texture and the like) of the fish phenotype plasticity contains rich ecological and culture source information. However, the current field faces the double challenges of difficult market supervision and technical bottlenecks of wild resource protection, namely, on one hand, the marketplaces of the nominal sea fishing large yellow croakers need to be accurately identified to prevent false marks and maintain market fairness, and on the other hand, illegal fishing individuals need to be prevented from being mixed into a culture channel to avoid supervision and ensure resource safety. The traditional two-dimensional visual method is difficult to accurately capture the three-dimensional geometric details of the fish body, and limits the technical breakthrough of the problems. Disclosure of Invention The invention provides a multimode fish phenotype plasticity classification method integrating two-dimensional key point detection and three-dimensional structured light, which realizes accurate identification of a large yellow croaker source based on phenotype charact