CN-121768046-B - Dynamic fish fry technical method and system based on computer vision

CN121768046BCN 121768046 BCN121768046 BCN 121768046BCN-121768046-B

Abstract

The invention discloses a fry dynamic technical method and system based on computer vision, which relate to the technical field of image processing and are used for synchronously collecting visible light video data and multispectral image data of fry groups, carrying out fusion analysis on the visible light video data and the multispectral image data, identifying and correlating individual fries on time sequence, and carrying out cooperative coding on kinematic features extracted from the visible light video data and spectral reflection features extracted from the multispectral image data to obtain a coding result. The invention establishes a multi-mode characteristic vector for each body, which fuses motion appearance and intrinsic physiological spectrum information. Even when the movement track of the individual is mutated or lost temporarily, the unique spectral characteristics of the individual can be used as a stable identity auxiliary identifier, and the continuity of the ID can be effectively maintained by combining an appearance matching mechanism in a multi-target tracking algorithm, so that the probability of following loss or misidentifying the individual in a complex and high-dynamic culture environment is obviously reduced.

Inventors

LI JIAJIAN
CHEN XIAOYAN
MA YUBIN
GUO WEICHEN
ZHANG SHENGZHI
LIN ZHENZHEN
He Zhuolun

Assignees

四川农业大学

Dates

Publication Date: 20260512
Application Date: 20260304

Claims (7)

1. A dynamic technical method for fish fries based on computer vision is characterized by comprising the following steps: Synchronously collecting visible light video data and multispectral image data of a fry group; Carrying out fusion analysis on the visible light video data and the multispectral image data, and identifying and correlating individual fries in time sequence; The method comprises the steps of carrying out cooperative coding on kinematic features extracted from visible light video data and spectral reflection features extracted from multispectral image data to obtain a coding result, wherein the cooperative coding specifically comprises the steps of calculating a behavior liveness index according to the kinematic features and dynamically adjusting a weight coefficient of the spectral reflection features during fusion according to the index; When the behavior activity index is lower than a preset threshold, the weight coefficient of the spectral reflection characteristic is improved; constructing a multi-modal feature vector for each associated individual fry based on the encoding result; Constructing a time-ordered group dynamic interaction map based on the associated individual fries and the multi-modal feature vectors; Inputting the group dynamic interaction spectrum into a pre-trained graph neural network anomaly detection model to calculate a group behavior dissimilarity index, wherein the graph neural network anomaly detection model is a space-time graph convolution network which is trained in an unsupervised manner by using historical normal data, and the group behavior dissimilarity index is a reconstruction error of the model on a current spectrum; And judging the abnormal state of the group based on the group behavior dissimilation index, and positioning the abnormal high-risk individual fries.
2. The method of claim 1, wherein in the step of synchronously collecting the video data of the visible light and the multispectral image data of the fry group, the data are collected by synchronously disposing a visible light camera and a multispectral camera, and the working band of the multispectral camera comprises a near infrared band.
3. The method of claim 1, wherein in the step of performing fusion analysis on the visible light video data and the multispectral image data to identify and temporally correlate individual fries, the fusion analysis specifically comprises performing identification and correlation of individuals by a target detection and multi-target tracking algorithm based on the visible light video data, and aligning the multispectral image data with video frames by using time stamps.
4. The method of claim 1, wherein in the step of constructing a multi-modal feature vector for each of the associated individual fries based on the encoding results, the multi-modal feature vector is formed by stitching the kinematic features and the weighted spectral reflectance features.
5. The method of claim 1, wherein in the step of constructing a time-ordered group dynamic interaction map based on the related individual fries and the multi-modal feature vectors, the time-ordered group dynamic interaction map is constructed by constructing an edge based on the fact that each tail related individual fry is taken as a node, the multi-modal feature vector is taken as a node attribute, and the space Euclidean distance between individuals is smaller than a preset neighborhood radius, so as to construct a dynamic map structure; the weight of the edge is calculated based on Euclidean distance between two nodes connected with the edge and cosine similarity between multi-mode feature vectors of the edge.
6. The method for dynamically processing fries based on computer vision according to claim 1, wherein in the step of determining a group abnormal state and locating the fries of high-risk individuals causing abnormality based on the group behavior dissimilation index, the method is characterized in that the attention weight of the graph neural network abnormality detection model is used for calculating the contribution degree of each node to the dissimilation index, and the node corresponding individual with the highest contribution degree is located as the high-risk individual.
7. A fry dynamic technical system based on computer vision for implementing the fry dynamic technical method based on computer vision as claimed in any one of claims 1 to 6, comprising: the data acquisition module is used for synchronously acquiring visible light video data and multispectral image data of the fry group; the individual association module is used for carrying out fusion analysis on the visible light video data and the multispectral image data, and identifying and associating individual fries in time sequence; the collaborative coding module is used for collaborative coding of the kinematic features extracted from the visible light video data and the spectral reflection features extracted from the multispectral image data to obtain a coding result; the characteristic construction module is used for constructing a multi-mode characteristic vector for each associated individual fish fry based on the coding result; The map construction module is used for constructing a time-sequence group dynamic interaction map based on the associated individual fries and the multi-mode feature vectors; the dissimilation detection module is used for inputting the group dynamic interaction map into a pre-trained graph neural network anomaly detection model so as to calculate group behavior dissimilation indexes; And the risk positioning module is used for judging the abnormal state of the group based on the group behavior dissimilation index and positioning the abnormal high-risk individual fries.

Description

Dynamic fish fry technical method and system based on computer vision Technical Field The invention relates to the technical field of image processing, in particular to a fry dynamic technical method and system based on computer vision. Background In recent years, computer vision-based intelligent aquaculture technology has been rapidly developed. In the field of fry counting and sorting, the mainstream technical solutions generally rely on building a standardized work environment to simplify the identification task. For example, by designing a slide or duct at a particular angle of inclination, the fry is guided through the detection zone in a relatively regular posture and orientation. Under such a controlled scene, a technical route combining target detection (such as a YOLO series model) and multi-target tracking (such as SORT, deepSORT algorithm) can realize high-precision dynamic counting and tracking of fish fries with stable and forward motion tracks. The method provides an effective automatic way for solving the problems of low efficiency and large error of the traditional manual counting. However, in a practical large-scale, high-density cultivation scenario, the behavior of fries is highly diverse and uncontrollable, resulting in a significant lack of generalization ability of the above-described environmental constraint-dependent methods. And is embodied as having poor fault tolerance for the identification of irregular athletic activity. When individual fries are in retrograde, stagnant, in-situ violent struggling or jumping and other actions caused by environmental stimulus, physical difference or stress reaction, the generated discontinuous, nonlinear and high-dynamic motion tracks are extremely easy to exceed the motion model prediction range of the traditional tracking algorithm, so that the tracking tracks are interrupted, the targets are lost or the Identity (ID) is frequently switched. The method not only directly causes inaccurate counting results (such as repeated counting or missing counting), but also can not effectively identify and quantitatively analyze the stress state and individual abnormal behaviors of the fry group, thereby limiting the reliable application of the intelligent management system in complex real scenes. Disclosure of Invention The invention aims to provide a fry dynamic technical method and system based on computer vision, which aims to solve the defects in the background technology. In order to achieve the purpose, the invention provides the following technical scheme that the fry dynamic technical method based on computer vision comprises the following steps: Synchronously collecting visible light video data and multispectral image data of a fry group; Carrying out fusion analysis on the visible light video data and the multispectral image data, and identifying and correlating individual fries in time sequence; The kinematic features extracted from the visible light video data and the spectral reflection features extracted from the multispectral image data are cooperatively encoded to obtain an encoding result; constructing a multi-modal feature vector for each associated individual fry based on the encoding result; Constructing a time-ordered group dynamic interaction map based on the associated individual fries and the multi-modal feature vectors; inputting the group dynamic interaction map to a pre-trained graph neural network anomaly detection model to calculate group behavior dissimilarity indexes; And judging the abnormal state of the group based on the group behavior dissimilation index, and positioning the abnormal high-risk individual fries. In a preferred embodiment, in the step of synchronously acquiring the visible light video data and the multispectral image data of the fry population, the data is acquired by synchronously disposing a visible light camera and a multispectral camera, wherein an operating band of the multispectral camera comprises a near infrared band. In a preferred embodiment, in the step of performing fusion analysis on the visible light video data and the multispectral image data to identify and temporally associate individual fries, the fusion analysis specifically comprises the steps of realizing identification and association of individuals through a target detection and multi-target tracking algorithm based on the visible light video data and aligning the multispectral image data with video frames by utilizing time stamps. In a preferred embodiment, the step of performing cooperative coding on the kinematic features extracted from the visible light video data and the spectral reflection features extracted from the multispectral image data to obtain a coding result, wherein the cooperative coding specifically comprises the steps of adaptively weighted fusion, calculating a behavior activity index according to the kinematic features, and dynamically adjusting the weight coefficient of the spectral reflection features during fu