Search

CN-121982771-A - Method for detecting activity of living fish based on behavior feature capture and analysis

CN121982771ACN 121982771 ACN121982771 ACN 121982771ACN-121982771-A

Abstract

The invention belongs to the technical field of computer vision and aquaculture intellectualization, and particularly relates to a living fish activity detection method based on behavior feature capture and analysis. According to the invention, by innovatively integrating CBAM and SE two attention mechanisms into the YOLOv model for optimization, the model can be better focused on the key characteristics of the fish body, the detection precision and the robustness of small targets and fine motions in a complex underwater environment are remarkably improved, compared with the original model, mAP is improved to 99%, and the model volume is reduced to 342MB. The method breaks through the limitation of single characteristic evaluation, creatively fuses the motion speed, the motion distance and the stress jump frequency, builds a double calculation and prediction model from behaviors to activity scores, ensures that the activity evaluation result is more objective, accurate and comprehensive, and has definite physical and physiological significance.

Inventors

  • SHEN XU
  • SHI JIYONG
  • ZOU XIAOBO
  • ZHANG JUNJUN

Assignees

  • 江苏大学

Dates

Publication Date
20260505
Application Date
20260127

Claims (10)

  1. 1. A method for detecting the activity of living fish based on behavior feature capture and analysis, which is characterized by comprising the following steps: S1, providing N experimental fishes, and placing the experimental fishes in a water tank with controllable environmental parameters, wherein the environmental parameters comprise water temperature, dissolved oxygen and pH value; the system comprises a water tank, a collecting device, a plurality of video sequences, a plurality of sampling device and a sampling device, wherein the collecting device is arranged above the water tank and is used for continuously collecting behavior videos of the experimental fish, the behavior videos of the experimental fish in a normal swimming state and at least one preset stress state are collected through the collecting device to form N video sequences containing multi-state behaviors and recorded as Vn as sample data, and N is a positive integer not less than 2; S2, constructing and marking a data set, namely acquiring K video sequences V1, V2 of different time periods from the video sequence Vn in the step S1, further extracting image frames to obtain an image set, preprocessing and enhancing the quality of the image to obtain a preprocessed image frame, and carrying out multi-level fine marking on the preprocessed image frame to obtain a marked image frame, and then applying at least one of space transformation enhancement and color transformation enhancement to the marked image frame to obtain an enhanced complete data set; S3, fish body real-time detection and tracking: The method comprises the steps of taking a YOLOv model as an infrastructure, optimizing by introducing a Convolution Block Attention Module (CBAM) and an extrusion excitation (SE) network module into a backbone network and/or a feature fusion network of the model to obtain a YOLOv-Attention model, detecting and tracking the fish body in real time by using the YOLOv-Attention model, inputting the labeling format of a fish body boundary frame in a training dataset into the YOLOv-Attention model by using the training dataset to train to obtain a trained YOLOv-Attention model, inputting the YOLOv-Attention model by using the training dataset to train to obtain a trained YOLOv-Attention model, outputting the boundary frame coordinates and confidence of all detected fish bodies in each frame of image by using the center point coordinates as the positions of the fish bodies, distributing and maintaining unique identity IDs for each fish body by using a multi-target tracking algorithm, and obtaining a continuous fish body tracking sequence with unique IDs to realize frame-crossing continuous tracking; S4, motion characteristic data acquisition and grading grade definition: Acquiring behavior characteristic parameters corresponding to each fish based on the fish body continuous tracking sequence with the unique ID output in the step S3, wherein the behavior characteristic parameters comprise kinematic parameters, track accumulation parameters and stress behavior parameters, and then constructing a fusion model based on the behavior characteristic parameters to quantify the activity level of the fish so as to realize the mapping from the behavior characteristics to the activity scores; Calculating corresponding activity grade assessment according to the activity grading model; The method comprises the steps of constructing a fusion model by adopting a composite structure combining a linear regression layer and a depth feature fusion module, optimizing weight parameters through a gradient descent algorithm, constructing the fusion model by taking a minimum mean square error as a training target, and finally realizing accurate and automatic prediction from behavior features to activity scores; The preset activity scoring model is obtained by taking multidimensional behavior characteristics of movement speed, accumulated distance and stress jump frequency as input, and constructing a quantitative mapping relation between the characteristics and the activity score through a weighted fusion or machine learning regression method; s5, establishing and verifying a prediction model: Based on the behavior characteristic parameters extracted and calculated in the step S4, an activity score prediction model is constructed by utilizing the characteristic variables after standardized processing, the characteristic engineering and the training of the model are completed, the performance and the interpretability of the model are systematically evaluated, and finally the whole model is deployed; S51, dividing behavior characteristic parameters of each fish in total duration H into continuous data blocks according to window length W and interval S, extracting five statistics of mean value, standard deviation, maximum value, minimum value and change trend slope in each data block to form a sample characteristic vector, and then carrying out batch Z-score standardization to obtain the characteristic variables after standardization; S52, model architecture and training: The activity score prediction model comprises a depth feature fusion module and a terminal linear regression layer, wherein the depth feature fusion module consists of 1-3 layers of fully-connected networks, the number of neurons in each layer is 32-256, and an activation function adopts ReLU or Sigmoid; training the model by using a mean square error as a loss function and using an Adam or SGD optimizer to obtain an S52 trained activity score prediction model, wherein the initial learning rate is set to be 0.001-0.01, the batch processing size is 16-64, the training round number is 100-500, and the early stop strategy of verification is adopted to prevent overfitting; S53, performance verification and explanation, namely, adopting K-fold cross verification and an independent test set to evaluate the performance of the model, wherein K=5 or 10, the independent test set accounts for 10% -30%, and core evaluation indexes comprise root mean square error, average absolute error and decision coefficient, wherein RMSE is required to be less than or equal to 8.0, MAE is required to be less than or equal to 6.5, and R2 is required to be more than or equal to 0.80; S54, deployment and reasoning, namely converting the activity score prediction model trained in the step S52 into ONNX or TensorRT format for solidification, and deploying the model in edge computing equipment or a server, wherein the reasoning time of a single frame image of the model is not more than 0.2 seconds so as to realize real-time activity score prediction of a new input video stream; S6, practical application of a model: And (3) selecting a living fish sample to be detected, acquiring behavior characteristic parameters of the sample according to the operation of S1 to S3, and substituting the behavior characteristic parameters into an activity score prediction model constructed in the step S5, so that the corresponding activity score can be rapidly predicted through the loaded model, and the living fish activity detection of the sample is realized.
  2. 2. The method for detecting the activity of the living fish based on behavior feature capture and analysis is characterized in that a collecting device in the step S1 comprises a visual collecting high-definition camera, the frame rate of the high-definition camera is 60fps, the experimental fish is fresh water fish with the same culture batch and similar health conditions and comprises at least one of grass carp, tilapia or carp, the number N of the experimental fish is a positive integer not less than 10, the water temperature is maintained at 15-30 ℃, the dissolved oxygen amount is maintained at 5.0 mg/L-8.0 mg/L, and the pH value is maintained at 6.5-8.0.
  3. 3. The method for detecting the activity of the living fish based on the behavior feature capture and analysis according to claim 1, wherein the preset stress state in the step S1 comprises at least one of a surprise stimulus, an anoxic stimulus, a feeding stimulus, a high-density stress and a temperature shock stress, wherein the specific operation of the surprise stimulus is realized by instantaneously beating a water tank or applying short-time strong light, the anoxic stimulus is realized by reducing the ventilation volume or injecting nitrogen to reduce the dissolved oxygen content to below 4.0 mg/L, the feeding stimulus is realized by throwing baits, the high-density stress is realized by increasing the fish quantity in a unit volume to 1.5-3 times of the initial density, and the temperature shock stress is realized by raising or lowering the water temperature by 4-6 ℃ in 10 minutes.
  4. 4. The method for detecting living fish activity based on behavior feature capture and analysis according to claim 1, wherein the step of extracting image frames in the step S2 is to analyze and extract the collected original video sequences V1, V2, VK, analyze and extract frames by using a video processing library according to a fixed time interval or frame rate to obtain an image frame set, wherein the fixed time interval is 0.1 to 0.5 seconds, or the fixed frame rate is 2 to 10 frames per second; The image preprocessing and quality enhancement steps comprise image preprocessing and quality enhancement, namely sequentially carrying out size standardization, color space conversion and image enhancement processing on an image frame set to improve image quality and consistency and obtain preprocessed image frames, wherein the size standardization comprises uniformly scaling the image to a fixed resolution, the size standardization comprises 640 pixels multiplied by 480 pixels, 1024 pixels multiplied by 768 pixels or 1280 pixels multiplied by 720 pixels, the color space conversion comprises conversion from an RGB format to a gray scale format or an HSV format, the image enhancement processing comprises at least one of histogram equalization and Contrast Limited Adaptive Histogram Equalization (CLAHE), and the enhancement intensity factor is 1.0 to 2.0; The multi-level fine labeling step comprises the steps of labeling the preprocessed image frames by using a labeling tool in at least two levels, wherein the first level is a target detection labeling for calibrating the position of a fish body in the image, the labeling is performed by using a rectangular boundary box or a polygonal area, the second level is a behavior classification labeling for calibrating the behavior state of the fish body in the corresponding frame, and the behavior state at least comprises one of stress behaviors mentioned in S14.
  5. 5. The method for detecting the activity of the living fish based on the behavior feature capturing and analyzing according to claim 1, wherein the spatial transformation enhancement in the step S2 comprises at least one of random horizontal overturn, random rotation angle of-15 degrees to +15 degrees and random scaling of 0.9 to 1.1, wherein the color transformation enhancement comprises random adjustment of image brightness, contrast or saturation, and adjustment amplitude is-10% to +10% of an original value; The method comprises the steps of training sets accounting for 70-80%, verifying sets accounting for 10-15% and testing sets accounting for 10-15%, wherein when dividing, the same fish individuals are ensured not to repeatedly appear in all sets, the distribution proportion deviation of each behavior state in the training sets, the verifying sets and the testing sets is not more than 5%, and the divided data sets are subjected to version identification and archiving, wherein the version identification rule is data set name_V major version number and minor version number.
  6. 6. The method for detecting the activity of the living fish based on the behavior feature capturing and analyzing as set forth in claim 1, wherein in the step S3, the serial sequence of the channel Attention and the space Attention of the convolution block Attention module is channel-first-channel-then-space, the channel compression ratio is set to 4 to 16, the compression ratio of the compression excitation Attention module is set to 8 to 32, and the input image resolution of the YOLOv-Attention model is fixed to 640 x 640 pixels, 800 x 800 pixels or 1024 x 1024 pixels; Inputting the YOLOv-Attention model by using a training data set for training, wherein in the training process, an optimizer is AdamW or SGD, the initial learning rate is set to be 0.001-0.01, a cosine annealing strategy is adopted for carrying out learning rate attenuation, the batch processing size is set to be 8, 16 or 32, the total training wheel number is set to be 100-300, in a training loss function, CIoU loss is adopted as a bounding box regression loss, and binary cross entropy loss is adopted as a confidence loss and a classification loss; The multi-objective tracking algorithm includes, but is not limited to DeepSORT, byteTrack or Bot-SORT, wherein the similarity threshold value used inside the algorithm to determine whether two objective frames belong to the same object is set to 0.3-0.7.
  7. 7. The method for detecting the activity of living fish based on the capturing and analyzing of the behavioral characteristics according to claim 1, wherein the step S4 of obtaining the behavioral characteristic parameters corresponding to each fish comprises the steps of: The method comprises the steps of calculating tangential speed, normal speed and combined speed of a fish body boundary frame based on displacement of a center point of the fish body boundary frame between continuous video frames, and further calculating average speed in a specified time, wherein the length of the specified time is T seconds, and T is 5 seconds to 60 seconds; track accumulation parameters, namely accumulating and calculating the sum of the single fish swimming track distances as the total movement distance of the fish within the specified time; the method comprises the steps of detecting the mutation of a fish body movement track under a set short-time stress stimulation condition according to stress behavior parameters, and counting the jumping times of the fish body movement track in unit time to be used as stress jumping frequency, wherein the unit time is 1-5 seconds, and the stress stimulation condition comprises illumination mutation, sound stimulation or water flow disturbance, and the stimulation duration is 1-10 seconds.
  8. 8. The method for detecting the activity of living fish based on the behavior feature capturing and analyzing as set forth in claim 1, wherein preferably, the calculating the corresponding activity level evaluation according to the activity scoring model in step S4 includes the steps of: S41, standard division of activity grades, namely dividing the activity level of fish into K discrete activity grades in advance, defining a corresponding activity fraction range and typical behavior description for each grade, and establishing an evaluation standard, wherein K is a positive integer from 3 to 5, the activity fraction range is 0-15 minutes, and the fraction interval occupied by each grade is 3-5 minutes; Taking the stress jump frequency as a core input, calculating a Z-score standardized value in the whole historical data set to eliminate dimension, then introducing skewness and kurtosis of data distribution to carry out weighted adjustment on the obtained standardized value so as to correct deviation caused by the data distribution form, and finally, linearly mapping the adjusted score to an activity score range preset by the activity grade standard division through a minimum-maximum normalization method; s42, generating an activity score, namely calculating the activity score of the fish body continuous tracking sequence with the unique ID in each appointed time according to an activity score model, determining the activity grade of the fish body continuous tracking sequence according to the score, and marking the activity grade as a final score result; And repeatedly executing the steps of motion characteristic data acquisition and activity score generation in the continuous observation time period for the same fish, wherein each time interval is t minutes, t is 0.5 to 5 minutes, G is a positive integer and is not less than 3 times, and the gradient time sequence activity score data sets R1, R2 and R_G are correspondingly generated.
  9. 9. The method for detecting the activity of the living fish based on the behavior feature capturing and analyzing according to claim 1, wherein the total duration H in the step S51 is not less than 5 minutes, the window length W is 10-120 seconds, and the interval time S is 5-60 seconds.
  10. 10. The method for detecting the activity of the living fish based on the behavior feature capture and analysis of claim 1, wherein the step S6 of visualizing and displaying the result after the activity detection in real time is performed by dynamically displaying a real-time evaluation conclusion through a multi-modal assembly in a significant area of a visual interface, wherein the real-time evaluation conclusion comprises: S61, displaying the current activity score in real time by using a font of not less than 24pt, wherein the score precision is reserved to the position behind the decimal point, and the score range is 0 to 15 points; S62, synchronously displaying the activity fraction by using a color gradual change progress bar with the measuring range of [0, 15], wherein the 0-5 partition section corresponds to red, the 5-10 partition section corresponds to yellow, and the 10-15 partition section corresponds to green; and S63, mapping the activity score into a specific grade label according to a preset activity grade and displaying the specific grade label, dividing the activity grade into K grades, wherein K is a positive integer from 3 to 5, highlighting the grade label with a background color corresponding to the grade, namely, the inactive grade corresponds to a dark gray background, and the active grade corresponds to a bright green background.

Description

Method for detecting activity of living fish based on behavior feature capture and analysis Technical Field The invention belongs to the technical field of computer vision and aquaculture intellectualization, and particularly relates to a living fish activity detection method based on behavior feature capture and analysis. Background The aquaculture industry plays a vital role in global food supply, and along with the expansion of the cultivation scale and the improvement of the intensive degree, real-time and accurate monitoring of the health state of fish has become a key link for guaranteeing the industrial sustainability, preventing disease outbreaks and reducing economic losses. Traditional fish activity monitoring methods mainly depend on artificial behavior observation, periodic water quality biochemical analysis and emerging physiological index detection (such as blood biochemical analysis, oxygen content monitoring at tissue cell level, muscle electrophysiological signal acquisition and the like). Although manual observation is visual, the natural behaviors of fishes are easily affected due to artificial interference, the natural behaviors of fishes are easily affected due to the fact that the natural behaviors are low in efficiency and high in subjectivity, the natural behaviors of fishes are easily affected due to artificial interference, water quality and physiological index analysis can reflect the information of the cultivation environment and part of the body state, the natural behaviors are mainly invasive or destructive detection means, the operation is complex, the cost is high, the fishes are stressed and damaged, long-term and continuous dynamic monitoring is difficult to realize, and the method has obvious bottlenecks in real-time, objectivity, non-invasiveness and large-scale application. In order to break through the limitations, an intelligent technology represented by computer vision and deep learning is gradually introduced into the field of fish behavior monitoring, new possibility is provided for realizing high-efficiency and objective continuous monitoring through non-invasive video acquisition and automatic analysis, the existing vision-based method still faces multiple challenges such as poor environmental adaptability, weak correlation between characteristics and states, insufficient model performance and the like, namely the problems of illumination fluctuation, water turbidity, fish shielding and the like in a complex underwater environment can obviously reduce the precision and robustness of a detection and tracking algorithm, and a plurality of researches focus on basic identification and track tracking only, lack of a quantization frame for correlating macroscopic behavior characteristics such as movement speed, swimming distance, jump frequency and the like with a health level system, and a general detection model has limited performance when dealing with underwater small targets and dense targets, has too high model complexity and is difficult to meet the real-time processing requirements in actual cultivation. In summary, the prior art has not realized an end-to-end solution for high-precision detection, accurate quantification of behaviors and effective evaluation of health states in a complex environment, and therefore, the invention provides a living fish activity detection method based on behavior feature capture and analysis. Disclosure of Invention Aiming at the defects of the prior art, the invention provides a living fish activity detection method based on behavior feature capturing and analysis, so as to solve the problems. The method for detecting the activity of the living fish based on behavior feature capturing and analyzing comprises the following steps: S1, providing N experimental fishes, and placing the experimental fishes in a water tank with controllable environmental parameters, wherein the environmental parameters comprise water temperature, dissolved oxygen and pH value; the system comprises a water tank, a collecting device, a plurality of video sequences, a plurality of sampling device and a sampling device, wherein the collecting device is arranged above the water tank and is used for continuously collecting behavior videos of the experimental fish, the behavior videos of the experimental fish in a normal swimming state and at least one preset stress state are collected through the collecting device to form N video sequences containing multi-state behaviors and recorded as Vn as sample data, and N is a positive integer not less than 2; Preferably, the acquisition device in step S1 includes a vision acquisition high-definition camera, and the frame rate of the high-definition camera is 60fps. The experimental fish is fresh water fish with the same cultivation batch and similar health conditions, and comprises at least one of grass carp, tilapia or carp, wherein the number N of the experimental fish is a positive integer not less than 10, the wa