CN-120927066-B - Multi-sensor fusion simulation fish underwater environment real-time monitoring method and simulation fish thereof

CN120927066BCN 120927066 BCN120927066 BCN 120927066BCN-120927066-B

Abstract

The invention discloses a multi-sensor fusion simulation fish underwater environment real-time monitoring method which comprises the following steps of S1, deploying a simulation fish to a target underwater environment, initializing and collecting video stream, audio stream, water quality data and hydrologic data, S2, respectively preprocessing the video stream, the audio stream, the water quality data and the hydrologic data, S3, fusing the preprocessed video stream, the preprocessed audio stream, the preprocessed water quality data and the preprocessed hydrologic data to generate a fusion characteristic matrix, and S4, compressing the fusion characteristic matrix to obtain final compressed data, and simultaneously transmitting the final compressed data through underwater acoustic communication. Therefore, the underwater acoustic communication can be adapted to the data compression and verification design through targeted preprocessing, space-time correlation and multi-feature fusion, the data quality and comprehensive utilization efficiency are improved, the transmission effect is guaranteed, the split type structure of the simulated fish is convenient to disassemble, assemble and maintain, flexible in movement and convenient and adaptive to monitoring are improved through the integrated design.

Inventors

MU YANTAO
CHEN HAO
WANG JIASU
WANG ZICONG
YANG XIAO
WU DONGLIN
Xue Guanying
GAO FANQI
HONG YAN
JIANG ZHAOBO

Assignees

郑州大学

Dates

Publication Date: 20260512
Application Date: 20250819

Claims (9)

1. The multi-sensor fusion simulation fish underwater environment real-time monitoring method is characterized by comprising the following steps of: S1, deploying a simulated fish to a target underwater environment, initializing and collecting video stream, audio stream, water quality data and hydrologic data; s2, respectively preprocessing the video stream, the audio stream, the water quality data and the hydrologic data, wherein the preprocessing is specifically as follows: s2.1, carrying out time-space domain joint denoising on the video stream to obtain a denoised frame sequence, and carrying out inter-frame redundancy inhibition on the denoised frame sequence to obtain a preprocessed video stream, wherein the method specifically comprises the following steps: Wherein, the For the video stream to be a video stream, For the de-noised sequence of frames, For the spatial domain gaussian filtering, As the time-weighting factor is used, For the video stream after the preprocessing, As a function of the norm, Is a difference threshold; S2.2, performing frequency domain conversion on the audio stream to obtain an audio frequency domain signal, performing adaptive noise frequency band suppression on the audio frequency domain signal to obtain a denoised audio frequency domain signal, and performing time domain conversion on the denoised audio frequency domain signal to obtain a preprocessed audio signal, wherein the method specifically comprises the following steps of: Wherein, the For the audio stream to be described, For the audio frequency domain signal, In the case of a fourier transform operator, For the denoised audio frequency domain signal, As a function of the weight of the frequency band, For the preset noise band threshold value, Is the audio signal after preprocessing; s2.3, removing jump noise of the water quality data to obtain the water quality data after the jump, and carrying out baseline drift correction on the water quality data after the jump to obtain the water quality data after the pretreatment, wherein the method specifically comprises the following steps: Wherein, the In order to provide the water quality data, For the data of the water quality after the jump removal, For the jump threshold value, For the water quality data after the pretreatment, Is the sliding window size; s2.4, performing high-frequency interference filtering on the hydrologic data to obtain filtered hydrologic data, and performing physical range constraint correction on the hydrologic data to obtain preprocessed hydrologic data, wherein the method specifically comprises the following steps of: Wherein, the In order to provide the said hydrological data, For the filtered hydrologic data, In order for the filter coefficients to be of a type, For the pre-processed hydrologic data, Is the upper and lower limits of the physical range of the parameter; S3, fusing the preprocessed video stream, the preprocessed audio stream, the preprocessed water quality data and the preprocessed hydrologic data to generate a fusion feature matrix; And S4, compressing the fusion feature matrix to obtain final compressed data, and transmitting the final compressed data through underwater acoustic communication.
2. The method for monitoring the underwater environment of the multi-sensor fusion simulation fish according to claim 1, wherein the step S3 comprises the following sub-steps: S3.1, unifying time references of the preprocessed video stream, the preprocessed audio stream, the preprocessed water quality data and the preprocessed hydrologic data to obtain a time-synchronized signal set, and performing spatial coordinate correlation on the time-synchronized signal set to obtain a spatial correlated signal set; S3.2, extracting the characteristics in the signal set after spatial correlation to obtain video low-frequency characteristics, audio characteristic vectors and environment characteristic vectors; s3.3, obtaining a fusion feature matrix for the frequency low-frequency feature, the audio feature vector and the environment feature vector.
3. The method for monitoring the underwater environment of the multi-sensor fusion simulation fish according to claim 2, wherein the step S3.1 comprises the following sub-steps: S3.1.1, taking the sampling frequency of the preprocessed video stream as a reference, performing linear interpolation alignment on the preprocessed audio stream, the water quality data and the hydrologic data to obtain the time-synchronous audio stream, the water quality data and the hydrologic data, and combining the preprocessed video stream, the time-synchronous audio stream, the time-synchronous water quality data and the hydrologic data into a time-synchronous signal set, wherein the method specifically comprises the following steps of: ; Wherein, the For the audio stream to be time-synchronized, For the time-synchronized data of said water quality, For the time-synchronized said hydrographic data, In order to indicate the function, A set of signals that are said time-synchronized; S3.1.2 mapping the water quality data and the hydrologic data after time synchronization to a video space of the video stream after preprocessing through simulating a fish motion track to obtain the water quality data and the hydrologic data which are spatially related, wherein the water quality data and the hydrologic data are specifically as follows: Wherein, the For the spatially correlated water quality data, For the spatially correlated said hydrographic data, As a dirac function, Is that The fish position is simulated at any time, Is the set of signals after spatial correlation.
4. The method for monitoring the underwater environment of the multi-sensor fusion simulation fish according to claim 3, wherein the step S3.2 comprises the following substeps: s3.2.1 extracting video low-frequency characteristics of the video streams in the signal set after spatial correlation through two-dimensional Fourier transform, specifically: Wherein, the As a feature of the low frequency of the video, As the frequency domain coordinates of the antenna array, Is a low frequency threshold; S3.2.2 calculating the short-time energy and zero crossing rate of the time-synchronized audio stream to extract audio feature vectors, specifically: Wherein, the For the audio feature vector to be used, For the length of the window it is, Is the sampling interval; s3.2.3 extracting the water quality data and the hydrologic data which are spatially related to extract environmental feature vectors, specifically: Wherein, the Is the environmental feature vector.
5. The multi-sensor fusion simulation fish underwater environment real-time monitoring method according to claim 4, wherein the step S3.3 comprises the following sub-steps: s3.3.1, respectively calculating weights of the video low-frequency feature, the audio feature vector and the environment feature vector based on signal-to-noise ratio to obtain a dynamic weight set; s3.3.2 generating a space-time fusion matrix based on the video low-frequency feature, the audio feature vector, the environment feature vector and the dynamic weight set, specifically: Wherein, the For the purpose of the space-time fusion matrix, Is the dynamic weight set.
6. The method for real-time monitoring of a multi-sensor fusion simulation fish under water environment according to claim 5, wherein in said step 4, parity bits are added to the last bit of the code of each element when said space-time fusion matrix is compressed.
7. The simulated fish based on the multi-sensor fusion simulated fish underwater environment real-time monitoring method disclosed by claim 1 is characterized by being of a split type structure, wherein the whole fish is detachably connected by a fish head section, a fish body section and a fish tail section in sequence, the fish head section, the fish body section and the fish tail section are of symmetrical split type structures, each of the fish head section, the fish body section and the fish tail section independently comprises two half shells matched with each other, and the two half shells are mutually buckled along a bilateral symmetry plane to jointly form a cavity structure; The connecting part of the fish body section and the fish tail section is provided with two connecting pieces which are matched with each other, the two connecting pieces are buckled to form a complete annular structure, the connecting part of the fish body section and the fish tail section is sealed after the two connecting pieces are buckled, and the fish body section and the fish tail section are fixed; a pressing plate is arranged in a cavity at the joint of the fish body section and the fish tail section; The tail end of the fish tail section is provided with a tail fin, the joint of the tail fin and the fish tail section is provided with another two connectors which are matched with each other, the connectors are buckled to form a complete annular structure, and the inner wall of the buckled fish tail section is respectively pressed and attached with the outer wall of the fish tail section and the outer wall of the tail fin; The left side and the right side of the outer wall of the fish body section are respectively provided with a flank, and the cavity of the fish body section is provided with two swinging pieces.
8. The artificial fish of claim 7, wherein the cavity of the fish head section is provided with an image sensor, an acoustic sensor, a water quality sensor, a hydrological sensor, a data processing module, an underwater acoustic communication module, and a power source.
9. The artificial fish according to claim 8, wherein the swinging member is provided with two cylinders and two steering gears, the piston is arranged in the cylinders, and the two steering gears drive the two pistons to move through the connecting rod structure respectively.

Description

Multi-sensor fusion simulation fish underwater environment real-time monitoring method and simulation fish thereof Technical Field The invention relates to the field of simulation machinery, in particular to a multi-sensor fusion simulation fish underwater environment real-time monitoring method and a simulation fish thereof. Background The underwater environment monitoring has important significance in the fields of water resource protection, ecological research and the like. Currently, acquisition of underwater environmental data by sensors is a common monitoring method, and these sensors include devices for image acquisition, devices for sound capture, and sensors for detecting water quality, hydrologic parameters, and the like. In terms of data processing, in the prior art, data collected by multiple sensors are often preprocessed separately and then integrated, and the mode may cause insufficient relevance of different types of data in time and space, so that the comprehensive utilization effect of the data is affected. Meanwhile, problems such as noise, data redundancy and the like in the underwater environment can also reduce the accuracy and effectiveness of monitoring data. In the aspect of monitoring equipment, the device for underwater monitoring is inconvenient in disassembly and assembly in structural design, and is inconvenient in equipment maintenance and sensor replacement. In addition, some devices have limited mobility, are difficult to accommodate in complex underwater environments, and may limit the range and efficiency of monitoring. Disclosure of Invention The present invention aims to solve at least some of the technical problems in the above-described technology. Therefore, on the one hand, the invention discloses a multi-sensor fusion simulation fish underwater environment real-time monitoring method, which comprises the following steps: S1, deploying a simulated fish to a target underwater environment, initializing and collecting video stream, audio stream, water quality data and hydrologic data; s2, respectively preprocessing the video stream, the audio stream, the water quality data and the hydrologic data; S3, fusing the preprocessed video stream, the preprocessed audio stream, the preprocessed water quality data and the preprocessed hydrologic data to generate a fusion feature matrix; And S4, compressing the fusion feature matrix to obtain final compressed data, and transmitting the final compressed data through underwater acoustic communication. In addition, the multi-sensor fusion simulation fish underwater environment real-time monitoring method disclosed by the invention can also have the following additional technical characteristics: Further, the step S2 includes the following substeps: s2.1, carrying out time-space domain joint denoising on the video stream to obtain a denoised frame sequence, and carrying out inter-frame redundancy inhibition on the denoised frame sequence to obtain a preprocessed video stream, wherein the method specifically comprises the following steps: Wherein, the For the video stream to be a video stream,For the de-noised sequence of frames,For the spatial domain gaussian filtering,As the time-weighting factor is used,For the video stream after the preprocessing,As a function of the norm,Is a difference threshold; S2.2, performing frequency domain conversion on the audio stream to obtain an audio frequency domain signal, performing adaptive noise frequency band suppression on the audio frequency domain signal to obtain a denoised audio frequency domain signal, and performing time domain conversion on the denoised audio frequency domain signal to obtain a preprocessed audio signal, wherein the method specifically comprises the following steps of: Wherein, the For the audio stream to be described,For the audio frequency domain signal,In the case of a fourier transform operator,For the denoised audio frequency domain signal,As a function of the weight of the frequency band,For the preset noise band threshold value,Is the audio signal after preprocessing; s2.3, removing jump noise of the water quality data to obtain the water quality data after the jump, and carrying out baseline drift correction on the water quality data after the jump to obtain the water quality data after the pretreatment, wherein the method specifically comprises the following steps: Wherein, the In order to provide the water quality data,For the data of the water quality after the jump removal,For the jump threshold value,For the water quality data after the pretreatment,Is the sliding window size; s2.4, performing high-frequency interference filtering on the hydrologic data to obtain filtered hydrologic data, and performing physical range constraint correction on the hydrologic data to obtain preprocessed hydrologic data, wherein the method specifically comprises the following steps of: Wherein, the In order to provide the said hydrological data,For the filtered hydrologic d