CN-121999156-A - Underwater three-dimensional terrain reconstruction method based on forward looking sonar, electronic equipment and storage medium

CN121999156ACN 121999156 ACN121999156 ACN 121999156ACN-121999156-A

Abstract

The invention relates to an underwater three-dimensional terrain reconstruction method, electronic equipment and storage medium based on forward-looking sonar, which comprises the following steps of firstly preprocessing a forward-looking sonar image sequence, adopting a convolutional neural network to encode multi-frame continuous sonar images, extracting and compressing the multi-frame continuous sonar images into acoustic feature vectors, secondly obtaining a body perception data sequence of an unmanned ship, and adopting a gating circulation unit network to conduct time sequence modeling on the body perception data to generate a motion state vector. According to the invention, acoustic feature vectors and motion state vectors are extracted through a deep learning model, historical information is fused by utilizing a feature fusion network, and finally an underwater three-dimensional terrain model is output through a terrain reconstruction network.

Inventors

He Elong
LIN YANJUN
LIU YONG

Assignees

中国船舶集团有限公司第七一五研究所
浙江大学

Dates

Publication Date: 20260508
Application Date: 20260119

Claims (9)

1. The underwater three-dimensional terrain reconstruction method based on the forward looking sonar is characterized by comprising the following steps of: Preprocessing a forward-looking sonar image sequence, encoding a plurality of continuous sonar images by adopting a convolutional neural network, and extracting and compressing the acoustic feature vectors; step two, acquiring a body sensing data sequence of the unmanned ship, and carrying out time sequence modeling on the body sensing data by adopting a gating circulation unit network to generate a motion state vector; thirdly, splicing the acoustic feature vector and the motion state vector to form a fusion feature vector, and inputting the fusion feature vector into a feature fusion network for processing to obtain a topographic feature vector containing environmental observation and motion state history information; And step four, inputting the terrain feature vector into a terrain reconstruction network based on a multi-layer perceptron, decoding to generate an elevation map, and realizing reconstruction of underwater three-dimensional terrain.
2. The underwater three-dimensional terrain reconstruction method based on forward-looking sonar of claim 1, wherein in the first step, the preprocessing specifically comprises the steps of intercepting a rectangular area with a fixed size from sector data of the forward-looking sonar as an input image, and splicing continuous multiframe input images in a channel dimension to form input data of a convolutional neural network.
3. The forward-looking sonar-based underwater three-dimensional terrain reconstruction method as set forth in claim 2, wherein the convolutional neural network is an encoder structure and comprises at least three convolutional layers, the number of channels of the convolutional layers increases gradually layer by layer, normalization processing and activation function mapping are sequentially carried out after each layer of convolution, and finally the acoustic feature vector with a fixed dimension is obtained through a global pooling layer.
4. The method for reconstructing underwater three-dimensional terrain based on forward-looking sonar of claim 1, wherein in the second step, the body sensing data comprises at least one of a three-dimensional position, a three-dimensional posture, a three-dimensional speed and a pair bottom height, and the gating cycle unit network processes the body sensing data of consecutive multiframes to capture a time sequence dependency relationship of unmanned ship motion.
5. The method for reconstructing underwater three-dimensional terrain based on forward-looking sonar of claim 1, wherein in the third step, the feature fusion network is a multi-layer neural network structure and is used for further information fusion and feature enhancement of the fusion feature vectors after splicing.
6. The method of claim 1, wherein in the fourth step, the multi-layer perceptron network comprises a plurality of fully connected layers, and the output dimension of the multi-layer perceptron network corresponds to the total number of pixels of the target elevation map, and is used for mapping the topographic feature vector into two-dimensional elevation data.
7. The method for reconstructing underwater three-dimensional terrain based on forward-looking sonar according to claim 1, wherein the method further comprises a model training step of forming a training data set by utilizing underwater terrain truth value data acquired by multi-beam sonar and synchronously acquired forward-looking sonar image sequences and unmanned ship body perception data, and performing end-to-end joint training on the convolutional neural network, the gate-controlled circulation unit network, the feature fusion network and the multi-layer perceptron network by taking a mean square error between a minimized reconstruction elevation map and the terrain truth value as a loss function.
8. An electronic device, comprising: A memory for storing a computer program; A processor for implementing a forward looking sonar-based underwater three-dimensional terrain reconstruction method as defined in any of claims 1 to 7 when executing said computer program.
9. A computer-readable storage medium, wherein a computer program is stored on the storage medium, which when executed by a processor, implements the forward-looking sonar-based underwater three-dimensional terrain reconstruction method as defined in any one of claims 1 to 7.

Description

Underwater three-dimensional terrain reconstruction method based on forward looking sonar, electronic equipment and storage medium Technical Field The invention belongs to the technical field of underwater environment sensing and three-dimensional reconstruction, and particularly relates to an underwater three-dimensional terrain reconstruction method based on forward looking sonar, electronic equipment and a storage medium. Background The obstacle avoidance is an important function of safe running of the unmanned ship, and the underwater obstacle avoidance of the unmanned ship is more challenging compared with the obstacle avoidance on the water surface due to single underwater remote detection and sensing. The Chinese patent application with the patent publication number of CN117472051A discloses a semi-submersible unmanned ship obstacle avoidance method and system, wherein underwater obstacle avoidance uses forward looking sonar, however, the traditional forward looking sonar has the problem of vertical direction information missing, only can provide azimuth and distance information of a target, is difficult to directly acquire elevation data, and brings great challenges to obstacle avoidance of the unmanned ship in the water surface navigation process. In addition, in the prior art, an orthogonal double sonar arrangement is adopted, and data is synchronously acquired through a visual field overlapping area to realize three-dimensional measurement, however, the hardware configuration of two sonars is relied on, and the visual field is smaller. In recent years, the deep learning technology provides new ideas for sonar data analysis, such as a sonar target recognition method based on GAF-D3Net and a reconstruction scheme of fusion of an acoustic image and an optical image. However, these approaches have focused on target recognition or rely on optical data assistance. Therefore, on the premise of not adding additional special three-dimensional detection hardware, the real-time and reliable reconstruction of the underwater local three-dimensional topography is realized by only utilizing the forward-looking sonar and the navigation sensor marked by the unmanned ship, which becomes a technical problem to be solved in the field. Disclosure of Invention The invention aims to solve the technical problem of providing an underwater three-dimensional terrain reconstruction method, electronic equipment and storage medium based on forward-looking sonar, which are used for overcoming the defect of information deficiency in the vertical direction of the forward-looking sonar in the prior art, extracting acoustic feature vectors and motion state vectors through a deep learning model, fusing historical information by utilizing a feature fusion network, and finally outputting an underwater three-dimensional terrain model through a terrain reconstruction network. The technical scheme of the invention is that the underwater three-dimensional terrain reconstruction method based on the forward looking sonar is provided and is characterized by comprising the following steps: preprocessing a forward-looking sonar image sequence, encoding a plurality of continuous sonar images by adopting a Convolutional Neural Network (CNN), extracting and compressing the acoustic feature vectors; step two, acquiring a body sensing data sequence of the unmanned ship, and performing time sequence modeling on the body sensing data by adopting a gate control loop unit (GRU) network to generate a motion state vector; thirdly, splicing the acoustic feature vector and the motion state vector to form a fusion feature vector, and inputting the fusion feature vector into a feature fusion network for processing to obtain a topographic feature vector containing environmental observation and motion state history information; And step four, inputting the terrain feature vector into a terrain reconstruction network based on a multi-layer perceptron (MLP), decoding to generate an elevation map, and realizing reconstruction of underwater three-dimensional terrain. According to the invention, by fusing the forward-looking sonar data with the historical state data of the unmanned ship and through a terrain reconstruction network, high-precision real-time underwater three-dimensional terrain reconstruction without additional sensing hardware can be realized on the unmanned ship embedded computing platform, and the underwater obstacle avoidance of the unmanned ship is assisted. Preferably, in the first step, the preprocessing specifically includes that a rectangular area with a fixed size is cut from sector data of the front-view sonar to serve as an input image, and continuous multiframes of the input images are spliced in a channel dimension to form input data of a convolutional neural network. Furthermore, the convolutional neural network is an encoder structure and comprises at least three convolutional layers, the channel number of the convolutional layers is