CN-121482657-B - Unmanned aerial vehicle group sea area target identification method and device

CN121482657BCN 121482657 BCN121482657 BCN 121482657BCN-121482657-B

Abstract

The embodiment of the application provides a method and a device for identifying a sea area target of an unmanned aerial vehicle group, which are used for realizing effective fusion of data by innovatively designing a data synchronization system, and performing calibration optimization and coordinate conversion. And constructing a feature extraction mechanism, combining depth complementation and attention optimization, and establishing reliable feature expression. And multi-scale detection is introduced, and the accuracy of identification is ensured through loss combination and position optimization. The method effectively solves the defects of the traditional technology in the aspects of data synchronization, feature extraction, target detection and the like, and provides technical support for sea area monitoring.

Inventors

ZHAO HAIYING
ZANG YIHUA
MA XINGMIN
LIU YIXIN
LI XINGHUA

Assignees

中国电子科技集团公司第十五研究所

Dates

Publication Date: 20260508
Application Date: 20260107

Claims (8)

1. A method for identifying a target in a sea area of an unmanned aerial vehicle, the method comprising: The method comprises the steps of deploying an unmanned aerial vehicle group to execute a sea area reconnaissance task, collecting image data acquired by a visible light camera and point cloud data acquired by a laser radar, recording sampling moments of the visible light camera and the laser radar, and establishing a time stamp based on an upper computer time reference, wherein the method comprises the steps of deploying the unmanned aerial vehicle group in a sea area target area, controlling a first unmanned aerial vehicle to collect image information, controlling a second unmanned aerial vehicle to collect point cloud information and generating collecting parameters comprising collecting positions, collecting time and data scale; Index matching is carried out on the image data and the point cloud data according to the latest moment, an internal reference matrix of a visible light camera is calculated through a Zhangor calibration method, an external reference conversion matrix is calculated by selecting a common viewpoint of an image and the point cloud, and the point cloud data is projected to a coordinate system of the image data based on the internal reference matrix and the external reference conversion matrix; Inputting the image data and the point cloud data into a sea area target preprocessing convolutional network, respectively extracting sea area ship image feature vectors and depth feature vectors, complementing the depth feature vectors through a conditional random field network, constructing a sea area depth weight matrix, optimizing the depth weight matrix through an attention mechanism, and generating dense depth features, wherein the method comprises the steps of inputting the depth feature vectors into the conditional random field network, calculating a unitary potential function of each node in a full connected graph, calculating a pair potential function between node pairs based on the image features, generating a node weight matrix, carrying out normalization processing on the node weight matrix, constructing a sea area depth weight matrix, extracting feature patches from the depth weight matrix, calculating dot product operation results of query vectors and key vectors, generating weight coefficients between patches, combining the weight coefficients with relative position coding information, optimizing the depth weight matrix based on the attention mechanism, and generating dense depth features; Inputting the image feature vector and the dense depth feature into a sea area target feature pyramid network, generating a multi-scale fusion feature, calculating the overlapping degree of a ship target boundary frame and a real boundary frame, generating a positioning loss value, calculating a cross entropy loss value of ship category and confidence, carrying out weighted combination on the positioning loss value and the cross entropy loss value, and determining the type and position information of a target ship.
2. The method for identifying the sea area target of the unmanned aerial vehicle according to claim 1, wherein the matching the image data with the point cloud data according to the index of the latest moment, calculating an internal reference matrix of the visible light camera by a zhangshi calibration method, selecting an external reference conversion matrix calculated by a common viewpoint of the image and the point cloud, and projecting the point cloud data to a coordinate system of the image data based on the internal reference matrix and the external reference conversion matrix comprises: Sequencing image data and point cloud data according to sampling time, calculating time intervals of adjacent data frames, selecting image and point cloud data pairs at the latest time based on the time intervals, generating a time sequence matching matrix, and establishing index association between the image data and the point cloud data in the time sequence matching matrix; calculating an internal reference matrix of the visible light camera based on a Zhang calibration method, selecting common-view feature points of an image and point clouds, constructing a three-dimensional coordinate mapping relation of the feature points, calculating an external reference conversion matrix of the laser radar reaching the visible light camera, and projecting the point cloud data to an image coordinate system through the internal reference matrix and the external reference conversion matrix.
3. The method for identifying the sea area target of the unmanned aerial vehicle according to claim 1, wherein the inputting the image data and the point cloud data into the sea area target preprocessing convolutional network respectively extracts the sea surface ship image feature vector and the depth feature vector comprises the following steps: Inputting image data into a first preprocessing convolution layer, setting the number of output channels to be forty-eight, inputting point cloud data into a second preprocessing convolution layer, setting the number of output channels to be sixteen, superposing output feature images of two preprocessing convolution layers to generate a fusion feature image, inputting the fusion feature image into a third preprocessing convolution layer, and setting the number of output channels to be sixty-four; And carrying out fuzzy enhancement processing on the fusion feature map, inputting the enhanced feature map into a feature extraction network, calculating an image feature vector of a sea surface ship based on a neural network, calculating a depth feature vector based on sparse convolution, and carrying out normalization processing on the depth feature vector.
4. The method for identifying a sea area target of an unmanned aerial vehicle according to claim 1, wherein the inputting the image feature vector and the dense depth feature into a sea area target feature pyramid network generates a multi-scale fusion feature, calculates an overlapping degree of a ship target bounding box and a real bounding box, and generates a positioning loss value, comprising: Respectively inputting the image feature vectors and dense depth features into a feature pyramid network, carrying out downsampling treatment on feature graphs of different levels, enhancing the depth features through a spatial attention mechanism and a channel attention mechanism, and superposing the enhanced depth features and the image features to generate multi-scale fusion features; And predicting a target boundary frame based on the multi-scale fusion characteristic, calculating the intersection area of the target boundary frame and a real boundary frame, calculating the union area of the target boundary frame and the real boundary frame, dividing the intersection area by the union area to obtain the overlapping degree, and generating a positioning loss value based on the overlapping degree.
5. The method for identifying the target in the sea area of the unmanned aerial vehicle according to claim 1, wherein calculating the cross entropy loss value of the ship class and the confidence coefficient, and performing weighted combination on the positioning loss value and the cross entropy loss value, and determining the type and the position information of the target ship comprises: Normalizing the predicted ship class probability, comparing the normalized probability with a real label, calculating a cross entropy loss value of class prediction, normalizing the predicted target confidence level, comparing the normalized confidence level with the real label, and calculating the cross entropy loss value of the confidence level; And linearly combining the positioning loss value and the cross entropy loss value, dynamically adjusting a weight coefficient according to a training stage, respectively calculating comprehensive scores for a large ship, a medium civil ship and a small fishing ship, generating sea area target position coordinates and type probabilities, and determining the type and position information of the target ship.
6. An unmanned aerial vehicle group sea area target recognition device, the device comprising: The sea area point cloud data processing module is used for deploying an unmanned aerial vehicle group to execute a sea area reconnaissance task, collecting image data acquired by a visible light camera and point cloud data acquired by a laser radar, recording sampling moments of the visible light camera and the laser radar, and establishing a time stamp based on an upper computer time reference, and comprises the steps of deploying the unmanned aerial vehicle group in a sea area target area, controlling a first unmanned aerial vehicle to carry out visible light camera acquisition, controlling a second unmanned aerial vehicle to carry out laser radar acquisition point cloud information, and generating acquisition parameters including acquisition positions, acquisition time and data scale; recording sampling time information of a visible light camera and a laser radar, correcting the sampling time information by taking an upper computer time reference as a reference standard, calculating hardware transmission delay deviation, and generating a unified time stamp based on the upper computer; index matching is carried out on the image data and the point cloud data according to the latest moment, an internal reference matrix of a visible light camera is calculated through a Zhangor calibration method, an external reference conversion matrix is calculated by selecting a common viewpoint of an image and the point cloud, and the point cloud data is projected to a coordinate system of the image data based on the internal reference matrix and the external reference conversion matrix; The model feature vector processing module is used for inputting the image data and the point cloud data into a sea area target preprocessing convolutional network, respectively extracting sea surface ship image feature vectors and depth feature vectors, complementing the depth feature vectors through a conditional random field network, constructing a sea area depth weight matrix, optimizing the depth weight matrix through an attention mechanism, and generating dense depth features, and comprises the steps of inputting the depth feature vectors into the conditional random field network, calculating a unitary potential function of each node in a full connected graph, calculating a pair potential function between node pairs based on the image feature, generating a node weight matrix, carrying out normalization processing on the node weight matrix, constructing a sea area depth weight matrix, extracting feature patches from the depth weight matrix, calculating a dot product operation result of query vectors and key vectors, generating weight coefficients between patches, combining the weight coefficients with relative position coding information, optimizing the depth weight matrix based on the attention mechanism, and generating dense depth features; The sea area target ship identification module is used for inputting the image feature vector and the dense depth feature into a sea area target feature pyramid network, generating a multi-scale fusion feature, calculating the overlapping degree of a ship target boundary frame and a real boundary frame, generating a positioning loss value, calculating a cross entropy loss value of ship category and confidence, carrying out weighted combination on the positioning loss value and the cross entropy loss value, and determining the type and position information of a target ship.
7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method for identifying a target in a sea area of an unmanned aerial vehicle as claimed in any one of claims 1 to 5 when the program is executed by the processor.
8. A computer readable storage medium having stored thereon a computer program, characterized in that the computer program when executed by a processor realizes the steps of the unmanned aerial vehicle sea area target recognition method according to any one of claims 1 to 5.

Description

Unmanned aerial vehicle group sea area target identification method and device Technical Field The application relates to the field of data processing, in particular to a method and a device for identifying a sea area target of an unmanned aerial vehicle group. Background The existing unmanned aerial vehicle group sea area target identification method has obvious defects. The traditional system has poor performance in terms of data acquisition and time synchronization, and cannot effectively integrate multi-source sensor information, so that identification accuracy is affected. Furthermore, the prior art has bottlenecks in feature extraction and depth completion. Most systems lack sophisticated image processing mechanisms and depth optimization strategies, resulting in incomplete feature expression. Existing systems exist with short technology boards in terms of target detection. The lack of deep analysis of multi-scale features makes it difficult to achieve accurate target positioning through loss optimization, affecting detection effect. The solution of these problems is of great importance for improving the monitoring ability of the sea. Disclosure of Invention Aiming at the problems in the prior art, the application provides a method and a device for identifying the sea area targets of unmanned aerial vehicle groups, which can effectively solve the defects of the traditional technology in the aspects of data synchronization, feature extraction, target detection and the like and provide technical support for sea area monitoring. In order to solve at least one of the problems, the application provides the following technical scheme: in a first aspect, the present application provides a method for identifying a target in a sea area of an unmanned aerial vehicle, including: Deploying an unmanned aerial vehicle group to execute a sea area reconnaissance task, collecting image data acquired by a visible light camera and point cloud data acquired by a laser radar, recording sampling moments of the visible light camera and the laser radar, establishing a time stamp based on an upper computer time reference, carrying out index matching on the image data and the point cloud data according to the latest moment, calculating an internal reference matrix of the visible light camera by a Zhang calibration method, selecting an image and point cloud common view point to calculate an external reference conversion matrix, and projecting the point cloud data to a coordinate system of the image data based on the internal reference matrix and the external reference conversion matrix; inputting the image data and the point cloud data into a sea area target preprocessing convolutional network, respectively extracting sea surface ship image feature vectors and depth feature vectors, complementing the depth feature vectors through a conditional random field network, constructing a sea area depth weight matrix, and optimizing the depth weight matrix through an attention mechanism to generate dense depth features; Inputting the image feature vector and the dense depth feature into a sea area target feature pyramid network, generating a multi-scale fusion feature, calculating the overlapping degree of a ship target boundary frame and a real boundary frame, generating a positioning loss value, calculating a cross entropy loss value of ship category and confidence, carrying out weighted combination on the positioning loss value and the cross entropy loss value, and determining the type and position information of a target ship. Further, the method further comprises the steps of deploying the unmanned aerial vehicle group in a sea area target area, controlling a first unmanned aerial vehicle to carry out visible light camera acquisition, controlling a second unmanned aerial vehicle to carry out laser radar acquisition point cloud information, and generating acquisition parameters including acquisition positions, acquisition time and data scale; And recording sampling time information of the visible light camera and the laser radar, correcting the sampling time information by taking an upper computer time reference as a reference standard, calculating hardware transmission delay deviation, and generating a unified time stamp based on the upper computer. Further, the method further comprises the steps of sorting image data and point cloud data according to sampling time, calculating time intervals of adjacent data frames, selecting image and point cloud data pairs at the latest time based on the time intervals, generating a time sequence matching matrix, and establishing index association between the image data and the point cloud data in the time sequence matching matrix; calculating an internal reference matrix of the visible light camera based on a Zhang calibration method, selecting common-view feature points of an image and point clouds, constructing a three-dimensional coordinate mapping relation of the feature points, calculating