CN-122023527-A - Visual positioning method and system for vortex-spraying manned aircraft based on deep learning

CN122023527ACN 122023527 ACN122023527 ACN 122023527ACN-122023527-A

Abstract

The invention provides a visual positioning method and a visual positioning system for a vortex-spraying manned aircraft based on deep learning, and belongs to the field of visual positioning of aircrafts. The method aims to solve the problems that the vortex-jet aircraft vibrates in tens to hundreds of hertz and has strong maneuverability when the accelerator is accelerated and the jet is coupled, and the movement blurring of a visual camera and the time-varying of IMU noise are induced. The method comprises the steps of extracting dotted line characteristics from a left-eye visual image based on a neural network model of dot line joint characteristic detection, carrying out inter-frame dotted line characteristic matching, judging whether the image is a key frame, inputting a right-eye image if the image is the key frame, carrying out binocular characteristic stereo matching, carrying out vortex spraying manned aircraft pose resolving if the image is not the key frame, and then constructing a factor graph for optimization. The invention only adopts binocular images to estimate in key frames to ensure algorithm instantaneity for the vortex-spraying manned aircraft, and estimates based on dotted line characteristics to ensure high-speed robustness for the vortex-spraying manned aircraft.

Inventors

SU HANG
DONG HUI
WU DONGMEI
DONG WEI
GAO YONGZHUO

Assignees

哈尔滨工业大学

Dates

Publication Date: 20260512
Application Date: 20260130

Claims (9)

1. The visual positioning method of the vortex-spraying manned aircraft based on deep learning is characterized by comprising the following steps of: s100, inputting a left-hand visual image, and extracting dotted line features from the left-hand visual image based on a neural network model for dot line joint feature detection; the neural network model comprises a feature extraction backbone network, a point feature detection sub-network and a line segment detection sub-network, wherein intermediate feature representation is obtained through the feature extraction backbone network, a point of interest thermodynamic diagram and a feature tensor are obtained through the point feature detection sub-network, the point of interest thermodynamic diagram subjected to downsampling processing and the multi-scale feature diagram output by the feature extraction backbone network are spliced in a channel dimension through the line segment detection sub-network, and weights are trained in a mode of multi-stage training by marking a related data set of the vortex-spraying manned aircraft, so that the line segment detection module can explicitly perceive the distribution condition of angular points in an image under the vortex-spraying manned flight scene; S200, performing inter-frame point line characteristic matching based on Lightglue networks and a geometric method; And S300, judging whether the video is a key frame, if so, matching the right-eye image based on the time stamp, and performing binocular stereo matching, if not, performing pose resolving of the vortex spraying manned aircraft, and then constructing a factor graph for optimization.
2. The visual positioning method of the vortex-spraying manned aircraft based on deep learning of claim 1, wherein in step S100, the feature extraction backbone network adopts a Super Point self-supervision feature Point detection algorithm to encode an input H×W gray image sequentially through 8 convolution layers and 3 maximum pooling layers, a multi-scale feature map is output, and after downsampling with a step size of 8, an intermediate feature representation with a size of about (H/8) × (W/8) is obtained.
3. The method for visual positioning of a deep learning-based vortex-spray manned aircraft of claim 2, wherein in step S100, the point feature detection sub-network includes a scoring branch and a description sub-branch, The scoring branch takes the middle characteristic as input, outputs tensor with the size of (H/8) x (W/8) x 65, performs reshape and space rearrangement in a preset mode to obtain a interest point thermodynamic diagram with the size of H x W x 1, and is used for representing the confidence degree of each pixel position as a key point, and the description sub-branch takes the middle characteristic as input, outputs characteristic tensor with the size of (H/8) x (W/8) x D, wherein D is depth, and is used for interpolating at the key point position to obtain a corresponding local descriptor to realize the discriminant characterization of the key point.
4. The visual positioning method of the vortex-spray manned aircraft based on deep learning of claim 3, wherein in step S100, the line segment detection sub-network is formed by cascading a convolutional neural network in a U-Net symmetrical U-shaped network structure form with an EPD LOI alignment line segment detection head module, The EPD LOI Align line segment detection head module is used for executing alignment and aggregation operation based on candidate areas on the enhanced features, carrying out fine sampling and feature aggregation on the line candidate areas, outputting segment prediction results of segment continuity, and improving the geometric shape and end point position estimation precision of the line segments.
5. The method for visual localization of a deep learning-based vortex-spray manned aircraft of claim 4 wherein the training phase of the neural network model includes, Training a feature extraction backbone network and a Point feature detection sub-network on a large-scale non-labeling image data set according to a Super Point self-supervision feature Point detection algorithm, so that an interest Point thermodynamic diagram and a descriptor are converged stably under multiple scenes; The second stage, on a small-scale data set marked with a line segment true value, keeping parameters of a feature extraction backbone network and a point feature detection sub-network unchanged, and performing supervised training on the line segment detection sub-network; In the training of the segment detection, the loss function comprises the first-order region attention loss and BCE binary cross entropy loss of the segment Attraction-RegionL1, N marked real segments in a training sample are provided, and the parameter vector is recorded as The corresponding line segment parameter vector of the network prediction is that The weight coefficient is Attraction-Region L1 loss Expressed as: In the formula, Represents an L1 norm; setting M sampling positions or candidate regions in line segment existence judgment, wherein the true mark is The corresponding existence probability of the network output is Binary Cross Entropy (BCE) loss Expressed as: total loss function of line segment detection sub-network The method comprises the following steps: In the formula, Is a weight coefficient for balancing geometric regression accuracy with presence classification accuracy.
6. The method for visual positioning of a deep learning-based vortex-spray manned aircraft of claim 5, wherein in step S200, matching between feature points is achieved by Lightglue network, and then the matching is based on the distance between the points and line segments Matching between points and line segments, and calculating matched characteristic points Occupy the total point number on the line segment Ratio of (3) : When (when) When, say, they are matched, wherein And each frame is matched with the nearest key frame by the same matching principle.
7. The method for visual positioning of a deep learning-based vortex-spray manned aircraft of claim 6, wherein in step S300, when Judging the frame as a key frame; when average square parallax When the threshold is exceeded, judging the frame as a key frame: Wherein H, W denotes the image height and width, A normalized ratio threshold representing parallax; Or greater than a time threshold When no key frame is generated in the time length of (2), determining the frame as the key frame.
8. A deep learning-based vortex-spraying manned aircraft visual positioning system is characterized by comprising a program module corresponding to the steps of any one of claims 1-7, wherein the steps in the deep learning-based vortex-spraying manned aircraft visual positioning method are executed in operation.
9. A computer readable storage medium, characterized in that it stores a computer program configured to implement the steps of a deep learning based vortex spraying manned aircraft visual localization method according to any one of claims 1 to 7 when called by a processor.

Description

Visual positioning method and system for vortex-spraying manned aircraft based on deep learning Technical Field The invention relates to the technical field of visual positioning of aircrafts, in particular to a visual positioning method and system of a vortex-spraying manned aircraft based on deep learning. Background High-precision positioning and navigation of an aircraft are key basic technologies for realizing autonomous flight, safety control and subsequent environment modeling and task execution. Currently, in the field of unmanned aerial vehicles and low-altitude flight platforms, the commonly used positioning methods mainly include a synchronous positioning and mapping method (LIDAR SLAM) based on a laser radar and a synchronous positioning and mapping method (Visual SLAM) based on a Visual sensor. The laser SLAM has the advantages of insensitivity to illumination change, high ranging precision and the like, but the perception result mainly depends on geometric information, lacks of environment texture and semantic features, and is unfavorable for subsequent three-dimensional reconstruction, target recognition, fine environment understanding and other applications. Meanwhile, the laser radar equipment has the problems of higher cost, larger volume and larger power consumption, and limited application on platforms with strict requirements on load and energy consumption, such as manned aircrafts. Therefore, the visual SLAM method which fuses the environment texture information and has stronger scene expression capability is gradually becoming an important direction of research and engineering application. Most of the existing visual SLAM methods are based on traditional geometric modeling and feature matching frames or combined with Inertial Measurement Units (IMU) to perform visual inertial fusion positioning, so that certain application effects are achieved on platforms such as multi-rotor unmanned aerial vehicles, ground robots and the like. However, such methods generally assume that the carrier is steady in motion, clear in imaging and controllable in vibration level, and the algorithm design is difficult to adapt to the high-speed motion, the large-scale attitude change and the high-frequency vibration characteristics caused by the thrust fluctuation and jet coupling of the turbojet engine, which are shown by the turbojet manned aircraft in actual operation. Under the working conditions, the vision sensor is easy to generate serious motion blur and low texture degradation, IMU noise presents obvious time-varying characteristics, and the problems of tracking failure, positioning drift and even system failure of the existing vision SLAM or vision inertial positioning method are easy to occur. Therefore, the existing positioning method based on the laser SLAM or the traditional visual SLAM is difficult to meet the requirements of the vortex-spraying manned aircraft on high-robustness and high-real-time visual positioning under the condition that GNSS (global navigation satellite system ) is limited or refused to be in the environment. Aiming at the problems, it is necessary to design a visual positioning method and a visual positioning system for the operation characteristics of the vortex spraying manned aircraft, and introduce advanced perception technologies such as deep learning to enhance the adaptability to motion blur, vibration interference and low-texture scenes, so as to realize stable and reliable visual positioning capability with engineering practical value. Disclosure of Invention The invention aims to solve the technical problems that: The method aims at solving the problems that the existing positioning method is difficult to adapt to high-speed motion and large-scale attitude change of the vortex spraying manned aircraft in actual operation and high-frequency vibration characteristics caused by thrust fluctuation and jet coupling of the vortex spraying engine. The invention adopts the technical scheme for solving the technical problems: the invention provides a visual positioning method of a vortex-spraying manned aircraft based on deep learning, which comprises the following steps: s100, inputting a left-hand visual image, and extracting dotted line features from the left-hand visual image based on a neural network model for dot line joint feature detection; the neural network model comprises a feature extraction backbone network, a point feature detection sub-network and a line segment detection sub-network, wherein intermediate feature representation is obtained through the feature extraction backbone network, a point of interest thermodynamic diagram and a feature tensor are obtained through the point feature detection sub-network, the point of interest thermodynamic diagram subjected to downsampling processing and the multi-scale feature diagram output by the feature extraction backbone network are spliced in a channel dimension through the line segment detection sub-network,