CN-121995938-A - Unmanned aerial vehicle track tracking control method and system based on visual inertial navigation fusion

CN121995938ACN 121995938 ACN121995938 ACN 121995938ACN-121995938-A

Abstract

The invention discloses a visual inertial navigation fusion unmanned aerial vehicle track tracking control method and a visual inertial navigation fusion unmanned aerial vehicle track tracking control system, which relate to the technical field of unmanned aerial vehicle navigation and control, wherein the method comprises the steps of acquiring multi-mode sensing data and performing time stamp alignment; the method comprises the steps of constructing a visual odometer network to extract relative pose variation, constructing an inertial integration network to extract short-time motion estimation, constructing an adaptive confidence fusion module to dynamically adjust visual modal weights according to the richness of environmental textures, dynamically adjust inertial navigation modal weights according to the maneuver intensity, realizing adaptive weighted fusion to output high-precision pose estimation through a learning attention mechanism, and inputting the fused pose estimation into a nonlinear model prediction controller to output expected pose angles and thrust instructions to realize accurate track tracking.

Inventors

Qiu Zhengtong
MU CHAONAN
CHEN AO
YUAN HAOYU

Assignees

海南大学

Dates

Publication Date: 20260508
Application Date: 20260210

Claims (10)

1. The unmanned aerial vehicle track tracking control method based on visual inertial navigation fusion is characterized by comprising the following steps of: S1, synchronously acquiring multi-mode sensing data in the flight process of an unmanned aerial vehicle, wherein the multi-mode sensing data comprise an environment image sequence acquired by a forward binocular camera, acceleration data and angular velocity data acquired by an inertial measurement unit, and aligning the environment image sequence with the acceleration and angular velocity data according to a time stamp to obtain synchronous sensing data flow for fusion positioning; s2, bimodal feature extraction, namely, constructing a visual odometer network to extract relative pose variation between continuous frames according to an environment image sequence in the synchronous perception data stream, and constructing an inertial integration network to extract short-time motion estimation between adjacent moments according to acceleration and angular velocity data in the synchronous perception data stream; S3, self-adaptive confidence coefficient fusion is constructed, namely a self-adaptive confidence coefficient fusion module is constructed, the module calculates the richness of the environmental texture according to the environmental image sequence, dynamically adjusts the weight of the visual mode according to the richness of the environmental texture, calculates the dynamic intensity according to the data of acceleration and angular velocity, dynamically adjusts the weight of the inertial navigation mode according to the dynamic intensity, normalizes the weight of the visual mode and the weight of the inertial navigation mode through a learning attention mechanism, realizes self-adaptive weighted fusion of the relative pose change quantity and the short-time motion estimation quantity, and outputs high-precision pose estimation; And S4, model predictive track control, namely taking the high-precision pose estimation as state feedback and inputting the state feedback into a nonlinear model predictive controller, solving an optimal control sequence in a predictive time domain by the nonlinear model predictive controller according to the deviation between the expected track and the high-precision pose estimation, and outputting an expected attitude angle and a thrust command to an unmanned plane flight control system to realize accurate tracking control on the expected track.
2. The unmanned aerial vehicle track tracking control method of visual inertial navigation fusion according to claim 1, wherein in step S1, the time stamp alignment process comprises performing linear interpolation alignment on exposure time of the environmental image sequence with reference to sampling time of the inertial measurement unit, so that time deviation between the visual frame and the inertial navigation measurement is not more than 1ms.
3. The unmanned aerial vehicle trajectory tracking control method of visual inertial navigation fusion according to claim 1, wherein in step S2, the visual odometer network adopts an encoder-decoder structure, the encoder part adopts a residual convolution network to extract multi-scale visual features, and the decoder part adopts full-connection layer regression to output six-degree-of-freedom relative pose variation.
4. The method for tracking and controlling the trajectory of the unmanned aerial vehicle by fusion of visual inertial navigation according to claim 1, wherein in the step S2, the inertial integration network performs integration processing on acceleration data and angular velocity data between adjacent key frames by using an IMU pre-integration method, and outputs a short-time motion estimator comprising relative rotation, relative translation and relative velocity.
5. The unmanned aerial vehicle track tracking control method based on visual inertial navigation fusion according to claim 1, wherein in the step S3, the calculation method of the environmental texture richness is characterized in that angular point features are extracted from an environmental image, the number and distribution uniformity of the angular point features are counted, when the number of the angular points is larger than a set threshold value and the distribution uniformity is higher than a set proportion, the environmental texture richness is judged to be high, the visual mode weight is correspondingly increased, and when the number of the angular points is smaller than the set threshold value or the distribution uniformity is lower than the set proportion, the environmental texture richness is judged to be low, and the visual mode weight is correspondingly reduced.
6. The unmanned aerial vehicle track tracking control method based on visual inertial navigation fusion according to claim 1 is characterized in that in the step S3, the calculation method of the maneuver intensity is that standard deviation of acceleration data and angular velocity data in a sliding time window is calculated, when the standard deviation is in a set stable interval, the maneuver intensity is judged to be low, the inertial integration accumulated error is small, the inertial navigation modal weight is correspondingly increased, and when the standard deviation exceeds the stable interval, the maneuver intensity is judged to be high, the inertial integration accumulated error is large, and the inertial navigation modal weight is correspondingly reduced.
7. The unmanned aerial vehicle trajectory tracking control method based on visual inertial navigation fusion according to claim 1, wherein in step S3, the learnable attention mechanism comprises mapping the visual modality weight and the inertial navigation modality weight into a query vector and a key vector respectively, calculating a correlation score between the two modalities through dot product attention, carrying out Softmax normalization on the correlation score to obtain a final fusion weight, and carrying out weighted summation on a relative pose change amount and a short-time motion estimation amount by the final fusion weight to obtain high-precision pose estimation.
8. The unmanned aerial vehicle trajectory tracking control method of visual inertial navigation fusion according to claim 1, wherein in step S4, the prediction time domain of the nonlinear model prediction controller is 10 to 30 sampling periods, the control time domain is 5 to 15 sampling periods, and the sampling period is 10ms to 50ms.
9. The method according to claim 1, wherein in step S4, the cost function of the nonlinear model predictive controller includes a position tracking error term, an attitude tracking error term, and a control amount change rate penalty term, wherein the weight of the position tracking error term is greater than the weight of the attitude tracking error term, and the control amount change rate penalty term is used to suppress a drastic change of the control command.
10. The unmanned aerial vehicle track tracking control system for realizing the visual inertial navigation fusion according to any one of claims 1 to 9, which is characterized by comprising: the multi-mode data acquisition module is configured to acquire multi-mode sensing data in the flight process of the unmanned aerial vehicle, perform time stamp alignment processing, and output synchronous sensing data flow, wherein the multi-mode sensing data comprises an environment image sequence acquired by a forward binocular camera, acceleration data and angular velocity data acquired by an inertial measurement unit; The system comprises a bimodal feature extraction module, a synchronous sensing data stream and a motion estimation module, wherein the bimodal feature extraction module is configured to extract visual mode features and inertial navigation mode features from the synchronous sensing data stream respectively, and comprises a visual odometer network and an inertial integration network, the visual odometer network is used for extracting relative pose variation between continuous frames, and the inertial integration network is used for extracting short-time motion estimation between adjacent moments; the self-adaptive confidence coefficient fusion module is configured to dynamically adjust the visual mode weight according to the richness of the environmental texture, dynamically adjust the inertial navigation mode weight according to the maneuver intensity, realize self-adaptive weighted fusion of the relative pose change quantity and the short-time motion estimation quantity through a learning attention mechanism and output high-precision pose estimation; The model prediction control module is configured to receive the high-precision pose estimation as state feedback, solve the optimal control sequence according to the expected track, output the expected attitude angle and the thrust command, and realize accurate tracking control on the expected track by adopting a nonlinear model prediction controller.

Description

Unmanned aerial vehicle track tracking control method and system based on visual inertial navigation fusion Technical Field The invention relates to the technical field of unmanned aerial vehicle navigation and control, in particular to a visual inertial navigation fusion unmanned aerial vehicle track tracking control method and system. Background Unmanned aerial vehicle track tracking control is one of the core technologies that realizes unmanned aerial vehicle autonomous flight, and its performance has directly decided unmanned aerial vehicle's application effect in fields such as taking photo by plane survey, logistics distribution, agricultural plant protection and emergency rescue. Precise trajectory tracking control relies on highly accurate state estimation, and the accuracy of the state estimation in turn depends on reliable fusion of the sensor data. At present, a visual sensor and an inertial measurement unit are two types of sensors most commonly used in unmanned aerial vehicle positioning navigation, the visual sensor can provide abundant environmental information but is easily influenced by illumination change and texture deficiency, and the inertial measurement unit can provide high-frequency motion measurement but has a cumulative drift problem. How to effectively integrate the two complementary sensor information is a key point for improving the tracking control precision of the unmanned aerial vehicle track. The Chinese patent with the bulletin number of CN120406506A discloses a unmanned aerial vehicle dynamic target tracking method based on end-to-end learning, which collects multi-mode input data comprising visual images and IMU motion information, extracts visual and motion characteristics through a double-flow convolutional neural network, fuses the multi-mode characteristics by using a graph attention network, models the spatial relationship between an unmanned aerial vehicle and a target and environment, and generates an unmanned aerial vehicle tracking track based on a deep reinforcement learning strategy. The method still has the following defects that firstly, when the method adopts a graph attention network to perform characteristic fusion, the influence of environmental condition change on the reliability of different modes is not considered, when a vision sensor is degenerated under a scene of weak texture or intense illumination change, the positioning accuracy is difficult to ensure by a fixed fusion strategy, secondly, the method adopts deep reinforcement learning to generate a track, the training of the strategy network needs a large number of samples and the generalization capability is limited, the tracking effect of the complex maneuver track which is not found is difficult to ensure, and thirdly, the method does not establish a tight coupling relation between vision inertial navigation fusion positioning and track tracking control, and the transmission of positioning errors to a control link lacks an effective inhibition mechanism. In addition, the existing visual inertial navigation fusion method is mainly divided into a loose coupling framework and a tight coupling framework. The loose coupling method is characterized in that a visual odometer and an inertial navigation are used as independent systems to operate respectively, and output results of the visual odometer and the inertial navigation are fused through a filter. The tight coupling method solves the visual characteristic points and the inertia measured values in a unified optimization framework in a combined way, so that higher positioning precision can be obtained, but the calculation complexity is higher and the requirements on the sensor synchronization precision are strict. Whether a loose coupling method or a tight coupling method is adopted, the prior art generally adopts a fixed fusion weight or a covariance matrix-based self-adaptive weight adjustment strategy, and the direct influence of environmental conditions and motion states on the reliability of the sensor cannot be fully considered. When the unmanned aerial vehicle flies through a weak texture area or performs intense maneuver, the measurement quality of the visual sensor or the inertial sensor can be significantly reduced, and the fusion strategy of fixed weight can lead to the deterioration of positioning accuracy. In the track tracking control aspect, the prior art mainly adopts methods such as PID control, sliding mode control, model predictive control and the like. PID control has a simple structure, is difficult to process multivariable coupling and input and output constraints, has strong robustness but has a buffeting problem, and model predictive control can explicitly process the constraints and optimize multi-step performance indexes. However, the existing model prediction control method generally processes state estimation and controller design separately, fails to establish a direct correlation between positioning accuracy and cont