CN-121594854-B - Unmanned aerial vehicle inspection accurate positioning method and device integrating vision and satellite navigation

CN121594854BCN 121594854 BCN121594854 BCN 121594854BCN-121594854-B

Abstract

The invention provides an unmanned aerial vehicle inspection accurate positioning method and device integrating vision and satellite navigation, and relates to the technical field of aerospace information. The method and the system have the advantages that satellite signal quality characteristics reflecting the reliability of space-sky information and multidimensional characteristics such as visual geometric textures reflecting the characteristics of the ground environment are extracted in a combined mode, the environment state is deeply perceived, a fusion strategy adapting to the current environment is dynamically generated in real time, data weight and algorithm models are intelligently adjusted, the limitation of a traditional fixed parameter fusion mode is broken through, a specific inspection stage and a target object where an unmanned aerial vehicle is located are understood through visual identification, and a corresponding priori task constraint model is loaded, so that the positioning process closely meets the precision and safety requirements of actual operation. And finally, fusion and calculation are carried out under the common guidance of the dynamic strategy and the task constraint, the problem of inaccurate unmanned aerial vehicle positioning in a complex environment is solved, and the environmental adaptability, the robustness and the task guidance accuracy of the positioning system are improved.

Inventors

LU SAISAI
Wei Xiutao
WANG FAN

Assignees

东之乔科技有限公司

Dates

Publication Date: 20260512
Application Date: 20260128

Claims (8)

1. The unmanned aerial vehicle inspection accurate positioning method integrating vision and satellite navigation is characterized by comprising the steps of acquiring satellite navigation data, vision sensor data and inertial measurement data of an unmanned aerial vehicle; Extracting multi-dimensional features based on satellite navigation data, vision sensor data and inertial measurement data, wherein the multi-dimensional features comprise satellite signal quality features, vision image texture features, geometric structure features and instant motion state features; Based on the multidimensional characteristics, dynamically generating a fusion positioning strategy matched with the current environment, wherein the fusion positioning strategy comprises fusion weights among various data, a state estimation algorithm and an interference source error compensation model; Based on the vision sensor data, identifying a current inspection stage and a target inspection object of the unmanned aerial vehicle, and determining a priori task constraint model corresponding to the current inspection stage and the target inspection object; Based on the fusion positioning strategy, taking the prior task constraint model as optimization constraint, carrying out fusion calculation on the satellite navigation data and the vision sensor data, and generating real-time positioning information of the unmanned aerial vehicle; Based on the multi-dimensional characteristics, carrying out real-time judgment through a lightweight class network, and determining a current environment mode, environment confidence and a key influence factor vector; the current environment mode comprises an open mode, a half-shielding mode, a full-shielding mode or a dynamic interference mode; based on the current environment mode and the environment confidence, dynamically determining a state estimation algorithm, wherein GNSS-based Kalman filtering is adopted in an open mode, visual-based particle filtering and GNSS loose coupling assistance are started in a semi-shielding mode, CNN-LSTM-based deep learning positioning model is started in a full-shielding mode, an anti-interference special filtering algorithm is started in a dynamic interference mode, the anti-interference special filtering algorithm introduces an adaptive robust estimation based on an information sequence in a standard Kalman filtering framework to inhibit abnormal values generated by multipath or electromagnetic interference in GNSS data, key parameters of the state estimation algorithm are automatically adjusted based on the multi-dimensional characteristics, the key parameters comprise a Kalman filtering process noise covariance Q and an observation noise covariance R, particle number and a resampling threshold value of the particle filtering, attention weight of the deep learning model, the type of an interference source comprises GNSS multipath effect, strong electromagnetic field interference, visual image motion blur or illumination mutation based on the current dominant interference source type, the electromagnetic error compensation error is carried out in a pre-dominant interference source type, and the electromagnetic error compensation model is established from a pre-dominant error compensation model comprises an error compensation error library, and the anti-electromagnetic error compensation model is obtained from the pre-dominant error model is obtained, the method comprises the steps of image deblurring and illumination invariance enhancement models, initializing key parameters for a compensation algorithm based on satellite signal quality characteristics in the multidimensional characteristics, and determining an interference source error compensation model based on the compensation algorithm and the key parameters.
2. The unmanned aerial vehicle inspection accurate positioning method integrating vision and satellite navigation according to claim 1, wherein the extracting multi-dimensional features based on satellite navigation data, vision sensor data and inertial measurement data comprises: Analyzing satellite navigation data, and calculating the number of visible satellites, the signal-to-noise ratio of each satellite, the carrier phase continuity index and the satellite space geometric distribution at the current moment; calculating a position accuracy factor based on the satellite space geometrical distribution; Generating satellite signal quality characteristics representing signal reliability based on the number of visible satellites at the current moment, the signal-to-noise ratio of each satellite, the carrier phase continuity index, the satellite space geometric distribution and the position precision factor; based on the vision sensor data, extracting key points and local descriptors in each frame of image through a convolutional neural network, and counting the number and the spatial distribution uniformity to obtain vision image texture characteristics; Based on the vision sensor data, identifying a remarkable linear structure and corner points in each frame of image through an edge detection and line segment detection algorithm, matching with a known model, deducing a plane, corner angles and geometric outlines, and generating geometric structure features; Based on inertial measurement data of the unmanned aerial vehicle, three-dimensional acceleration and three-dimensional angular velocity of the unmanned aerial vehicle under a machine body coordinate system are obtained through calculation, and a filtering algorithm is adopted to estimate the current instantaneous linear velocity and angular velocity, so that instantaneous motion state characteristics are generated; time alignment is carried out on satellite signal quality features, visual image texture features, geometric structure features and instant motion state features by utilizing a space-time synchronization mechanism, so as to obtain time alignment features; based on the time alignment feature, combining with the pose initial value of the unmanned aerial vehicle, mapping the vision and geometric features to a global reference coordinate system, and generating a structured multidimensional feature.
3. The unmanned aerial vehicle inspection accurate positioning method integrating vision and satellite navigation according to claim 1, wherein the identifying the unmanned aerial vehicle current inspection stage and the target inspection object based on the vision sensor data comprises: Carrying out semantic segmentation and instance segmentation on each frame of image in the vision sensor data, extracting to obtain scene categories and preliminarily detecting potential inspection targets, wherein the scene categories comprise farmlands, tower groups or ruins, and the potential inspection targets comprise insulators, cracks or crops; based on the vision sensor data, judging a current inspection stage of the unmanned aerial vehicle through a time sequence convolution network, wherein the current inspection stage comprises approaching, encircling, linear cruising or fixed-point detailed inspection; Based on the scene category and the inspection stage, establishing a space, function and task logic relation among potential inspection targets by using a graph attention network, and deducing the target inspection targets; And carrying out multi-source verification based on the current inspection stage and the target inspection object by combining a satellite positioning point type, an IMU motion mode and a task plan, and determining the current inspection stage and the target inspection object with the same verification.
4. The unmanned aerial vehicle inspection accurate positioning method integrating vision and satellite navigation according to claim 1, wherein the determining a priori task constraint model corresponding to a current inspection stage and a target inspection object comprises: based on the current inspection stage and the target inspection object, searching a matched constraint rule set from a pre-constructed task-scene-constraint knowledge graph, wherein the constraint rule set comprises a positioning precision upper limit, a track smoothness requirement, a safety obstacle avoidance distance, an observation point residence time and a data acquisition integrity index; The constraint rule set is subjected to parameter dynamic adjustment by combining the multidimensional feature to obtain an adjusted constraint rule set, wherein the upper limit of the positioning accuracy is set in a grading manner according to the key degree of the target object, and the track smoothness coefficient is optimized on line according to the current wind speed and the flying speed; Based on the adjusted constraint rule set, consistency check is carried out, if constraint conflict exists, weight redistribution is automatically carried out according to the predefined task priority and the safety criterion, and a feasible constraint set is generated; And based on the feasible constraint set and a preset data model, fusing the feasible constraint set and a state estimation process to obtain the prior task constraint model, wherein the data model comprises an inequality constraint group, a Lyapunov function or a factor graph node.
5. The method for accurately positioning the unmanned aerial vehicle by integrating vision and satellite navigation according to claim 1, wherein the real-time positioning information of the unmanned aerial vehicle is generated by integrating the satellite navigation data and the vision sensor data by taking the prior task constraint model as an optimization constraint based on the integrated positioning strategy, and comprises the following steps: according to the fusion positioning strategy, configuring and starting a corresponding data fusion calculation pipeline to finish time stamp alignment and preprocessing of multi-source data to obtain fusion data; Based on the fusion data, carrying out optimization iteration by combining a state estimation algorithm, and injecting the prior task constraint model as optimization constraint to solve the optimal state estimation of the unmanned aerial vehicle meeting the constraint; Based on the optimal state estimation of the unmanned aerial vehicle, the credibility of each sensor and the fusion result is estimated; and carrying out local repositioning and track correction according to the credibility until the credibility reaches a credibility threshold value, and obtaining real-time positioning information of the unmanned aerial vehicle.
6. The unmanned aerial vehicle inspection accurate positioning method integrating vision and satellite navigation according to claim 1, wherein the method further comprises: In the initial stage of take-off, a visual feature map is constructed and is bound with the absolute coordinates of satellites to serve as a global positioning reference; In the cruising stage, a tight coupling fusion mechanism is adopted, satellite navigation data and inertial measurement data are used as absolute references, and vision sensor data are used as relative motion constraints to position the unmanned aerial vehicle; when the satellite signal is weakened or interrupted, switching to a vision leading mode, and maintaining the unmanned aerial vehicle positioning continuity through key frame repositioning; And at the task ending stage, starting a back-end graph optimization algorithm, and fusing global observation data to correct track errors so as to realize full-flow self-adaptive positioning.
7. The unmanned aerial vehicle inspection accurate positioning method integrating vision and satellite navigation according to claim 1, wherein the method further comprises: when a plurality of unmanned aerial vehicles execute a collaborative inspection task, each unmanned aerial vehicle is independently positioned, and positioning information of each unmanned aerial vehicle is obtained; constructing a co-location constraint of the unmanned aerial vehicle group based on the satellite time reference and the relative position relation among the unmanned aerial vehicles; and based on the unmanned aerial vehicle group co-location constraint, performing cross verification and joint optimization on the location information of each unmanned aerial vehicle, and determining the location information after verification of each unmanned aerial vehicle.
8. Unmanned aerial vehicle of fusion vision and satellite navigation patrols and examines accurate positioner, a serial communication port, include: the communication module is used for acquiring satellite navigation data, vision sensor data and inertial measurement data of the unmanned aerial vehicle; The system comprises a processing module, a real-time positioning module, a processing module and a control module, wherein the processing module is used for extracting multi-dimensional characteristics based on satellite navigation data, vision sensor data and inertia measurement data, the multi-dimensional characteristics comprise satellite signal quality characteristics, vision image texture characteristics, geometric structure characteristics and instant motion state characteristics, dynamically generating a fusion positioning strategy matched with the current environment based on the multi-dimensional characteristics, the fusion positioning strategy comprises fusion weights among various data, a state estimation algorithm and an interference source error compensation model, identifying a current inspection stage and a target inspection object of the unmanned aerial vehicle based on the vision sensor data, determining an priori task constraint model corresponding to the current inspection stage and the target inspection object, and carrying out fusion calculation on the satellite navigation data and the vision sensor data based on the fusion positioning strategy by taking the priori task constraint model as optimization constraint to generate real-time positioning information of the unmanned aerial vehicle; The processing module is specifically used for determining a current environment mode, environment confidence and a key influence factor vector through real-time judgment through a lightweight class network based on the multidimensional characteristics; the current environment mode comprises an open mode, a half-shielding mode, a full-shielding mode or a dynamic interference mode; based on the current environment mode and the environment confidence, dynamically determining a state estimation algorithm, wherein GNSS-based Kalman filtering is adopted in an open mode, visual-based particle filtering and GNSS loose coupling assistance are started in a semi-shielding mode, CNN-LSTM-based deep learning positioning model is started in a full-shielding mode, an anti-interference special filtering algorithm is started in a dynamic interference mode, the anti-interference special filtering algorithm introduces an adaptive robust estimation based on an information sequence in a standard Kalman filtering framework to inhibit abnormal values generated by multipath or electromagnetic interference in GNSS data, key parameters of the state estimation algorithm are automatically adjusted based on the multi-dimensional characteristics, the key parameters comprise a Kalman filtering process noise covariance Q and an observation noise covariance R, particle number and a resampling threshold value of the particle filtering, attention weight of the deep learning model, the type of an interference source comprises GNSS multipath effect, strong electromagnetic field interference, visual image motion blur or illumination mutation based on the current dominant interference source type, the electromagnetic error compensation error is carried out in a pre-dominant interference source type, and the electromagnetic error compensation model is established from a pre-dominant error compensation model comprises an error compensation error library, and the anti-electromagnetic error compensation model is obtained from the pre-dominant error model is obtained, the method comprises the steps of image deblurring and illumination invariance enhancement models, initializing key parameters for a compensation algorithm based on satellite signal quality characteristics in the multidimensional characteristics, and determining an interference source error compensation model based on the compensation algorithm and the key parameters.

Description

Unmanned aerial vehicle inspection accurate positioning method and device integrating vision and satellite navigation Technical Field The invention relates to the technical field of aerospace information, in particular to an unmanned aerial vehicle inspection accurate positioning method and device integrating vision and satellite navigation. Background With rapid development and wide application of aerospace information technology, an aerospace information system with satellite navigation, remote sensing and communication as cores has become an important space infrastructure in modern society. In the field of unmanned aerial vehicle inspection, space information, in particular to a global navigation system (GNSS), provides an indispensable global coverage and all-weather absolute positioning reference for the unmanned aerial vehicle, and is a key technical support for realizing wide-area operation and space coordinate unification. However, positioning technology that relies on a single aerospace information source presents a significant last mile bottleneck in facing complex terrestrial application scenarios. On the one hand, GNSS signals in the aerospace information are easy to be interfered by complex electromagnetic and physical environments such as urban high-rise building groups, dense forests, high-voltage transmission corridors and the like, so that signal attenuation, multipath reflection and even complete shielding are caused, positioning accuracy is deteriorated from a meter level to a ten-meter level and even is invalid, and severe requirements of fine inspection, accurate agricultural monitoring and the like of electric power facilities on the positioning accuracy of the meter level and the centimeter level are difficult to meet. On the other hand, most of the current integration modes of the space-sky information and the ground sensing information still stay at a relatively primary static combination level, and a loose coupling or parameter solidification-unchanged filtering algorithm is generally adopted, so that the information reflecting the environmental quality, such as signal intensity, carrier-to-noise ratio, visible satellite space distribution and the like, carried by the space-sky information cannot be fully analyzed and utilized. Once the aerospace information is interfered or interrupted, the overall robustness of the system is obviously reduced, and the positioning accuracy and the operation continuity of the unmanned aerial vehicle in a complex environment are seriously affected. Disclosure of Invention The invention provides an unmanned aerial vehicle inspection accurate positioning method and device integrating vision and satellite navigation, and solves the technical problem of inaccurate unmanned aerial vehicle positioning in a complex environment. The invention provides an unmanned aerial vehicle routing inspection accurate positioning method combining vision and satellite navigation, which comprises the steps of obtaining satellite navigation data, vision sensor data and inertial measurement data of an unmanned aerial vehicle, extracting multi-dimensional features based on the satellite navigation data, the vision sensor data and the inertial measurement data, wherein the multi-dimensional features comprise satellite signal quality features, vision image texture features, geometric structure features and instant motion state features, dynamically generating a fusion positioning strategy matched with the current environment based on the multi-dimensional features, wherein the fusion positioning strategy comprises fusion weights among various data, a state estimation algorithm and an interference source error compensation model, identifying a current routing inspection stage and a target routing inspection object of the unmanned aerial vehicle based on the vision sensor data, determining an priori task constraint model corresponding to the current routing inspection stage and the target routing inspection object, and performing fusion calculation on the satellite navigation data and the vision sensor data based on the fusion positioning strategy by taking the priori task constraint model as optimization constraint to generate real-time positioning information of the unmanned aerial vehicle. The embodiment of the invention provides an unmanned aerial vehicle routing inspection accurate positioning device integrating vision and satellite navigation, which comprises a communication module, a processing module and a real-time positioning information generating module, wherein the communication module is used for acquiring satellite navigation data, vision sensor data and inertial measurement data of an unmanned aerial vehicle, the processing module is used for extracting multi-dimensional features based on the satellite navigation data, the vision sensor data and the inertial measurement data, the multi-dimensional features comprise satellite signal quality features, vision image