CN-122018381-A - Unmanned aerial vehicle lifting control method and system based on industrial vision

Abstract

The invention relates to the technical field of unmanned aerial vehicle lifting and control, in particular to an unmanned aerial vehicle lifting control method based on industrial vision, which comprises the following steps that S1, multi-angle visual image data of a load and real-time tension information of each lifting rope are collected when an unmanned aerial vehicle is in lifting operation; S2, reconstructing three-dimensional point cloud according to the data, extracting characteristic points to calculate a visual gravity center, determining the mechanical gravity center of the lifting rope according to the stress analysis of the lifting rope, and dynamically adjusting the visual gravity center based on the coordinate difference and the stress balance relationship of the two to obtain the comprehensive gravity center position and the speed state of the load. According to the invention, through comprehensive gravity center modeling and self-adaptive control based on industrial vision and mechanical fusion, dynamic coordination and stable control of the load gesture and the unmanned aerial vehicle gesture are realized, so that the problem that the load is easy to swing and unstably due to the lack of a gravity center dynamic correction mechanism in a mode that the traditional unmanned aerial vehicle lifting method mostly depends on single vision identification is solved.

Inventors

• CHEN SHIYONG
• Shen Wanke
• CHEN LINGFENG
• LI DONGYUE
• Cai Yuezhen
• LI JUN
• FAN XINGKAI
• LI YOUMING
• FANG BAILIN
• YIN ZHIQIANG
• GAO KUNZHI
• Deng Baichuan
• CHEN JINGCHUAN
• ZHANG JIAWEI
• CHEN YIJIA
• WU ZHENDONG
• CHEN XIONGWEI

Assignees

• 南方电网通用航空服务有限公司

Dates

Publication Date

20260512

Application Date

20251106

Claims (10)

1. 1. An unmanned aerial vehicle lifting control method based on industrial vision is characterized by comprising the following steps: S1, collecting multi-angle visual image data of a load and real-time tension information of each lifting rope when an unmanned aerial vehicle is in lifting operation; s2, reconstructing three-dimensional point cloud according to the data, extracting characteristic points to calculate a visual gravity center, determining a mechanical gravity center of the lifting rope according to the stress analysis of the lifting rope, and dynamically adjusting the visual gravity center based on the coordinate difference and the stress balance relationship of the two to obtain the comprehensive gravity center position and the speed state of the load; s3, establishing a nonlinear dynamics model of the unmanned aerial vehicle, the lifting rope and the load based on the obtained comprehensive gravity center state, wherein the model considers the elasticity of the lifting rope, the inertial coupling of the load and external disturbance and is used for describing the dynamic response characteristic of a lifting system of the unmanned aerial vehicle; s4, generating an unmanned aerial vehicle gesture and hoisting control instruction by adopting a self-adaptive control algorithm according to the dynamics model, and realizing dynamic correction and stable control of the load gravity center by preferentially adjusting a position error in a hovering state and preferentially adjusting a speed error in a moving state; S5, the unmanned aerial vehicle executes a control instruction and simultaneously monitors the visual gravity center and the change of the lifting rope tension in real time, and the monitoring result is fed back to the control module to form closed-loop control so as to inhibit load swing and keep balance; And S6, when the monitoring result shows that the load gesture or the lifting rope tension exceeds a threshold value, the system triggers a safety mechanism to automatically slow down, hover or emergency stop until the load is restored to be stable or the lifting task is completed.
2. 2. The unmanned aerial vehicle lifting control method based on industrial vision according to claim 1, wherein the image data and the tension information comprise: The unmanned aerial vehicle acquires multi-angle images of the load through a plurality of industrial cameras before lifting; Inputting each image into a three-dimensional point cloud reconstruction module, generating a point cloud model of a load by adopting a reconstruction algorithm based on stereo matching, registering the point cloud, extracting features, and calculating a geometric center to obtain a visual center of gravity; The stress value of each lifting rope is obtained through the lifting rope tension sensor, a moment balance equation is established by combining the coordinates of the lifting points, and the gravity center of mechanics is calculated; and dynamically adjusting the visual center of gravity based on the coordinate difference and mechanical constraint of the visual center of gravity and the mechanical center of gravity to obtain the comprehensive center of gravity position and speed state of the load, and inputting the comprehensive center of gravity state into the control module.
3. 3. The unmanned aerial vehicle lifting control method based on industrial vision according to claim 1, wherein the visual center of gravity and the mechanical center of gravity comprise: establishing a nonlinear dynamics model of the unmanned plane, the lifting rope and the load based on a comprehensive gravity center state obtained through dynamic adjustment of the visual gravity center and the mechanical gravity center; Wherein the hoist rope is modeled as a flexible rod member having elastic and damping characteristics; Deducing the relation between kinetic energy and potential energy of the system by adopting a Lagrangian equation, and establishing a kinetic equation comprising unmanned plane thrust, lifting rope tension, load inertia and external disturbance; And a mechanical model parameter self-adaptive updating mechanism is introduced for automatically correcting model coefficients when the load quality or the lifting point is changed.
4. 4. The unmanned aerial vehicle lifting control method based on industrial vision according to claim 1, wherein the nonlinear dynamics model comprises: the control module is provided with an upper load gravity center adjusting control unit and a lower attitude control unit; the upper control unit generates a load adjusting instruction according to the comprehensive gravity center deviation, and the lower control unit generates a posture and thrust control instruction; Increasing the position error weight in a hovering state and increasing the speed error weight in a moving state; And updating the control gain in real time by adopting an adaptive algorithm, and outputting a composite control instruction.
5. 5. The unmanned aerial vehicle lifting control method based on industrial vision according to claim 1, wherein the unmanned aerial vehicle attitude and lifting control instruction comprises: Continuously acquiring visual gravity center and lifting rope tension data in the process of executing a control instruction by the unmanned aerial vehicle; The control module compares the target state with the current state, calculates deviation and automatically adjusts a feedback gain factor; When the swing amplitude change is detected, the thrust response rate is adjusted, and the control gain is reduced when the gesture is stabilized; And periodically update system parameters to maintain control accuracy.
6. 6. The unmanned aerial vehicle lifting control method based on industrial vision according to claim 1, wherein the visual gravity center and lifting rope tension change comprises: setting a multi-level safety threshold in the system, wherein the multi-level safety threshold comprises a lifting rope stress threshold, a load attitude angle threshold and an unmanned plane attitude threshold; When the detection value reaches the warning threshold value, the control module reduces the flying speed and adjusts the gesture; When the detection value exceeds the early warning threshold value, the control module is switched to a hovering mode; when the detection value exceeds the limit threshold value, an emergency stop instruction is executed and a lifting rope buffering or locking device is triggered; and records the anomaly information in the task log.
7. 7. The unmanned aerial vehicle lifting control method based on industrial vision according to claim 1, wherein the safety mechanism comprises: after the task is finished, the system extracts attitude data, gravity center change data and lifting rope stress data in the lifting process; Analyzing the control deviation and updating the control parameters; Loading updated parameters in the subsequent tasks and performing parameter convergence operation; forming a control parameter library for task classification; and writing new parameters into the control module after the task is executed.
8. 8. The unmanned aerial vehicle lifting control method based on industrial vision according to claim 1, wherein the comprehensive gravity center of S2 and the nonlinear dynamics model of S3 comprise: Based on the comprehensive gravity center state and the nonlinear dynamics model, the vision system carried by the unmanned aerial vehicle and the laser radar synchronously acquire surrounding environment information of the lifting path; dividing the acquired data and detecting obstacles to generate an environment three-dimensional model; calculating wind direction, wind speed and turbulence intensity by combining meteorological sensor data; the path planning module recalculates the flight path according to the environment model and the dynamic load state; the control module adjusts the flight altitude, speed and attitude parameters according to the updated path, and inputs the adjustment result into the closed-loop control module for execution.
9. 9. The unmanned aerial vehicle lifting control method based on industrial vision according to claim 1, wherein the control instruction of S4 and the monitoring data of S5 comprise: When a plurality of unmanned aerial vehicles participate in lifting, a synchronous communication mechanism is established through a main control node; The main control node distributes a tension share and a space position target to each unmanned aerial vehicle; Each unmanned aerial vehicle shares comprehensive gravity center and attitude data; The control algorithm updates the thrust distribution matrix according to the stress variation; when any one of the unmanned aerial vehicles deviates, the remaining unmanned aerial vehicles perform compensation control to maintain load balance.
10. 10. An unmanned aerial vehicle lifting control system based on industrial vision, which is characterized by being applied to the unmanned aerial vehicle lifting control method based on industrial vision as claimed in any one of claims 1-9, and comprising the following modules: the data acquisition module is used for acquiring multi-angle visual image data of the load and real-time tension information of each lifting rope when the unmanned aerial vehicle is in lifting operation; The gravity center fusion module is used for carrying out three-dimensional point cloud reconstruction according to the data and extracting characteristic points to calculate a visual gravity center, determining the mechanical gravity center of the lifting rope according to the stress analysis of the lifting rope, and dynamically adjusting the visual gravity center based on the coordinate difference and the stress balance relation of the two to obtain the comprehensive gravity center position and the speed state of the load; The dynamic modeling module is used for establishing a nonlinear dynamic model of the unmanned aerial vehicle, the lifting rope and the load based on the obtained comprehensive gravity center state, wherein the model considers the elasticity of the lifting rope, the inertial coupling of the load and external disturbance and is used for describing the dynamic response characteristic of a lifting system of the unmanned aerial vehicle; The control instruction generation module is used for generating an unmanned aerial vehicle gesture and hoisting control instruction by adopting a self-adaptive control algorithm according to the dynamics model, and realizing dynamic correction and stable control of the load center of gravity by preferentially adjusting a position error in a hovering state and preferentially adjusting a speed error in a moving state; The feedback monitoring module is used for monitoring the visual gravity center and the change of the lifting rope tension in real time while the unmanned aerial vehicle executes the control instruction, and feeding back the monitoring result to the control module to form closed-loop control so as to inhibit load swing and keep balance; And the safety control module is used for triggering a safety mechanism to automatically slow down, hover or emergency stop when the monitoring result shows that the load gesture or the lifting rope tension exceeds the threshold value, until the load is restored to be stable or the lifting task is completed.

Description

Unmanned aerial vehicle lifting control method and system based on industrial vision Technical Field The invention relates to the technical field of unmanned aerial vehicle lifting and control, in particular to an unmanned aerial vehicle lifting control method and system based on industrial vision. Background In recent years, unmanned aerial vehicles rapidly develop in the application of industrial fields, and play an important role in the scenes such as high-altitude hoisting, power transmission line overhaul, building construction and logistics transportation. As the complexity of the handling task increases, the handling of heavy components by single or multiple machines in tandem is normal. However, complex environmental factors such as wind turbulence, load nonlinear oscillations, multi-machine attitude collaboration, and imperfections in the sensory information make unmanned aerial vehicle handling systems a significant challenge in terms of stability, accuracy, and safety. The traditional unmanned aerial vehicle lifting method mostly depends on a single visual recognition mode, and the problem that the load is easy to swing and unstably is caused due to the lack of a gravity center dynamic correction mechanism. Disclosure of Invention In order to make up for the defects, the invention provides an unmanned aerial vehicle lifting control method and system based on industrial vision, and aims to solve the problem that the load is easy to swing and unstably due to the lack of a gravity center dynamic correction mechanism in a mode that the traditional unmanned aerial vehicle lifting method mostly depends on single vision identification. In a first aspect, the present invention provides a method for controlling lifting of an unmanned aerial vehicle based on industrial vision, comprising the following steps: S1, collecting multi-angle visual image data of a load and real-time tension information of each lifting rope when an unmanned aerial vehicle is in lifting operation; s2, reconstructing three-dimensional point cloud according to the data, extracting characteristic points to calculate a visual gravity center, determining a mechanical gravity center of the lifting rope according to the stress analysis of the lifting rope, and dynamically adjusting the visual gravity center based on the coordinate difference and the stress balance relationship of the two to obtain the comprehensive gravity center position and the speed state of the load; s3, establishing a nonlinear dynamics model of the unmanned aerial vehicle, the lifting rope and the load based on the obtained comprehensive gravity center state, wherein the model considers the elasticity of the lifting rope, the inertial coupling of the load and external disturbance and is used for describing the dynamic response characteristic of a lifting system of the unmanned aerial vehicle; s4, generating an unmanned aerial vehicle gesture and hoisting control instruction by adopting a self-adaptive control algorithm according to the dynamics model, and realizing dynamic correction and stable control of the load gravity center by preferentially adjusting a position error in a hovering state and preferentially adjusting a speed error in a moving state; S5, the unmanned aerial vehicle executes a control instruction and simultaneously monitors the visual gravity center and the change of the lifting rope tension in real time, and the monitoring result is fed back to the control module to form closed-loop control so as to inhibit load swing and keep balance; And S6, when the monitoring result shows that the load gesture or the lifting rope tension exceeds a threshold value, the system triggers a safety mechanism to automatically slow down, hover or emergency stop until the load is restored to be stable or the lifting task is completed. By adopting the technical scheme, the dynamic coordination and stable control of the load gesture and the unmanned aerial vehicle gesture are realized based on the comprehensive gravity center modeling and the self-adaptive control of the industrial vision and the mechanical fusion, so that the problem that the load is easy to swing and unstably due to the lack of a gravity center dynamic correction mechanism in a mode that the traditional unmanned aerial vehicle lifting method mostly depends on single vision identification is solved. Preferably, the image data and the tension information include: The unmanned aerial vehicle acquires multi-angle images of the load through a plurality of industrial cameras before lifting; Inputting each image into a three-dimensional point cloud reconstruction module, generating a point cloud model of a load by adopting a reconstruction algorithm based on stereo matching, registering the point cloud, extracting features, and calculating a geometric center to obtain a visual center of gravity; The stress value of each lifting rope is obtained through the lifting rope tension sensor, a moment balance equati