CN-122005102-A - Flexible surgical robot operation area force feedback teleoperation control method and device

Abstract

The invention discloses a force feedback teleoperation control method and device for a flexible surgical robot operation area, and relates to the technical field of medical robots. Acquiring signals of an operator main end, acquiring a terminal expected gesture instruction in a space mapping mode, acquiring visual image information through an endoscope camera to acquire terminal stress data, terminal pose estimation, tension data, an environment model and an artificial potential field model, carrying out offset correction on the terminal pose estimation, carrying out judder correction, carrying out periodic pose disturbance correction to obtain a corrected terminal expected gesture, converting the corrected terminal expected gesture instruction into a new target position instruction, and realizing guiding and supporting of a flexible actuator in a deep tortuous path. The invention can obviously map the operation intention of an operator at the main end to the tail end of the slave end flexible robot and feed back the contact force of the operation area to the main end in real time, thereby improving the safety, stability and operation precision of the operation.

Inventors

• XIE LUN
• Du Kangyou

Assignees

• 北京科技大学

Dates

Publication Date

20260512

Application Date

20260127

Claims (10)

1. 1. A method for controlling force feedback teleoperation of a surgical field of a flexible surgical robot, the method comprising: S1, acquiring signals of a main end of an operator, and obtaining an end expected gesture instruction through a spatial mapping mode, wherein the spatial mapping mode comprises mapping the main end gesture into an end expected gesture of a slave end based on a transformation matrix, a proportionality coefficient and a gesture mapping algorithm, the end expected gesture instruction is used for driving a flexible actuator, and the communication link comprises a wired communication link or a wireless communication link; S2, acquiring visual image information through an endoscope camera, extracting image data, potential obstacle information and tail end characteristic positions of an operation area, and preprocessing to obtain tail end stress data, tail end pose estimation, tension data, an environment model and an artificial potential field model; S3, when the device enters an intraoperative dynamic environment, after judging a deviation state, a limited state, an obstacle avoidance state and a temperature safety state, correcting the offset to obtain a corrected terminal expected posture; S4, when tremble exists in operation, calculating pose errors and driving joint moments based on visual deviation and force feedback signals to obtain corrected expected tail end poses; S5, when periodic pose disturbance of a target tissue exists, a periodic displacement function model is established, path optimization is carried out by adopting model predictive control or reinforcement learning strategies, and a corrected terminal expected gesture instruction is obtained, and enables the robot terminal to apply reverse displacement in advance to offset periodic motion of the target; and S6, receiving the corrected terminal expected gesture command, and converting the terminal expected gesture command to obtain a new target position command, wherein the new target position command is used for guiding and supporting the flexible actuator in a deep tortuous path.
2. 2. The flexible surgical robot surgical field force feedback teleoperation control method according to claim 1, wherein the S1 signal of the main end of the acquisition operator obtains an end desired gesture command through a spatial mapping manner, the spatial mapping manner includes mapping the main end pose into a slave end desired pose based on a transformation matrix, a scaling coefficient and a gesture mapping algorithm, the end desired gesture command is used for driving a flexible actuator, and the communication link includes a wired communication link or a wireless communication link, and the method includes: s11, acquiring signals of an operator main end to obtain main end original data, wherein the main end original data comprise position information, posture information, force/pressure signals and zero position posture data, and the zero position posture data are obtained by setting a main end zero position posture; S12, calibrating a master-slave end coordinate system by adopting a hand-eye coordination calibration method to obtain a transformation matrix, wherein the transformation matrix is used for determining the conversion relation among a master operation handle coordinate system, a slave end robot base coordinate system and/or an operation endoscope coordinate system; S13, setting a position scaling and/or a speed scaling according to the operation requirement to obtain a scaling coefficient, wherein the scaling coefficient is used for scaling the movement increment of the master end to obtain the target displacement of the slave end actuator; S14, collecting a rolling angle, a yaw angle and/or a pitch angle of the attitude of the master end, performing angle mapping by adopting a one-to-one correspondence, and setting an attitude mapping algorithm, wherein the attitude mapping algorithm is used for calculating expected attitude change of an end effector of the slave end flexible robot; s15, obtaining a main end control instruction according to the main end original data through a signal processing and instruction generating algorithm, wherein the main end control instruction comprises a position instruction, a gesture instruction or a force control instruction; S16, mapping the master end control instruction into a slave end expected pose based on a space mapping mode, wherein the space mapping mode comprises mapping the master end pose into the slave end expected pose based on a transformation matrix, a proportionality coefficient and a pose mapping algorithm; s17, inputting the expected pose of the slave end terminal into an inverse kinematics solving algorithm or a virtual joint mapping algorithm to obtain the expansion and contraction amount and the driving current of each section of cable, wherein the inverse kinematics solving algorithm comprises an analytic method solving or a numerical iteration solving; And S18, according to the expansion and contraction amount and the driving current of each section of the cable, packaging and transmitting through a communication link to obtain an end expected gesture instruction, wherein the end expected gesture instruction is used for driving the flexible actuator, and the communication link comprises a wired communication link or a wireless communication link.
3. 3. The flexible surgical robot surgical field force feedback teleoperation control method according to claim 1, wherein S2, acquiring visual image information through an endoscope camera, extracting image data, potential obstacle information and terminal characteristic positions of a surgical field, and preprocessing to obtain terminal stress data, terminal pose estimation, tension data, an environmental model and an artificial potential field model, comprises: S21, acquiring visual image information through an endoscope camera, and processing the visual image information by adopting a denoising and enhancing algorithm to obtain image data of an operation area, potential barrier information and tail end characteristic positions, wherein the image data comprises visual characteristics of an operation target and an anatomical structure; S22, acquiring force sense information through a force sense sensor at the tail end, and performing force sense preprocessing to obtain tail end stress data, wherein the force sense preprocessing comprises the steps of adopting a low-pass filtering algorithm to eliminate motor vibration noise of the force sense information and then obtaining the tail end stress data; S23, acquiring temperature information through a temperature sensor, and performing temperature pretreatment to obtain temperature data of the surface of an operation part or an instrument, wherein the temperature pretreatment comprises the step of performing smoothing treatment on the temperature information by adopting a sliding average algorithm to obtain the temperature data; s24, predicting the tail end position according to the robot kinematic model to obtain a predicted tail end position; S25, comparing the tail end characteristic position with the predicted tail end position, and correcting the prediction deviation to obtain tail end pose estimation, wherein the tail end pose estimation comprises a space position and a pose angle of the tail end; S26, estimating the pose of the tail end, and reconstructing the shape of the flexible arm to obtain the reconstructed shape of the flexible arm; S27, acquiring tension information of the cable through a pressure sensor to obtain tension data; s28, modeling an environment through a visual three-dimensional reconstruction algorithm to obtain an environment model; S29, identifying a forbidden access area or an important structure based on the environment model and the potential obstacle information, and setting a virtual boundary, wherein the virtual boundary comprises a safe distance threshold; S210, matching the reconstructed flexible arm shape with the environment model and the target to obtain relative position information of the tail end in the environment; s211, modeling a potential field based on the virtual boundary to obtain an artificial potential field model, wherein the artificial potential field model comprises an attractive potential field, a repulsive potential field and gradients thereof, the attractive potential field is applied to the tail end by an operation target, the repulsive potential field is applied to the tail end by an obstacle, and the potential field modeling comprises the steps of calculating the position of the obstacle and the position of the tail end of the robot by adopting a gradient calculation method.
4. 4. The method for controlling the teleoperation of the force feedback of the surgical area of the flexible surgical robot according to claim 1, wherein the step S3 of correcting the offset after the step S3 of determining the deviation state, the limited state, the obstacle avoidance state and the temperature safety state when the surgical dynamic environment is entered, to obtain the corrected expected terminal posture comprises the steps of: S31, acquiring information in real time through an endoscope camera, an end force sensor and a temperature sensor in operation to obtain current temperature, current force feedback information, current potential obstacle information and current relative position information of the end in the environment; s32, judging the relative position information of the current tail end in the environment according to a preset route to obtain a deviation state, and triggering a deviation adjustment strategy when the deviation state indicates that the tail end deviates from the preset route; S33, comparing and analyzing the current relative position information of the tail end in the environment with the deformation quantity of the actual attitude sensor to obtain deformation difference, and evaluating the deformation difference by combining the stress data of the tail end to obtain a limited state judgment result, wherein the limited state judgment result is used for indicating whether the tail end enters a limited state; S34, triggering a local obstacle avoidance strategy when the limited state judgment result indicates that the tail end is in a limited state; S35, comparing the real-time force feedback information with a preset force threshold, and judging an obstacle avoidance state when the force in a certain direction of the tail end is close to the preset force threshold, and triggering a local obstacle avoidance strategy; S36, judging the temperature information according to a safety threshold value to obtain a temperature safety state, and triggering a speed regulation strategy and a warning signal when the temperature safety state indicates that the temperature exceeds the safety threshold value, wherein the speed regulation strategy is used for reducing the forward speed of the tail end or suspending the tail end from a safety distance; S37, when a deviation adjustment strategy, a local obstacle avoidance strategy or a speed adjustment strategy is triggered, inputting the current temperature, the current force feedback information, the current potential obstacle information and the current relative position information of the tail end in the environment into the artificial potential field model to obtain an updated artificial potential field model, wherein the updated artificial potential field model comprises updated obstacle position parameters and repulsion parameters; S38, inputting the updated artificial potential field model into an admittance control module for force-position mapping to obtain the end position offset; S39, inputting the end position offset into an inverse kinematics solver, and performing inverse kinematics calculation to obtain a joint angle increment or a cable length increment; And S310, planning a path according to the joint angle increment or the cable length increment to obtain the corrected expected tail end gesture.
5. 5. The flexible surgical robot surgical field force feedback teleoperation control method according to claim 1, wherein when tremble exists in the surgical operation in S4, based on the visual deviation and the force feedback signal, calculating a pose error and a driving joint moment to obtain a corrected distal end desired pose, comprising: S41, extracting an expected tail end pose based on a path updating instruction, and calculating a pose difference value between the expected tail end pose and an actual pose to obtain a pose error, wherein the pose error comprises a position deviation and a pose deviation; S42, adopting a parameter self-adaptive strategy, taking visual deviation and a force feedback signal as input to obtain an impedance control parameter correction increment, wherein the impedance control parameter correction increment is used for adjusting an elastic coefficient and a damping coefficient in real time so as to realize self-adaptive adjustment of contact force; S43, establishing an equivalent mass-spring-damping model in a Cartesian space, and correcting an increment based on the equivalent mass-spring-damping model and impedance control parameters according to the pose error to obtain a tail end adjusting force; S44, converting the tail end adjusting force into a preliminary driving joint moment through a jacobian matrix based on a joint space dynamics equation; S45, calculating by adopting a gravity model to obtain the static balance force of each cable; S46, according to the motion speed, obtaining a dynamic compensation term through a Coriolis force and centrifugal force calculation model; s47, superposing the static balance force of each cable and the dynamic compensation term into the preliminary driving joint moment to compensate, so as to obtain the driving joint moment after comprehensive calculation; s48, the comprehensively calculated driving joint moment is subjected to a moment-current conversion algorithm to obtain the corrected expected tail end gesture.
6. 6. The flexible surgical robot surgical field force feedback teleoperation control method according to claim 1, wherein when the periodic pose disturbance of the target tissue exists in S5, a periodic displacement function model is established, path optimization is performed by using model predictive control or reinforcement learning strategy, and a corrected end expected pose instruction is obtained, and the corrected end expected pose instruction enables the robot end to apply reverse displacement in advance to counteract the periodic motion of the target, including: s51, collecting sensor data to obtain periodic pose disturbance of a target tissue, wherein the sensor data comprises target point position data collected by a vision sensor and acceleration data collected by an internal accelerometer, and the periodic pose disturbance is displacement of the target tissue relative to a fixed reference coordinate system; s52, according to the periodic pose disturbance of the target tissue, obtaining a periodic parameter, an amplitude parameter and a phase parameter of physiological motion through data filtering and periodic detection, wherein the data filtering and the periodic detection comprise the steps of estimating a main frequency through fast Fourier transformation, and estimating the amplitude and the phase through self-adaptive filtering or a least square method; s53, establishing a periodic displacement function model according to the periodic parameter, the amplitude parameter and the phase parameter to obtain a prediction model; s54, inputting a prediction model into a time parameter at a future moment to obtain a compensation control quantity, wherein the compensation control quantity is a reverse displacement correction quantity to be applied to the tail end of the robot; s55, performing path optimization by adopting model predictive control or reinforcement learning strategy according to the compensation control quantity to obtain a path optimization result, wherein the reinforcement learning strategy comprises updating the adjustment quantity of output path planning parameters or control gains through interaction of states, actions and rewarding functions; s56, converting the path optimization result into pose instructions of all motors for correction, and carrying out on-line correction on the expected track to obtain corrected tail end expected pose instructions.
7. 7. The flexible surgical robot surgical field force feedback teleoperation control method according to claim 1, wherein the receiving the corrected end expected gesture command of S6 converts the end expected gesture command to obtain a new target position command, where the new target position command is used to implement guiding and supporting of the flexible actuator in a deep tortuous path, and the method includes: S61, acquiring zero position length of each driving wire based on a flexible actuator to obtain reference length data, wherein the zero position length is the reference length of each driving wire when the flexible actuator is straightened, and the flexible actuator comprises at least 2 serial continuous body sections which comprise a plurality of driving wires, a guide cavity and a built-in telescopic framework; S62, receiving a corrected terminal expected gesture command, converting the corrected terminal expected gesture command to obtain target curvature and azimuth angles of each section of the serial continuum section, and calculating the length variation of each driving wire in each section of the serial continuum section relative to the zero position length to obtain the length variation of the driving wire and the total length variation of the driving wire; And S63, based on the total length variation of the driving wire, obtaining a new target position instruction through calculation of the motor rotation angle increment, wherein the new target position instruction is used for realizing guiding and supporting of the flexible actuator in a deep tortuous path.
8. 8. A flexible surgical robotic surgical field force feedback teleoperation control device for implementing a flexible surgical robotic surgical field force feedback teleoperation control method according to any one of claims 1-7, characterized in that the device comprises: The system comprises a main end signal module, a main end signal module and a flexible actuator, wherein the main end signal module is used for acquiring signals of a main end of an operator and obtaining an end expected gesture instruction through a space mapping mode, the space mapping mode comprises mapping the main end gesture into an end expected gesture of a slave end based on a transformation matrix, a proportionality coefficient and a gesture mapping algorithm, the end expected gesture instruction is used for driving the flexible actuator, and the communication link comprises a wired communication link or a wireless communication link; The data acquisition module is used for acquiring visual image information through an endoscope camera, extracting image data of an operation area, potential obstacle information and tail end characteristic positions, and preprocessing to obtain tail end stress data, tail end pose estimation, tension data, an environment model and an artificial potential field model; The dynamic environment module is used for correcting the offset after entering the intra-operation dynamic environment and judging the entering deviation state, the limited state, the obstacle avoidance state and the temperature safety state to obtain the corrected expected tail end gesture; the tremble module is used for calculating pose errors and driving joint moments based on visual deviation and force feedback signals when tremble exists in operation, so as to obtain corrected expected tail end poses; the periodic pose disturbance module is used for establishing a periodic displacement function model when periodic pose disturbance of a target tissue exists, carrying out path optimization by adopting model predictive control or reinforcement learning strategy to obtain a corrected terminal expected pose instruction, and enabling the robot terminal to apply reverse displacement in advance by the corrected terminal expected pose instruction so as to offset periodic motion of the target; And the tail end expected gesture module is used for receiving the corrected tail end expected gesture command, converting the tail end expected gesture command to obtain a new target position command, and the new target position command is used for realizing the guiding and supporting of the flexible actuator in the deep tortuous path.
9. 9. A flexible surgical robot surgical field force feedback teleoperational control device characterized by a flexible surgical robot surgical field force feedback teleoperational control processor, a memory having stored thereon computer readable instructions which, when executed by the processor, implement the method of any one of claims 1 to 7.
10. 10. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a program code, which is callable by a processor for executing the method according to any one of claims 1 to 7.

Description

Flexible surgical robot operation area force feedback teleoperation control method and device Technical Field The invention relates to the technical field of medical robots, in particular to a method and a device for controlling force feedback teleoperation of a surgical area of a flexible surgical robot. Background Natural transluminal surgery (NOTES, natural Orifice Transluminal Endoscopic Surgery) has received great attention because of its lack of body surface incisions and its low trauma. However, the natural cavity is narrow in space, tortuous, and flexible tissue is easy to deform, and higher requirements are put on the flexibility, force sensing and teleoperation control of the robot. The existing interventional operation mainly depends on manual operation of doctors, and is high in operation difficulty and risk. The traditional rigid surgical robot is difficult to penetrate into a target part in a narrow and tortuous natural cavity, has insufficient flexibility and is easy to damage cavity wall tissues. In recent years, flexible continuum robots have shown great potential in natural orifice minimally invasive surgery by virtue of their high flexibility and small diameter characteristics. However, the prior art still has the following problems that the structural design and the driving control of the flexible actuator are difficult to achieve both precision and flexibility, the path drift is caused by dynamic changes (such as limited, shaking, breathing and other physiological movements) in the operation environment, an effective real-time compensation mechanism is lacked, a man-machine cooperative control mechanism is imperfect, and the operation experience of doctors is not matched with the response of a robot. Thus, there is a strong need for a flexible interventional surgical robotic system that can be accurately and safely operated in a dynamic, tortuous path. Disclosure of Invention The embodiment of the invention provides a method and a device for controlling force feedback teleoperation of a flexible surgical robot operation area, which aims to solve the technical problems that a flexible actuator in the prior art is difficult to achieve both precision and flexibility, path drift is caused by dynamic changes (such as limitation, shaking, breathing and other physiological movements) in the operation area, an effective real-time compensation mechanism is lacked, and doctor operation experience is not matched with robot response. The technical scheme is as follows: In one aspect, a method for controlling force feedback teleoperation of a surgical area of a flexible surgical robot is provided, the method is implemented by a device for controlling force feedback teleoperation of the surgical area of the flexible surgical robot, and the method comprises: S1, acquiring signals of a main end of an operator, and obtaining an end expected gesture instruction through a spatial mapping mode, wherein the spatial mapping mode comprises mapping the main end gesture into an end expected gesture of a slave end based on a transformation matrix, a proportionality coefficient and a gesture mapping algorithm, the end expected gesture instruction is used for driving a flexible actuator, and the communication link comprises a wired communication link or a wireless communication link; S2, acquiring visual image information through an endoscope camera, extracting image data, potential obstacle information and tail end characteristic positions of an operation area, and preprocessing to obtain tail end stress data, tail end pose estimation, tension data, an environment model and an artificial potential field model; S3, when the device enters an intraoperative dynamic environment, after judging a deviation state, a limited state, an obstacle avoidance state and a temperature safety state, correcting the offset to obtain a corrected terminal expected posture; S4, when tremble exists in operation, calculating pose errors and driving joint moments based on visual deviation and force feedback signals to obtain corrected expected tail end poses; S5, when periodic pose disturbance of a target tissue exists, a periodic displacement function model is established, path optimization is carried out by adopting model predictive control or reinforcement learning strategies, and a corrected terminal expected gesture instruction is obtained, and enables the robot terminal to apply reverse displacement in advance to offset periodic motion of the target; and S6, receiving the corrected terminal expected gesture command, and converting the terminal expected gesture command to obtain a new target position command, wherein the new target position command is used for guiding and supporting the flexible actuator in a deep tortuous path. Preferably, the step of collecting the signal of the master end of the operator in step S1 obtains an end expected gesture instruction through a spatial mapping mode, wherein the spatial mapping mode include