Search

CN-121989226-A - Remote control method and system for remote control robot based on man-machine interaction

CN121989226ACN 121989226 ACN121989226 ACN 121989226ACN-121989226-A

Abstract

The invention discloses a remote control method and a remote control system for a remote control robot based on man-machine interaction, wherein the method comprises the following steps: the method comprises the steps of collecting multi-mode interaction signals of a user and environment perception data obtained by a robot, generating a control instruction matched with the requirement of the user through a dynamic delay compensation algorithm and a self-adaptive authority allocation mechanism according to the multi-mode interaction signals and the environment perception data, sending the control instruction to the robot, obtaining execution state and environment interaction information of the robot when the fact that the robot executes the control instruction is detected to be operated is finished, and feeding back the execution state and the environment interaction information to the user. Therefore, the invention can solve the technical problems of longer response time of the effective robot, lower control efficiency and accuracy of the robot and poorer operation flexibility, and realize intelligent multi-mode control of the robot, thereby being beneficial to improving the control efficiency and accuracy of the robot and the operation flexibility.

Inventors

  • ZHANG HAIJUN

Assignees

  • 深圳智动无界科技有限公司

Dates

Publication Date
20260508
Application Date
20251223

Claims (10)

  1. 1. A remote control method for a remote control robot based on man-machine interaction, the method comprising: Collecting multi-mode interaction signals of a user and environment perception data obtained by a robot; Generating a control instruction matched with the requirement of the user according to the multi-mode interaction signal and the environment sensing data and through a dynamic delay compensation algorithm and a self-adaptive authority allocation mechanism; the control instruction is sent to the robot and is used for driving a motion executing mechanism and an end effector of the robot; when the robot is detected to finish executing the control instruction, acquiring the execution state and environment interaction information of the robot, and feeding back the execution state and environment interaction information to the user.
  2. 2. The remote control method of the remote control robot based on human-computer interaction of claim 1, wherein the multi-modal interaction signals comprise electromyographic signals, visual capturing signals and handheld control signals of the user, the environment sensing data comprise position data, gesture data, force feedback data and visual feedback data of the robot, the electromyographic signals are acquired through an electromyographic signal acquisition module, the visual capturing signals are acquired through a visual capturing module, the handheld control signals are acquired through a handheld control module, and the environment sensing data are acquired through a robot sensing module.
  3. 3. The remote control method of a remote control robot based on man-machine interaction according to claim 1, wherein the specific implementation manner of the dynamic delay compensation algorithm is as follows: measuring the delay time of network transmission between the robot and the terminal equipment of the user, wherein the delay time comprises uplink delay time and downlink delay time; establishing a motion prediction model of the robot, and predicting a predicted motion state of the robot within a delay time through the motion prediction model based on historical motion data and a current control instruction of the robot; generating a compensation control instruction according to the predicted motion state of the robot; when the robot is detected to execute the compensation control instruction, feedback data generated after the robot executes the compensation control instruction are obtained, and a prediction error is calculated according to the feedback data; Adjusting parameters of the motion prediction model according to the prediction error, fusing the predicted motion state and the actual feedback state through a Kalman filter to obtain target fusion data, inputting the target fusion data into the adjusted motion prediction model, and generating an optimal estimated motion state; the motion prediction model is realized through a long-term and short-term memory network, the feedback data comprise actual feedback states of the robot, and the optimal estimated motion states are used for generating control instructions matched with requirements of the user.
  4. 4. The remote control method of a remote control robot based on man-machine interaction according to claim 1, wherein the specific implementation manner of the adaptive authority allocation mechanism is as follows: Calculating a task complexity index according to the task type, the environment complexity and the operation difficulty; calculating an operator proficiency index according to historical operation data, operation success rate and operation fluency of an operator; according to the task complexity index and the operator proficiency index, calculating a weight corresponding to a man-machine control authority, wherein the weight corresponding to the man-machine control authority comprises a manual control weight and an autonomous control weight, the sum of the manual control weight and the main control weight is 1, and the manual control weight and the main control weight are used for generating a control instruction matched with the requirement of the user; And smoothly transitioning the manual control weight and the autonomous control weight according to an exponential smoothing algorithm.
  5. 5. The remote control method for a remote robot based on man-machine interaction according to any one of claims 2 to 4, further comprising, after said sending the control instruction to the robot: generating an articulation locus of the robot according to a control instruction, and calculating an articulation angle of the robot according to the articulation locus and an inverse kinematics algorithm; adjusting the grabbing force corresponding to the end effector of the robot according to the force feedback signal, and controlling the end effector according to the joint movement track, the joint movement angle and an impedance control algorithm; Monitoring a motion state and a moment state of the robot when the robot is controlled, and triggering the end effector to stop emergently when detecting that the motion state and the moment state are abnormal; the control period of the motion controller is smaller than or equal to a first time length threshold value, and the force control precision of the force controller is smaller than or equal to a first stress threshold value.
  6. 6. The remote control method for a remote control robot based on man-machine interaction according to claim 2, wherein the obtaining the execution state and environment interaction information of the robot comprises: Acquiring first person vision and augmented reality information of the robot through a visual feedback module; acquiring contact force data and grabbing force feedback data corresponding to an end effector of the robot through a force touch feedback module; Acquiring the running sound and the peripheral environment sound of the robot through an auditory feedback module; The method further comprises the steps of: performing band-pass filtering, notch filtering and normalization processing on the electromyographic signals to obtain processed electromyographic signals; extracting time domain features and frequency domain features corresponding to the processed electromyographic signals, wherein the time domain features comprise root mean square values, average absolute values and waveform lengths, and the frequency domain features comprise median frequencies and average power frequencies; And inputting the time domain features and the frequency domain features into a preset motion behavior model for recognition to obtain the motion intention of the user.
  7. 7. The human-machine interaction based remote control method of a remote control robot of claim 5, further comprising: recording interaction data of the user, motion data of the robot and task execution data; Evaluating the operation efficiency, the operation precision and the task success rate of the user according to the interaction data of the user, the motion data of the robot and the task execution data; Training and optimizing a motion prediction model of the robot, and a permission allocation strategy and control parameters based on the interaction data of the user, the motion data and the task execution data of the robot.
  8. 8. A remote control system for a remote control robot based on human-machine interaction, the system comprising: the acquisition module is used for acquiring multi-mode interaction signals of a user and environment perception data acquired by the robot; The generation module is used for generating a control instruction matched with the requirement of the user according to the multi-mode interaction signal and the environment sensing data acquired by the acquisition module and through a dynamic delay compensation algorithm and a self-adaptive authority allocation mechanism; The sending module is used for sending the control instruction generated by the generating module to the robot, and the control instruction is used for driving a motion executing mechanism and an end effector of the robot; and the acquisition feedback module is used for acquiring the execution state and environment interaction information of the robot when the control instruction sent by the sending module is detected to be executed by the robot, and feeding back the execution state and environment interaction information to the user.
  9. 9. A remote control system for a remote control robot based on human-machine interaction, the system comprising: A memory storing executable program code; A processor coupled to the memory; the processor invokes the executable program code stored in the memory to perform the telerobot tele-control method based on human-computer interaction as claimed in any one of claims 1-7.
  10. 10. A computer storage medium storing computer instructions for performing the man-machine interaction based telerobot tele-control method according to any one of claims 1-7 when called.

Description

Remote control method and system for remote control robot based on man-machine interaction Technical Field The invention relates to the technical field of man-machine interaction, in particular to a remote control method and a remote control system for a remote control robot based on man-machine interaction. Background With the rapid development of robot technology, remote control robots are widely used in the fields of dangerous environment operation, telemedicine, space exploration and the like. However, the existing remote control robot has the following technical problems: The system of the robot can not fully utilize the multi-mode interaction capability of a human body, has single interaction mode and low operation efficiency, has delay time, lacks an effective delay compensation mechanism and influences operation experience due to network delay, and generally adopts a purely manual control mode or a purely autonomous control mode and lacks a flexible man-machine cooperation mechanism, thereby causing the problems of longer response time of the robot, lower control efficiency and accuracy of the robot and poorer operation flexibility. Therefore, the remote control method and the remote control system for the remote control robot based on human-computer interaction are provided at present, and can solve the technical problems of longer response time of an effective robot, lower control efficiency and accuracy of the robot and poorer operation flexibility, and realize intelligent multi-mode control of the robot, thereby being beneficial to improving the control efficiency and accuracy of the robot and the operation flexibility. Disclosure of Invention The invention provides a remote control method and a remote control system for a remote control robot based on man-machine interaction, which can realize intelligent multi-mode control of the robot and are beneficial to improving the control efficiency, accuracy and operation flexibility of the robot. In order to solve the technical problems, a first aspect of the present invention discloses a remote control method for a remote control robot based on man-machine interaction, the method comprising: Collecting multi-mode interaction signals of a user and environment perception data obtained by a robot; Collecting multi-mode interaction signals of a user and environment perception data obtained by a robot; Generating a control instruction matched with the requirement of the user according to the multi-mode interaction signal and the environment sensing data and through a dynamic delay compensation algorithm and a self-adaptive authority allocation mechanism; the control instruction is sent to the robot and is used for driving a motion executing mechanism and an end effector of the robot; When the operation of the robot executing the control instruction is detected, acquiring the execution state and environment interaction information of the robot, and feeding back the execution state and environment interaction information to the user. In a first aspect of the present invention, the multi-mode interaction signal includes an electromyographic signal, a visual capturing signal and a handheld control signal of the user, the environmental perception data includes position data, gesture data, force feedback data and visual feedback data of the robot, the electromyographic signal is acquired by an electromyographic signal acquisition module, the visual capturing signal is acquired by a visual capturing module, the handheld control signal is acquired by a handheld control module, and the environmental perception data is acquired by a robot perception module. As an optional implementation manner, in the first aspect of the present invention, a specific implementation manner of the dynamic delay compensation algorithm is: measuring the delay time of network transmission between the robot and the terminal equipment of the user, wherein the delay time comprises uplink delay time and downlink delay time; establishing a motion prediction model of the robot, and predicting a predicted motion state of the robot within a delay time through the motion prediction model based on historical motion data and a current control instruction of the robot; generating a compensation control instruction according to the predicted motion state of the robot; when the robot is detected to execute the compensation control instruction, feedback data generated after the robot executes the compensation control instruction are obtained, and a prediction error is calculated according to the feedback data; Adjusting parameters of the motion prediction model according to the prediction error, fusing the predicted motion state and the actual feedback state through a Kalman filter to obtain target fusion data, inputting the target fusion data into the adjusted motion prediction model, and generating an optimal estimated motion state; the motion prediction model is realized through a long-term and sho