CN-115329922-B - Digital twin system of underground multifunctional four-foot robot for coal mine and operation method thereof

CN115329922BCN 115329922 BCN115329922 BCN 115329922BCN-115329922-B

Abstract

The invention discloses a digital twin system of a multifunctional four-foot robot in a coal mine and an operation method thereof, and belongs to the technical field of digital twin. The system comprises a robot virtual space and a robot physical entity, wherein the robot physical entity is of a four-foot structure and comprises an autonomous sensing module, an autonomous decision-making module and an autonomous control module, and the robot virtual space comprises four subsystems, namely a robot virtual simulation subsystem, a robot virtual planning subsystem, a robot virtual debugging subsystem and a robot virtual monitoring subsystem. The robot virtual simulation, virtual planning and virtual debugging subsystem provides information and reference for the design and operation of the robot physical entity, and the robot virtual monitoring subsystem provides three-dimensional operation state monitoring and a remote visual man-machine interaction interface for the robot physical entity. Through real-time bidirectional information interaction mapping of physical and virtual spaces, a digital twin solution is provided for design, operation and maintenance and monitoring of the underground coal mine robot, and the intelligent level of the coal mine is improved.

Inventors

WANG XUEWEN
MENG HAO
LIU SHUGUANG
XIE JIACHENG
HUANG ZHUO
SHI XINCUI
LIU JUN
Hao Zixiang
GE FUXIANG
YAN ZEWEN

Assignees

太原理工大学
太原理工大学

Dates

Publication Date: 20260421
Application Date: 20220922
Priority Date: 20220922

Claims (10)

1. The utility model provides a colliery is multi-functional four-legged robot digital twin system in pit, includes robot virtual space and robot physical entity, its characterized in that: the robot virtual space comprises a robot virtual simulation subsystem, a robot virtual planning subsystem, a robot virtual debugging subsystem and a robot virtual monitoring subsystem; the robot virtual simulation subsystem is developed by Unity3d and is used as a test verification platform for early-stage robot body design and kinematic modeling; The robot virtual planning subsystem is developed by Unity3d and is used for completing the path planning of the underground multifunctional four-legged robot of the coal mine in a virtual environment; the robot virtual debugging subsystem is developed by Webots and is used for testing and optimizing the robot virtual planning subsystem; The robot virtual monitoring subsystem is developed by Unity3d and comprises a robot virtual monitoring module and a teleoperation module; The robot virtual debugging subsystem is communicated with the Unity3d upper computer of the robot virtual monitoring subsystem through a wireless serial port communication module; The autonomous sensing module is used for sensing the state of the robot and the environmental information; the autonomous decision module is used for completing three self-adaption to unknown complex terrain, namely robot body height self-adaption, body gesture self-adaption and robot gait planning; The autonomous control module is used for receiving a control instruction from the Unity3d upper computer and converting the control instruction into a robot action.
2. The underground coal mine multifunctional quadruped robot digital twin system of claim 1, wherein in a robot virtual monitoring subsystem, a virtual monitoring module reads the angle information of each joint of a robot sent by an autonomous sensing module through a serial port, disassembles each joint data according to a sending rule, assigns the data to each joint angle of a digital twin body of the robot, realizes three-dimensional visual monitoring of the real-time running state of the robot, observes the pose state of the robot in real time in a virtual environment, can also read an environment map around the robot sent by the autonomous sensing module, and observes the surrounding environment of the robot in real time in the virtual environment.
3. The underground coal mine multifunctional quadruped robot digital twin system of claim 2, wherein in the robot virtual monitoring subsystem, the teleoperation module remotely controls the robot digital twin body in the robot physical entity or the robot virtual debugging subsystem to complete the complex task of uncertainty through a UGUI human-computer interaction panel in the Unity3 d.
4. The underground coal mine multifunctional quadruped robot digital twin system according to claim 1 or 3, wherein the autonomous sensing module comprises a three-dimensional laser radar, a monocular camera, a body posture sensor, a steering engine internal angle sensor and a foot end ground contact sensor; The three-dimensional laser radar, the monocular camera and the body attitude sensor are used for constructing an environment map in the robot virtual planning subsystem, and simultaneously, the environment map is autonomously positioned by utilizing environment information, so that the robustness and the accuracy of the system can be improved by fusing the three; the steering engine internal angle sensor is used for feeding back angle information of each joint of the robot; The foot end ground contact sensor is used for providing a feedback signal when the robot foot end touches the ground, so that the foot end stops falling down to adapt to unknown complex ground.
5. The underground coal mine multifunctional quadruped robot digital twin system of claim 4, wherein the body posture sensor is used for constructing an environment map and autonomous positioning, and is also used for sensing the posture information of the robot body in real time so as to keep the stability of the posture of the robot.
6. The method of operating a multi-functional four-legged robot digital twinning system in a coal mine well of claim 5, comprising the steps of: S1, completing the construction of a digital twin body of a coal mine underground multifunctional robot in a Unity3d environment in a robot virtual simulation subsystem, wherein the method comprises the following specific processes: s101, designing the size of a robot and establishing a robot model in modeling software; S102, importing a robot model into three-dimensional rendering software, adjusting coordinate axes of all parts, rendering the model, and importing the model into a robot virtual simulation subsystem, wherein the adjustment of the coordinate axes of all parts is to adjust a local coordinate system of all parts of the robot model to a proper position so as to ensure that the model imported into Unity3d does not rotate the local coordinate axes; S103, establishing a parent-child relationship for a robot model in a robot virtual simulation subsystem, and adding a Transform assembly for each component to enable each component to have basic translation and rotation transformation capabilities; s2, verifying and optimizing a structure, a kinematic model and a gait model of the robot in a robot virtual simulation subsystem, wherein the specific process is as follows: s201, establishing a coal mine underground multifunctional quadruped robot kinematic model, writing a kinematic model C# script, and mounting the kinematic model C# script on a digital twin body of the robot; s202, performing gait planning of a coal mine underground multifunctional quadruped robot, writing a planning result into a gait planning C# script, and mounting the gait planning C# script on a digital twin body of the robot; S203, running a robot digital twin body in a robot virtual simulation subsystem, and testing and optimizing the structure, the kinematic model and the gait of the robot; S3, importing the optimized underground multifunctional four-legged robot digital twin body into a robot virtual planning subsystem, and building a robot physical entity according to the robot digital twin body; S4, establishing a digital twin body construction of the underground multifunctional four-legged robot of the coal mine in the environment of Webots in a robot virtual debugging subsystem, wherein the digital twin body construction comprises the following specific processes: s401, importing a robot model into Webots, and establishing a main node of the robot model in a scene tree; s402, adding an inertial unit (InertialUnit), a position sensor (PositionSensor), a rotating motor (RotationalMotor) and a contact sensor (touch sensor) to the robot model, and setting proper parameters; s403, establishing a robot model main controller program and a kinematic program, and constructing interfaces between the main controller and an inertial unit, a position sensor, a rotating motor and a contact sensor; S5, respectively importing underground offline maps of the coal mine into the robot virtual planning subsystem and the robot virtual debugging subsystem; S6, debugging the underground multifunctional four-legged robot in the robot virtual debugging subsystem, planning a robot path through the robot virtual planning subsystem, executing a planning result in the robot virtual debugging subsystem, and performing semi-autonomous control and manual control on the robot through the robot virtual monitoring subsystem, wherein the specific process is as follows: S601, synchronizing initial pose of a robot digital twin body in a robot virtual planning subsystem and a robot virtual debugging subsystem; S602, establishing a base The method comprises the steps of compiling a path planning C# script of an algorithm of a coal mine underground multifunctional quadruped robot path planning model, and mounting the path planning C# script on a robot digital twin body in a robot virtual planning subsystem; s603, planning a robot path in a robot virtual planning subsystem, synchronizing a planning result into the robot virtual debugging subsystem, and completing walking according to the path planning result; s604, semi-autonomous control and manual control are carried out on the digital twin bodies in the robot virtual debugging subsystem through the robot virtual monitoring subsystem, so that the digital twin bodies of the robot complete the appointed operation; S7, comparing and analyzing a planning result of the robot virtual planning subsystem with a result of execution of a robot digital twin body in the robot virtual debugging subsystem, and optimizing the robot virtual planning subsystem until the robot virtual planning subsystem can accurately plan a path for the robot; s8, placing the robot physical entity in the underground coal mine, and realizing the self-adaptive walking and complex operation of the underground multifunctional four-legged robot in the underground coal mine through the coordinated operation of the robot physical entity, the robot virtual planning subsystem and the robot virtual monitoring subsystem, wherein the specific process is as follows: s801, an autonomous sensing module, an autonomous decision module and an autonomous control module are installed for a robot physical entity, a multi-functional four-legged robot software system under a coal mine is established, and the multi-functional four-legged robot software system is placed under an actual coal mine; S802, a robot physical entity operates in a coal mine well, an environment map around the robot is constructed in real time through an autonomous sensing module, and the map is synchronized to a robot virtual planning subsystem; s803, planning a robot path in a robot virtual planning subsystem, synchronizing a planning result into an autonomous control module of a robot physical entity, and completing walking according to the path planning result to autonomously complete a patrol task; s804, semi-autonomous control and manual control are carried out on the physical entity of the robot through a teleoperation module in a virtual monitoring subsystem of the robot, so that the physical entity of the robot completes a complex task with uncertainty, and the real-time pose state and the surrounding environment of the robot are monitored in the virtual monitoring module; S805, verifying the accuracy and reliability of the kinematic model, the gait model and the path planning model in the virtual simulation system and the virtual planning system according to the real-time monitoring data of the virtual monitoring system, and performing iterative optimization and dynamic correction on the model.
7. The method for operating a multifunctional four-legged robot digital twinning system in a coal mine well according to claim 6, wherein: in step S201, the multi-functional four-legged robot kinematic model under the coal mine includes a robot forward kinematic model and a robot reverse kinematic model; The robot positive kinematics model is used for calculating the position of the foot end according to the posture of the robot body, the length of the connecting rod and the rotation angle of each joint; The inverse kinematics model of the robot is to calculate the rotation angle of each joint according to the posture of the robot body, the length of the connecting rod and the position of the foot end.
8. The method for operating a multi-functional four-legged robot digital twinning system in a coal mine well according to claim 7, wherein: In step S201, the kinematic model c# script includes a forward solution calculation script, an inverse solution calculation script, and an interpolation calculation script; the forward motion model of the robot is compiled in the forward calculation script, and can be used for solving the motion space of the foot end of the robot to judge whether the robot structure is reasonable or not; the inverse solution calculation script is provided with a robot inverse kinematics model, can calculate the angles of all joints and input an interpolation calculation script, and is a basis for realizing the control of the robot; The interpolation calculation script calls Mathf Lerp () function in Update () function, obtains the robot target joint corner from the inverse calculation script every frame, and makes the robot transfer from the existing corner to the target corner.
9. The method for operating a multifunctional four-legged robot digital twinning system in a coal mine well according to claim 6 or 8, wherein: in step S202, the gait planning of the underground multifunctional four-legged robot of the coal mine adopts a static gait planning method, including robot stepping sequence planning and robot gravity center track planning; The gait planning C# script is input into the expected moving speed, attitude angle and body height of the robot, and is output into expected foot end coordinates.
10. The method for operating the multifunctional four-legged robot digital twin system in the coal mine well according to claim 9, wherein in step S5, the offline map in the coal mine well specifically refers to a complete map in the coal mine well constructed in advance based on geological exploration and three-dimensional mapping, and is used for testing the path planning function of the robot virtual planning subsystem.

Description

Digital twin system of underground multifunctional four-foot robot for coal mine and operation method thereof Technical Field The invention belongs to the technical field of digital twinning, and particularly relates to a digital twinning system of a multifunctional four-foot robot in a coal mine and an operation method. Background The intellectualization is a core technical support for the high-quality development of the coal industry, and is the future of the development of the coal industry. The coal mine robot can effectively improve the safety of coal mine production by working dangerous and heavy work of the ultra-six coal mine personnel and developing and applying the coal mine robot. The technical difficulty of researching the coal mine robot is large, the period is long, the cost is high, certain key technologies are not solved, and the design and the application of the coal mine robot still have a plurality of defects, and the main aspects are as follows: (1) Currently, existing coal mine robots are only applied to the fields related to detection such as inspection, rescue and the like, and have no substantial progress for operation under complex conditions; (2) Because the underground environment of the coal mine has complexity and unpredictability, compared with a ground robot, the coal mine robot has higher field debugging difficulty and stronger unknown property, and is easy to cause waste of manpower and material resources; (3) At present, the coal mine robot is difficult to replace a worker to realize complete autonomous operation, the cooperation between the worker and the robot is still needed, and a human-computer interaction channel between the worker and the coal mine robot is still needed to be researched; (4) The underground topography of the coal mine is complex and changeable, but the coal mine robot lacks an effective three-dimensional visual remote monitoring means, and the real-time pose state of the robot is difficult to intuitively and comprehensively obtain. With the continuous progress and maturity of digital twin technology, a bridge crossing the physical world and the virtual world is built by combining the wide application and rapid development of modern communication technology. The digital twin technology can map a digital model of the physical world in a virtual space, sense, diagnose and predict the state of a physical entity object in the virtual space in real time, and regulate and control the physical entity through optimization and instructions. The method also provides a brand new thought and an important reference for the design, operation and maintenance and monitoring of the coal mine robot. In the prior art, a digital twin-driven inspection robot remote control system and method disclosed in the patent document with publication number CN113050649A comprise an inspection robot physical entity, a data sensing and transmitting module, an inspection robot virtual entity, a control module and twin data, and a digital twin technology is utilized to establish a virtual remote control platform of the coal mine fully-mechanized working face inspection robot so as to realize virtual and actual synchronization, state monitoring and remote control of the coal mine fully-mechanized working face inspection robot. In the coal mine dispatching robot system based on the artificial intelligence technology disclosed in the patent document of publication No. CN114398773A, a dispatching twin model of a virtual space which is mutually mapped with an underground physical space dispatching system is constructed through a digital twin technology, and acquired real-time data in the mine production process and physical entities are fused and cooperated to realize three-dimensional visualization of coal mine production command dispatching. However, in the two schemes, the digital twin technology is only used for remote state monitoring and control scheduling of the coal mine robot, and the powerful analysis and prediction capabilities of the digital twin technology in the virtual space are not fully exerted, and the digital twin technology is not applied to prepositive links such as virtual simulation, virtual planning, virtual debugging and the like of the coal mine robot, so that information and reference are provided for autonomous execution of a physical system of the robot. And the man-machine interaction method for connecting the physical space and the virtual space in the process of robot state monitoring and remote control is not involved. In the prior art, in a testing and evaluating method for an intelligent fully-mechanized mining robot production system disclosed in a patent document with publication number CN113128109A, a fully-mechanized mining working face virtual off-line operation system is constructed, working face virtual operation mining conditions are reproduced, simulation initial data and virtual scene operation data are determined, an AI robot analysis system is c