Search

CN-121662331-B - Infant airway collaborative intubation safety control system based on off-site mixed reality

CN121662331BCN 121662331 BCN121662331 BCN 121662331BCN-121662331-B

Abstract

The invention discloses an infant airway collaborative intubation safety control system based on remote mixed reality. The system aims to solve the problems of space synchronization robustness and predictive delay compensation in remote guidance, establishes a mapping relation between a local coordinate system and a remote coordinate system through a space synchronization module, and utilizes a long-term memory network (LSTM) to carry out predictive compensation on a remote expert track so as to eliminate visual lag. The core safety mechanism comprises an airway dynamic model module for generating a real-time dynamic signed distance field (D-SDF), and a risk prediction module for calculating the dynamic collision risk probability of the laryngoscope tail end and the dynamic airway wall in real time. The haptic feedback device responds to the risk probability, adopts a variable admittance control strategy, generates operation viscous feeling through nonlinear increase of virtual damping, activates a virtual stiffness item pointing to a safety area when the risk exceeds a critical value, so as to apply reverse elastic restoring force, realize active safe avoidance of a physical layer, and ensure sub-millimeter precision and safety of infant airway operation.

Inventors

  • NI XIN
  • ZHAO XIN

Assignees

  • 首都医科大学附属北京儿童医院

Dates

Publication Date
20260508
Application Date
20260204

Claims (7)

  1. 1. Infant airway collaborative intubation safety control system based on mixed reality in different places, which is characterized by comprising: the local state sensing module is configured to acquire and calculate the kinematic state data of the tail end of the laryngoscope in real time; The airway dynamic model module is configured to update geometric deformation of a pre-stored airway three-dimensional model according to real-time physiological motion signals of an infant patient to generate real-time dynamic airway wall position information, and concretely comprises a static signed distance field, a real-time normal vector direction, a dynamic airway wall position information processing module and a dynamic airway wall position information processing module, wherein the static signed distance field is constructed based on preoperative medical images and used for representing the minimum Euclidean distance from space points in the airway to the inner wall surface of the airway; The system comprises a space synchronization module, a remote virtual coordinate system and a local physical coordinate system, wherein the space synchronization module is configured to establish a real-time mapping relation between the local physical coordinate system and the remote virtual coordinate system; the remote coordination module is configured to receive remote guidance data, compensate the transmission delay by using a prediction algorithm, generate a prediction guidance track and display the prediction guidance track in a local visual field in a superposition manner; the risk prediction module is configured to calculate the probability of dynamic collision risk according to the spatial relationship and the motion trend between the kinematic state of the laryngoscope tail end and the real-time dynamic airway wall position information, and specifically comprises the steps of calculating the projection of a laryngoscope tail end speed vector in the real-time normal vector direction of the dynamic airway wall surface, obtaining a relative approaching speed, constructing a nonlinear probability density function, outputting a risk probability value which rises exponentially when the relative approaching speed points to the airway wall and the distance is smaller than a safety threshold value, and introducing an attenuation factor to inhibit the risk probability value when the relative approaching speed deviates from the airway wall; the haptic feedback device is configured to adaptively adjust a mechanical resistance parameter output to an operation handle in response to the dynamic collision risk probability to apply active haptic intervention when the risk rises, and specifically comprises a virtual damping coefficient and a virtual stiffness coefficient, wherein the mechanical resistance parameter comprises the virtual damping coefficient and the virtual stiffness coefficient, the haptic feedback device is configured to apply reverse elastic restoring force pointing to a safety area on a physical layer by adjusting the virtual stiffness coefficient to construct a forbidden entrance field preventing the safety area, wherein the virtual damping coefficient is increased in a nonlinear manner to generate operation viscous feeling when the dynamic collision risk probability rises, and the reverse elastic restoring force pointing to the safety area is generated when the dynamic collision risk probability exceeds a preset critical value.
  2. 2. The system of claim 1, wherein the predictive algorithm in the remote collaboration module employs a long-short-term memory network (LSTM) configured to input a sequence of remote expert gestures within a historical time window received by the input layer, to extract time-sequential motion characteristics of expert operations by the hidden layer using a gating mechanism, and to output a gesture position of a time step predicted in the future to counteract visual lag caused by network transmissions.
  3. 3. The system according to claim 1, wherein the local state sensing module adopts an Error State Kalman Filtering (ESKF) algorithm to calculate the kinematic state data of the laryngoscope end, and specifically comprises dividing the kinematic state into a nominal state and an error state, integrating the high-frequency inertial measurement data and the low-frequency position tracking data to predict the nominal state of the laryngoscope end in an integral way and observe and correct the error state, and outputting the kinematic state data comprising the position uncertainty, wherein the position uncertainty is input into the risk prediction module as a parameter for adjusting the sensitivity of risk calculation.
  4. 4. The system of claim 1, wherein the method for establishing the real-time mapping relationship by the spatial synchronization module comprises identifying anatomical semantic anchors in the endoscope video stream using computer vision algorithms, constructing a weighted registration objective function comprising topologically rigid constraint terms based on the anatomical semantic anchors, and solving a rigid transformation matrix between the local and remote coordinate systems.
  5. 5. The system of claim 4, wherein the anatomical semantic anchor points are spatially-coordinated fiducials extracted for critical anatomical structures within the identified airways in an endoscopic video stream, including pre-glottic unions and/or arytenoid cartilaginous vertices and/or tracheal carina.
  6. 6. The system of claim 5, wherein the topologically rigid constraint term is configured to ensure anatomical accuracy and robustness of spatial synchronization by calculating a pre-stored relative geometric distance or angular relationship of critical anatomical structures within the airway in three-dimensional model space and penalizing any rigid transformation matrix solution that results in a significant deviation of the pre-stored relative geometric distance or angular relationship.
  7. 7. The system according to claim 1, wherein when the dynamic collision risk probability exceeds a preset critical value, generating a reverse elastic restoring force directed to a safe area, specifically comprising constructing a forbidden entry field, and activating when a minimum distance is smaller than a preset safety margin distance, wherein the reverse elastic restoring force is consistent with a normal vector direction of the dynamic airway wall surface, and the normal vector direction is directed to the safe area of the airway center, so as to physically limit the displacement of the laryngoscope tip to a dangerous direction.

Description

Infant airway collaborative intubation safety control system based on off-site mixed reality Technical Field The invention belongs to the fields of computer communication, mixed Reality (MR) technology, medical digital twin and remote medical guidance, and particularly relates to an infant airway collaborative intubation safety control system based on remote mixed reality, which is realized by using a 5G network. Background With the development of telemedicine technology, it has become possible to use the 5G low-latency high-bandwidth feature for remote guidance. However, in infant airway intubation procedures requiring extremely high precision (e.g., sub-millimeter scale), remote collaborative guidance faces two major technical bottlenecks: Firstly, since the anatomical structure of the local infant and the laryngoscope used by the operator are located in the local physical coordinate system, and the guiding label and the virtual model of the remote expert are located in the remote or cloud virtual coordinate system, the robustness of the remote spatial synchronization is very poor. Traditional coordinate transmission is easily affected by network jitter, so that an arrow or a line drawn in a virtual space by an expert cannot be accurately adsorbed on a specific anatomical structure (such as glottic crack) of a local infant, and guidance deviation is caused. Second, while 5G technology provides very low latency, in practical applications, data packing, decoding, rendering, and complex cloud computing still result in a small "lag" of remote expert gesture instructions in the local operator's view. This delay is unacceptable in minimally invasive procedures requiring millisecond feedback, and can greatly impact the accuracy of the instruction and the fluency of the procedure. Therefore, the prior art lacks a high-precision collaborative guiding mechanism capable of overcoming the off-site spatial fluctuation and realizing predictive time delay compensation. Disclosure of Invention In order to solve the technical problems, the invention provides an infant airway collaborative intubation safety control system based on remote mixed reality, which is characterized by comprising the following steps: the local state sensing module is configured to acquire and calculate the kinematic state data of the tail end of the laryngoscope in real time; the airway dynamic model module is configured to update geometric deformation of a prestored airway three-dimensional model according to real-time physiological motion signals of the child patient, and generate real-time dynamic airway wall position information; The space synchronization module is configured to establish a real-time mapping relation between a local physical coordinate system and a remote virtual coordinate system; the remote coordination module is configured to receive remote guidance data, compensate the transmission delay by using a prediction algorithm, generate a prediction guidance track and display the prediction guidance track in a local visual field in a superposition manner; The risk prediction module is configured to calculate dynamic collision risk probability according to the spatial relationship and the motion trend between the kinematic state of the laryngoscope tail end and the real-time dynamic airway wall position information; A haptic feedback device configured to adaptively adjust a mechanical resistance parameter output to the operating handle in response to the dynamic collision risk probability to apply active haptic intervention at elevated risk. The airway dynamic model module generates real-time dynamic airway wall position information and specifically comprises the steps of constructing a static signed distance field based on a preoperative medical image and used for representing the minimum Euclidean distance from a space point in an airway to the inner wall surface of the airway, wherein numerical values in the static signed distance field are positive and negative and used for distinguishing an inner safety area of the airway and a wall surface penetrating area, calculating a real-time respiratory phase according to a physiological motion signal acquired in real time through a respiratory motion function and calculating a deformation field vector corresponding to the respiratory phase based on a preset statistical shape model, reversely shifting and inquiring and correcting the static signed distance field by utilizing the deformation field vector to generate a dynamic signed distance field containing a time dimension, outputting the dynamic airway wall position information in real time based on the dynamic signed distance field, and determining the real-time normal vector direction of the airway wall surface by calculating the gradient direction of the dynamic signed distance field. Particularly, a prediction algorithm in the remote collaboration module adopts a long-short-term memory network (LSTM) which is configured to enable