CN-121837569-B - Heart soft tissue model correction method and device based on physical information neural network

CN121837569BCN 121837569 BCN121837569 BCN 121837569BCN-121837569-B

Abstract

The invention discloses a heart soft tissue model correction method and device based on a physical information neural network, comprising the steps of obtaining preoperative images to construct a personalized heart biomechanics model and an active motion priori library, constructing a double-flow coupling physical information neural network which comprises an active deformation prediction branch for predicting heart autonomous pulsation and a passive residual correction branch for correcting deformation caused by external contact force, defining a loss function comprising data matching loss and biomechanics control equation loss, training the network, receiving cardiac phase data and sparse surface displacement data acquired during operation, inputting the data into the trained network, outputting a global three-dimensional deformation field, and updating the preoperative three-dimensional model. According to the invention, the active and passive mixed deformation of the heart is effectively decoupled through the double-flow coupling framework, and the embedded physical control equation is used as strong constraint, so that the whole-field volume deformation can be inferred only by using sparse surface observation points, and the problem of data sparsity is overcome.

Inventors

Cheng Gangyi
LIN LUQI
HU ZHENZHEN
Cai can
LIU ZHENGSHENG
ZHANG HAIHAO

Assignees

厦门大学附属第一医院(厦门市第一医院、厦门市红十字医院、厦门市糖尿病研究所)

Dates

Publication Date: 20260508
Application Date: 20260310

Claims (10)

1. The heart soft tissue model correction method based on the physical information neural network is characterized by comprising the following steps of: Step one, constructing a personalized biomechanical model of a heart based on preoperative time sequence image data of a patient, and extracting heart beat displacement field data to construct an active motion manifold space; The method comprises the steps of establishing a double-flow coupling physical information neural network, wherein the double-flow coupling physical information neural network receives space three-dimensional coordinates, cardiac phase variables and sparse boundary displacement vectors of grid nodes of the personalized biomechanics model as input, and comprises an active deformation prediction branch, a passive residual error correction branch and a total displacement field, wherein the active deformation prediction branch is used for predicting a reference displacement field constrained by the active motion manifold space based on the space three-dimensional coordinates of the grid nodes and the cardiac phase variables so as to accord with a heart physiological beating mode; step three, defining a total loss function training network containing data matching loss and physical control equation loss, wherein the data matching loss calculates the error of a total displacement field predicted by the network at a sparse observation point, and the physical control equation loss calculates the residual error of the total displacement field substituted into a soft tissue mechanical balance equation so as to realize self-supervision learning under physical constraint; step four, receiving electrocardiosignal data and endoscope video stream data acquired in operation, and determining the cardiac phase variable and sparse boundary displacement vector through signal processing and visual tracking; Inputting the cardiac phase variable and the sparse boundary displacement vector into a trained network, performing forward propagation calculation on all grid vertices of the personalized biomechanical model to obtain the total displacement field, and updating the three-dimensional form of the personalized biomechanical model by using the total displacement field.
2. The method according to claim 1, wherein in the first step, the constructing an active motion manifold space specifically comprises extracting heart beat displacement field data of the heart in a plurality of continuous cardiac cycles from the preoperative time sequence image data, and performing dimension reduction processing on the heart beat displacement field data by adopting a principal component analysis or a variation self-encoder to extract a low-dimension manifold representing time-space characteristics of heart physiological motion, so as to construct the active motion manifold space with a low dimension.
3. The method of claim 1, wherein in the step two, the network structure of the passive residual correction branch comprises a graph neural network layer, and the graph neural network layer adopts an anisotropic inductive graph message transfer mechanism, wherein the anisotropic inductive graph message transfer mechanism fuses myocardial fiber direction field information obtained from the personalized biomechanical model when the message is transferred, and an anisotropic graph convolution operator is defined, so that when the neighborhood node characteristics are aggregated, the aggregation weight is dynamically calculated according to the dot product relationship between the edge vector of the connected neighborhood node and the local myocardial fiber direction vector.
4. The method according to claim 1, wherein in the third step, the soft tissue mechanical balance equation is a hydrostatic balance equation, and the relationship between the total displacement field and the stress tensor is described by a Holzapfel-Ogden constitutive model, the stress tensor being derived from the strain tensor calculated from the total displacement field.
5. The method of claim 1, wherein the training in the third step comprises an off-line pre-training step, wherein the off-line pre-training step is specifically implemented by performing numerical simulation by using the personalized biomechanical model, and generating a composite training sample by randomly applying virtual external forces with different positions, sizes and directions on the surface of the heart model and combining randomly sampled cardiac phases, wherein the composite training sample is used for pre-training the dual-flow coupled physical information neural network.
6. The method according to claim 1, wherein in the fourth step, natural texture points in a visible region of a heart surface are tracked in real time by performing a visual tracking algorithm based on an optical flow method or sparse feature point matching on the endoscopic video stream data, and three-dimensional displacement vectors of the natural texture points are solved to construct the sparse boundary displacement vector.
7. The method of claim 1, wherein in the second step, the output of the active deformation prediction branch is constrained by regularization of the active motion manifold space in such a way that an output layer of the active deformation prediction branch outputs a set of coordinate coefficients under the active motion manifold space, and the reference displacement field is obtained by multiplying and summing the coordinate coefficients with a base vector of the active motion manifold space.
8. The method of claim 1, wherein the cardiac phase variable is a normalized value obtained from R-wave detection and cycle normalization of the electrocardiographic signal data, and is used to indicate the current systolic or diastolic phase of the heart.
9. The method of claim 1, further comprising the step of projecting the three-dimensional morphology of the personalized biomechanical model updated in step five onto an endoscopic image plane to generate an augmented reality navigation image.
10. A heart soft tissue model correction device based on a physical information neural network, comprising: one or more processors, and A memory having stored therein computer instructions; Wherein the one or more processors are configured to, when executing the computer instructions, perform the method of any one of claims 1 to 9.

Description

Heart soft tissue model correction method and device based on physical information neural network Technical Field The application relates to the technical field of artificial intelligence and medical image processing, in particular to a heart soft tissue model correction method and device based on a physical information neural network. Background In cardiac surgery, especially minimally invasive cardiac surgery, augmented reality navigation systems play a vital role. The system can display the shielded heart internal anatomical structures such as coronary arteries and valves in a superimposed manner in the visual field of an operator in the form of three-dimensional images, so that the accuracy and safety of the operation are improved. However, high-precision augmented reality navigation systems face a key technical bottleneck, namely real-time deformation correction of heart soft tissue. The heart acts as a dynamic organ and its deformation is of high complexity. On the one hand, the rhythmic beating of the heart itself can produce periodic large-amplitude active deformations. On the other hand, during surgery, the contact, pressing and pulling of surgical instruments (e.g., fasteners, retractors, etc.) against heart tissue can cause nonlinear passive deformation. The two deformations are closely coupled in time and space, so that a remarkable geometric deviation is generated between a three-dimensional anatomical model constructed based on static medical images (such as Computed Tomography (CT) or Magnetic Resonance Imaging (MRI)) before operation and heart morphology observed in real time during operation, and the reliability of navigation information is reduced. In order to solve the above problems, two types of schemes are mainly adopted in the prior art. The first category is computer vision based solutions, where the geometric position of the model surface is updated by tracking feature points of the heart surface. The limitation of this type of solution is that it cannot infer the volumetric deformation of the internal structure of the heart and the robustness of the tracking is greatly reduced when there are disturbances of blood, smoke, etc. in the surgical field. The second category is finite element analysis methods based on biomechanical models. Although the method has advantages in physical accuracy, the method requires huge calculation resources to solve a control equation describing the nonlinear superelastic behavior of the soft tissue, the time consumption of single deformation calculation is usually in the order of minutes or even hours, and the method can not meet the severe requirement of surgical navigation on real-time performance (delay is usually less than 40 milliseconds). In recent years, physical information neural networks (Physics-Informed Neural Networks, PINN) have been proposed for accelerating the solution of physical equations. The technique constrains the output of the neural network to conform to the laws of physics by taking the residuals of the physical control equations as part of the loss function. However, the direct application of existing PINN technology to cardiac surgery navigation scenarios remains a challenge. First, existing models typically treat all deformations as a single physical process, and cannot effectively distinguish and decouple active deformations caused by heart autonomous beats from passive deformations caused by external instrumentation, which results in a model that produces physical ambiguity in predicting the mixed deformations. Second, in minimally invasive surgery, the field of view acquired through the endoscope is limited, and only a localized area of the heart surface can be observed, resulting in highly spatially sparse visual feature points available to drive the model. For traditional deep learning models driven by dense boundary conditions, solving the internal volume deformation of the whole heart with such sparse surface displacement data is an extremely ill-conditioned inverse problem, where the network is difficult to converge or can produce predictions that do not conform to physical laws. Therefore, the technical scheme capable of performing decoupling correction on the active and passive mixed deformation of the heart in real time by using sparse surface data is a technical problem to be solved in the current field. Disclosure of Invention The invention aims to solve the technical problems that in the prior art, when the deformation correction of a heart soft tissue model in an augmented reality operation navigation system is processed, the real-time performance is insufficient, the active deformation and the passive deformation cannot be decoupled, sparse observation data are difficult to utilize and the like, and provides a heart soft tissue model correction method and device based on a physical information neural network. In order to solve the technical problems, the embodiment of the invention provides a heart soft tissu