CN-122023449-A - Multi-mode image processing method and device and electronic equipment

CN122023449ACN 122023449 ACN122023449 ACN 122023449ACN-122023449-A

Abstract

The invention provides a multi-mode image processing method, a multi-mode image processing device and electronic equipment, which can be applied to the technical field of medical image processing. The method comprises the steps of performing pose conversion on an intermediate conversion segmentation image representing a historical moment state of a target object based on a coronal plane segmentation image and a sagittal plane segmentation image representing the current state of the target object to obtain a target registration segmentation image representing the current state of the target object, wherein the intermediate conversion segmentation image is determined by performing three-dimensional pixel point mapping on a second magnetic resonance segmentation image based on a first magnetic resonance segmentation image to obtain an initial conversion segmentation image mapped to the first magnetic resonance segmentation image under the space coordinates, and performing three-dimensional pixel point mapping on the initial conversion segmentation image based on a three-dimensional contrast segmentation image to obtain an intermediate conversion segmentation image mapped to the three-dimensional contrast segmentation image under the space coordinates.

Inventors

LIU SHIQI
ZHAO HAINING
ZHANG XIAO
GUO SHUAIWEI
WANG TAO
ZHANG LINSEN
LUO JICHANG
XIE XIAOLIANG
ZHOU XIAOHU
HOU ZENGGUANG
Jiao Liqun
HU HAOYU

Assignees

中国科学院自动化研究所
首都医科大学宣武医院

Dates

Publication Date: 20260512
Application Date: 20260414

Claims (10)

1. A multi-modal image processing method, the method comprising: Performing pose conversion on an intermediate conversion segmentation image representing the historical moment state of the target object based on a coronal plane segmentation image and a sagittal plane segmentation image representing the current state of the target object to obtain a target registration segmentation image representing the current state of the target object, wherein the coronal plane segmentation image and the sagittal plane segmentation image are respectively obtained by performing different-azimuth two-dimensional radiography on the target object based on a rotary imaging device; Wherein the intermediate transformed segmented image is determined by: Performing three-dimensional pixel point mapping on a second magnetic resonance segmented image based on a first magnetic resonance segmented image to obtain an initial conversion segmented image mapped to a space coordinate of the first magnetic resonance segmented image, wherein the first magnetic resonance segmented image and the second magnetic resonance segmented image are obtained by performing three-dimensional information acquisition on a target object by using different pulse mechanisms of the same magnetic resonance imaging equipment; And carrying out three-dimensional pixel point mapping on the initial conversion segmentation image based on the three-dimensional contrast segmentation image to obtain an intermediate conversion segmentation image mapped to the space coordinates of the three-dimensional contrast segmentation image, wherein the three-dimensional contrast segmentation image is obtained by carrying out three-dimensional contrast on the target object based on a rotary imaging device.
2. The method according to claim 1, wherein performing pose conversion on the intermediate conversion segmented image representing the historical moment state of the target object based on the coronal segmented image and the sagittal segmented image representing the current state of the target object to obtain a target registration segmented image representing the current state of the target object comprises: Based on a refinement algorithm, respectively refining the coronal segmented image, the sagittal segmented image and the intermediate conversion segmented image to obtain a first centerline point corresponding to the coronal segmented image, a second centerline point corresponding to the sagittal segmented image and a third centerline point corresponding to the intermediate conversion segmented image, wherein the centerline points represent a point set of discrete distribution on a blood vessel centerline; Processing the first central line point, the second central line point and the third central line point by using a first double-projection matching optimization function and a first constraint condition to obtain a target space pose transformation vector, wherein the first double-projection matching optimization function is used for optimizing a total distance sum obtained based on a distance sum of squares between the first central line point and a converted third central line point and a distance sum of squares between the second central line point and a converted third central line point, and the first constraint condition is used for constraining a difference value between the total distance sums of two adjacent iterations to be smaller than a preset threshold; And performing pose conversion on the intermediate conversion segmentation image based on the target space pose conversion vector to obtain the target registration segmentation image.
3. The method of claim 2, wherein processing the first centerline point, the second centerline point, and the third centerline point using a first dual-projection matching optimization function and a first constraint to obtain a target spatial pose transformation vector comprises: Performing two-dimensional pixel point mapping on the third central line point based on a coronal plane projection matrix to obtain a plurality of first mapping points, wherein the coronal plane projection matrix is a ray intensity distribution matrix generated based on a rotary imaging device configured as a coronal plane rotation angle external parameter, and the first mapping points represent the mapping points of the third central line point on the coronal plane; Determining a first matching point corresponding to each of a plurality of first mapping points from the first centerline points based on a distance matching algorithm; Performing two-dimensional pixel point mapping on the third central line point based on a sagittal plane projection matrix to obtain a plurality of second mapping points, wherein the sagittal plane projection matrix represents a ray intensity distribution matrix generated based on a rotary imaging device configured as a sagittal plane rotation angle external parameter, and the second mapping points represent mapping points of the third central line point on a sagittal plane; Determining a second matching point corresponding to each of a plurality of second mapping points from the second centerline points based on a distance matching algorithm; and processing a plurality of first matching points, a plurality of second matching points, a plurality of first mapping points and a plurality of second mapping points by using a first double-projection matching optimization function and a first constraint condition to obtain a target space pose transformation vector.
4. A method according to claim 3, wherein said processing the plurality of first matching points, the plurality of second matching points, the plurality of first mapping points, and the plurality of second mapping points using the first dual projection matching optimization function and the first constraint condition to obtain the target spatial pose transformation vector comprises: Processing a spatial translation variable of the ith iteration by using a spatial translation function to obtain an ith translation matrix; processing a spatial rotation variable of the ith iteration by using a spatial rotation function to obtain an ith rotation matrix; Converting the plurality of first mapping points based on the ith translation matrix and the ith rotation matrix to obtain first conversion points corresponding to the plurality of first mapping points; Converting the plurality of second mapping points based on the ith translation matrix and the ith rotation matrix to obtain second conversion points corresponding to the plurality of second mapping points; determining a first distance square sum between a first matching point and a first conversion point corresponding to each of a plurality of first mapping points; determining a second distance square sum between a second matching point and a second conversion point corresponding to each of a plurality of second mapping points; obtaining a total distance square sum of the ith iteration according to the first distance square sum and the second distance square sum; and under the condition that the difference value between the sum of the squares of the total distances of the ith iteration and the sum of the squares of the total distances of the ith-1 th iteration is smaller than a preset threshold value, obtaining a target space pose transformation vector according to the space translation variable of the ith iteration and the space rotation variable of the ith iteration, wherein i is an integer larger than 1.
5. The method according to claim 2, wherein the method further comprises: Calculating the distance from the point to the blood vessel boundary by using a distance transformation algorithm aiming at any point in the third central line point to obtain a distance value corresponding to each of a plurality of points; obtaining the diameter of the target blood vessel according to the minimum distance value in the distance values; and obtaining the radius of the vascular inlet according to the distance value corresponding to the point with the minimum bottom distance of the intermediate conversion segmentation image.
6. The method of claim 2, wherein the method further comprises: For the ith point in the third centerline points, Determining a plurality of neighborhood points corresponding to the ith point; Processing a plurality of neighborhood points by using a fitting algorithm to obtain a fitting curve; When the fitting error of the fitting curve is smaller than a preset error threshold value, a curvature function is utilized to process the first derivative and the second derivative of the ith point, so that the curvature of the ith point is obtained; And under the condition that the curvature of the ith point is larger than the historical maximum curvature, updating the historical maximum curvature based on the curvature of the ith point to obtain a target curvature, wherein the historical maximum curvature represents the maximum value in the curvatures corresponding to the previous i-1 points, and i is a positive integer larger than 1.
7. The method of claim 1, wherein the performing voxel point mapping on the second magnetic resonance segmented image based on the first magnetic resonance segmented image to obtain an initial transformed segmented image mapped to spatial coordinates of the first magnetic resonance segmented image comprises: Based on a refinement algorithm, refining the first magnetic resonance segmentation image and the second magnetic resonance segmentation image respectively to obtain respective center line points of the first magnetic resonance segmentation image and the second magnetic resonance segmentation image; Processing respective center line points of the first magnetic resonance segmented image and the second magnetic resonance segmented image by using a first point matching optimization function and a second constraint condition to obtain an initial spatial pose transformation vector, wherein the first point matching optimization function is used for optimizing a sum of squares of distances between the center line points of the first magnetic resonance segmented image and the center line points of the converted second magnetic resonance segmented image, and the second constraint condition is used for constraining a difference value between the sum of squares of distances of two adjacent iterations to be smaller than a preset threshold value; Performing pose conversion on the second magnetic resonance segmented image based on the initial spatial pose conversion vector to obtain an initial mapping segmented image mapped to the first magnetic resonance segmented image under the spatial coordinates; And fusing the initial mapping segmentation image and the first magnetic resonance segmentation image to obtain the initial conversion segmentation image.
8. The method of claim 1, wherein the performing three-dimensional pixel point mapping on the initial transformed segmented image based on the three-dimensional contrast segmented image to obtain an intermediate transformed segmented image mapped to the spatial coordinates of the three-dimensional contrast segmented image comprises: Based on a refinement algorithm, refining the initial conversion segmentation image and the three-dimensional contrast segmentation image respectively to obtain respective center line points of the initial conversion segmentation image and the three-dimensional contrast segmentation image; Processing respective center line points of the initial conversion segmentation image and the three-dimensional contrast segmentation image by using a second point matching optimization function and a third constraint condition to obtain a middle space pose transformation vector, wherein the second point matching optimization function is used for optimizing the sum of squares of distances between the center line points of the three-dimensional contrast segmentation image and the center line points of the converted initial conversion segmentation image, and the third constraint condition is used for constraining the difference between the sum of squares of distances of two adjacent iterations to be smaller than a preset threshold; Performing pose conversion on the initial conversion segmentation image based on the intermediate space pose conversion vector to obtain an intermediate mapping segmentation image mapped to the space coordinates of the three-dimensional contrast segmentation image; And fusing the intermediate mapping segmentation image and the three-dimensional contrast segmentation image to obtain the intermediate conversion segmentation image.
9. A multi-modality image processing apparatus comprising: The position and orientation conversion module is used for carrying out position and orientation conversion on an intermediate conversion segmentation image representing the current state of the target object based on a coronal plane segmentation image and a sagittal plane segmentation image representing the historical state of the target object to obtain a target registration segmentation image representing the current state of the target object, wherein the coronal plane segmentation image and the sagittal plane segmentation image are respectively obtained by carrying out different-azimuth two-dimensional radiography on the target object based on a rotary imaging device, the intermediate conversion segmentation image is determined by carrying out three-dimensional pixel point mapping on a second magnetic resonance segmentation image based on a first magnetic resonance segmentation image to obtain an initial conversion segmentation image mapped to the first magnetic resonance segmentation image under the space coordinates of the first magnetic resonance segmentation image, the first magnetic resonance segmentation image and the second magnetic resonance segmentation image are obtained by carrying out three-dimensional information acquisition on the target object by utilizing different pulse mechanisms of the same magnetic resonance imaging device, and carrying out three-dimensional pixel point mapping on the initial conversion segmentation image based on the three-dimensional radiography segmentation image to obtain an intermediate conversion image mapped to the three-dimensional radiography under the space coordinates of the three-dimensional segmentation image, and the three-dimensional radiography image is obtained by carrying out three-dimensional imaging on the target imaging device based on the three-dimensional imaging device.
10. An electronic device, comprising: one or more processors; A memory for storing one or more computer programs, Characterized in that the one or more processors invoke the one or more computer programs to implement the steps of the method according to any of claims 1-8.

Description

Multi-mode image processing method and device and electronic equipment Technical Field The present invention relates to the field of medical image processing technologies, and in particular, to a method and an apparatus for processing a multi-mode image, and an electronic device. Background At present, when the intracranial arterial stenosis degree is high and the patient has obvious cerebral ischemic symptoms, the treatment is needed by intravascular interventional operation. However, due to the spatial shape tortuosity of the intracranial artery, the lesion form is complex, and an operator cannot intuitively observe the three-dimensional shape of the blood vessel under a two-dimensional view, so that the guide wire tip damages the wall of the blood vessel in the delivery process, and the difficulty and risk of the operation are increased. Disclosure of Invention In view of the above problems, the present invention provides a multi-mode image processing method, a device and an electronic apparatus. According to a first aspect of the invention, a multi-mode image processing method is provided, which comprises the steps of performing pose conversion on an intermediate conversion segmentation image representing a historical moment state of a target object based on a coronal segmentation image and a sagittal segmentation image representing the current state of the target object to obtain a target registration segmentation image representing the current state of the target object, wherein the coronal segmentation image and the sagittal segmentation image are respectively obtained by performing different-orientation two-dimensional radiography on the target object based on a rotary imaging device, the intermediate conversion segmentation image is determined by performing three-dimensional pixel point mapping on a second magnetic resonance segmentation image based on a first magnetic resonance segmentation image, obtaining an initial conversion segmentation image mapped to space coordinates of the first magnetic resonance segmentation image, performing three-dimensional information acquisition on the target object by using different pulse mechanisms of the same magnetic resonance imaging device by the first magnetic resonance segmentation image and the second magnetic resonance segmentation image, and performing three-dimensional pixel point mapping on the initial conversion segmentation image based on the three-dimensional imaging device to obtain an intermediate conversion image mapped to space coordinates of the three-dimensional radiography segmentation image, wherein the three-dimensional radiography imaging device is obtained by performing three-dimensional imaging on the target object based on the rotary imaging device. The second aspect of the invention provides a multi-mode image processing device, which comprises a pose conversion module, a target registration segmentation image and a three-dimensional contrast segmentation image, wherein the pose conversion module is used for carrying out pose conversion on an intermediate conversion segmentation image representing the historical moment state of a target object based on a coronal segmentation image and a sagittal segmentation image representing the current state of the target object to obtain a target registration segmentation image representing the current state of the target object, the coronal segmentation image and the sagittal segmentation image are respectively obtained by carrying out different-orientation two-dimensional contrast on the target object based on a rotary imaging device, the intermediate conversion segmentation image is determined by carrying out three-dimensional pixel point mapping on a second magnetic resonance segmentation image based on a first magnetic resonance segmentation image, obtaining an initial conversion segmentation image mapped to the space coordinates of the first magnetic resonance segmentation image, the first magnetic resonance segmentation image and the second magnetic resonance segmentation image are obtained by carrying out three-dimensional information acquisition on the target object by utilizing different pulse mechanisms of the same magnetic resonance imaging device, and carrying out three-dimensional pixel point mapping on the initial conversion image to obtain an intermediate conversion image mapped to the space coordinates of the three-dimensional contrast segmentation image, wherein the three-dimensional contrast imaging device carries out three-dimensional contrast imaging on the target object based on the three-dimensional contrast imaging device. A third aspect of the invention provides an electronic device comprising one or more processors and a memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method described above. According to the multi-mode image processing method, the device