Search

CN-117058082-B - Bone mineral density prediction method and system based on knowledge distillation

CN117058082BCN 117058082 BCN117058082 BCN 117058082BCN-117058082-B

Abstract

A bone density prediction method and system based on knowledge distillation are provided, wherein the method is characterized in that image features of an X-ray flat sheet and a CT are respectively extracted through a computer vision technology, 2D image features of the X-ray flat sheet and 3D image features of the CT in a teacher model are mapped to the same feature space through a deep learning technology, the 2D image features of the X-ray flat sheet and the 3D image features of the CT in the same feature space in the teacher model are spliced and fused to be used as fusion features, distillation is carried out between the fusion features of the teacher model and the 2D image features of the X-ray flat sheet of a student model through a fusion feature and knowledge distillation method, and distillation loss is calculated, and a bone density prediction regression device of the teacher model and the student model is trained jointly through an online distillation mode. The method is beneficial to predicting the bone density value by using X-ray flat data in the prediction stage, helps patients to screen osteoporosis earlier, reduces the cost of osteoporosis screening and reduces the risk of fracture.

Inventors

  • BU JIAJUN
  • Qi Zhongda
  • YU ZHI
  • Shi Keyue
  • YANG JUNYAO

Assignees

  • 浙江大学

Dates

Publication Date
20260512
Application Date
20230803

Claims (6)

  1. 1. A bone mineral density prediction method based on knowledge distillation, comprising the steps of: S110, respectively extracting image features of an X-ray flat sheet and an electronic computer tomography CT through a computer vision technology, wherein a teacher model contains two image features, and a student model only has the image features of the X-ray flat sheet; s120, mapping the 2D image features of the X-ray flat and the 3D image features of the CT in the teacher model to the same feature space by using a deep learning technology; s130, splicing and fusing the 2D image features of the X-ray flat sheet and the 3D image features of the CT in the same feature space in the teacher model to obtain fusion features; s140, distilling the fusion characteristic of the teacher model and the 2D image characteristic of the student model X-ray flat sheet by using a fusion characteristic and knowledge distillation method and calculating distillation loss, wherein the method comprises the following steps: s1401, through designing a multi-layer perceptron 2D image feature vector of X-ray flat sheet in student model Mapping to fused feature vectors In the same feature space and in the same dimension to obtain feature vectors I.e. S1402, fusing characteristics of the teacher model by knowledge distillation method Distilled into student model Thus, the student model only needs to use the data of the X-ray flat sheet in the prediction process, and the distillation loss is calculated through the average square error loss function MSE I.e. ; S150, training a bone density prediction regressor of a teacher model and a student model together in an online distillation mode, wherein the bone density prediction regressor comprises: S1501, designing teacher model to fuse feature vectors Multi-layer perceptron regressor for bone mineral density prediction as input ; S1502, utilize this multilayer perceptron regressor Based on the fused feature vector Obtaining a predicted value of bone mineral density I.e. ; S1503, designing a student model after processing by using X-ray flat feature vectors Multi-layer perceptron regressor for bone mineral density prediction as input ; S1504, utilizing the multi-layer perceptron regressor Processed according to X-ray flat feature vector Obtaining a predicted value of bone mineral density I.e. ; S1505, the true bone density y and the true bone density y are calculated by the mean square error loss function MSE Loss between I.e. ( )、 ( ); S1506, a weighting loss method is designed to combine three kinds of weighting loss methods The model is trained as the final penalty.
  2. 2. The knowledge-based bone density prediction method according to claim 1, wherein step S110 comprises: S1101, extracting image feature map of X-ray flat by using 2D-ResNet network model pre-trained on natural image dataset ; S1102, extracting an image feature map of CT data by using a pre-trained 3D-ResNet network model on a medical image dataset ; S1103, obtaining feature vectors from the extracted image feature images through a 2D global average pooling and 3D global average pooling technology And 。
  3. 3. The knowledge-based bone density prediction method according to claim 1, wherein step S120 comprises: s1201, through designing a multi-layer perceptron Feature vector in teacher model Mapping to a certain feature space to obtain feature vector I.e. S1202, designing a multi-layer perceptron Feature vector in teacher model Mapping to Feature space to obtain feature vector I.e.
  4. 4. The knowledge-based bone density prediction method according to claim 1, wherein step S130 comprises: s1301, feature vector of X-ray flat sheet Feature vector with CT Performing splicing operation to obtain feature vectors =[ ]; S1302, through designing a multi-layer perceptron Will be And Simply spliced feature vector Further fusing to obtain feature vector I.e.
  5. 5. The knowledge-based bone density prediction method according to claim 1, wherein step S1506 includes: (1) Will be Giving equal weight learning; (2) By means of learnable parameters To give Weight, parameters Normalized to 0-1, and the rest weight gives # - ); (3) Final loss of 。
  6. 6. A knowledge distillation-based bone density prediction system, comprising: the image feature extraction module is used for respectively extracting image features of the X-ray flat sheet and the electronic computer tomography CT through a computer vision technology, the teacher model contains two image features, and the student model only has the image features of the X-ray flat sheet; The image feature mapping module is used for mapping the 2D image features of the X-ray flat sheet and the 3D image features of the CT in the teacher model to the same feature space by using a deep learning technology; the image characteristic splicing and fusing module is used for splicing and fusing 2D image characteristics of the X-ray flat sheet and 3D image characteristics of CT on the same characteristic space in a teacher model to be used as fusion characteristics, and comprises the following steps: s1401, through designing a multi-layer perceptron 2D image feature vector of X-ray flat sheet in student model Mapping to fused feature vectors In the same feature space and in the same dimension to obtain feature vectors I.e. S1402, fusing characteristics of the teacher model by knowledge distillation method Distilled into student model Thus, the student model only needs to use the data of the X-ray flat sheet in the prediction process, and the distillation loss is calculated through the average square error loss function MSE I.e. ; The distillation module is used for distilling the fusion characteristics of the teacher model and the 2D image characteristics of the X-ray flat piece of the student model by utilizing a fusion characteristic and knowledge distillation method and calculating distillation loss, and comprises the following steps: S1501, designing teacher model to fuse feature vectors Multi-layer perceptron regressor for bone mineral density prediction as input ; S1502, utilize this multilayer perceptron regressor Based on the fused feature vector Obtaining a predicted value of bone mineral density I.e. ; S1503, designing a student model after processing by using X-ray flat feature vectors Multi-layer perceptron regressor for bone mineral density prediction as input ; S1504, utilizing the multi-layer perceptron regressor Processed according to X-ray flat feature vector Obtaining a predicted value of bone mineral density I.e. ; S1505, the true bone density y and the true bone density y are calculated by the mean square error loss function MSE Loss between I.e. ( )、 ( ); S1506, a weighting loss method is designed to combine three kinds of weighting loss methods Training the model as a final loss; and the bone density prediction regression training module is used for training the bone density prediction regression of the teacher model and the student model together in an online distillation mode.

Description

Bone mineral density prediction method and system based on knowledge distillation Technical Field The invention relates to the field of computer vision, in particular to a bone density prediction method and system based on knowledge distillation. Background Osteoporosis is a skeletal systemic disease, the bone density and bone quality are reduced, the bone microstructure is destroyed, and the bone fragility is increased, so that the fracture is more easy to occur, the incidence rate of pulmonary infection is increased, and the like. Bone density is the first method to diagnose osteoporosis, and by measuring bone density, osteoporosis can be diagnosed. The X-ray flat plate is an imaging mode with low cost and wide application. The bone mineral density value predicted from the X-ray flat data can help patients to carry out early screening, help patients to screen osteoporosis earlier, reduce the cost of osteoporosis screening and reduce the risk of fracture. The existing related technology is lacking in the current problem of predicting bone density values from X-ray flat sheets, and meanwhile, the existing data of hospitals cannot be fully utilized. Disclosure of Invention The present invention overcomes the above-identified shortcomings of the prior art by providing a method and system for bone density prediction based on knowledge distillation. In order to solve the technical problems, the invention provides a bone density prediction method based on knowledge distillation, which comprises the following steps: S110, respectively extracting image features of an X-ray flat sheet and CT (computed tomography) through a computer vision technology, wherein a teacher model contains two image features, and a student model only has the image features of the X-ray flat sheet; S120, mapping the 2D image features of the X-ray flat and the 3D image features of the CT in the teacher model to the same feature space by using a deep learning technology. S130, splicing and fusing the 2D image features of the X-ray flat sheet and the 3D image features of the CT in the same feature space in the teacher model to obtain fusion features. And S140, distilling the fusion characteristics of the teacher model and the 2D image characteristics of the X-ray flat piece of the student model by using a fusion characteristic and knowledge distillation method, and calculating distillation loss. S150, training a bone density prediction regressor of the teacher model and the student model together in an online distillation mode. Further, in step S110, the image features of the X-ray flat slice and the CT (computed tomography) are extracted respectively by the computer vision technology, the teacher model contains two image features, and the student model has only the image features of the X-ray flat slice, and specifically includes: S1101, extracting an image characteristic graph g x of the X-ray flat by using a 2D-ResNet network model pre-trained on a natural image data set. S1102, extracting an image feature map g ct of CT data by using a 3D-ResNet network model pre-trained on the medical image dataset. And S1103, respectively obtaining feature vectors v x and v ct from the extracted image feature map through a 2D global average pooling technology and a 3D global average pooling technology. Further, in step S120, the mapping the 2D image feature of the X-ray flat slice and the 3D image feature of the CT in the teacher model to the same feature space by using the deep learning technique specifically includes: S1201, the feature vector v fx, i.e. v fr＝fx(vx, is obtained by designing the multi-layer perceptron f x to map the feature vector v x to a certain feature space in the teacher model. S1202, the feature vector v fct, namely v fct＝fct(vct, is obtained by designing the multi-layer perceptron f ct to map the feature vector v ct to the v fx feature space in the teacher model. Further, in step S130, the fusion of the 2D image features of the X-ray flat slice and the 3D image features of the CT in the same feature space in the teacher model, which are used as fusion features, specifically includes: S1301, the characteristic vector v fx of the X-ray flat sheet and the characteristic vector v fct of the CT are spliced to obtain a characteristic vector v r＝[vfx,vfct. S1302, the feature vector v r obtained by simply splicing v fx and v fct is further fused by designing the multi-layer perceptron f r to obtain a feature vector v fr, namely v fr＝fr(vr. Further, the method of distilling the fusion feature and knowledge in step S140 distills the fusion feature of the teacher model and the 2D image feature of the student model X-ray flat sheet and calculates a distillation loss, which specifically includes: S1401, mapping the 2D image feature vector v x of the X-ray flat in the student model into the same feature space of the fusion feature vector v fr by designing a multi-layer perceptron f sx, and obtaining a feature vector v sx, namely v sx＝fsx(vx,