CN-116229070-B - Image segmentation method, device, electronic equipment and storage medium

CN116229070BCN 116229070 BCN116229070 BCN 116229070BCN-116229070-B

Abstract

The application discloses an image segmentation method, an image segmentation device, electronic equipment and a storage medium, which are applied to the field of image processing and comprise the steps of obtaining real-time parameters of second segmentation models corresponding to a plurality of second clients, obtaining a batch standardization layer according to a preset first segmentation model, carrying out dynamic sequencing and cyclic knowledge distillation on the first segmentation model and the second segmentation model according to the first segmentation model and the real-time parameters to obtain a general model, carrying out personalized training on the model of the first clients according to the general model, the batch standardization layer and the real-time parameters to obtain an image segmentation model, obtaining an image to be segmented, and carrying out image segmentation on the image to be segmented through the image segmentation model to obtain a segmentation result. According to the application, the image segmentation is realized by a federal learning mode of cyclic knowledge distillation and personalized training, the effective information of other site models can be effectively utilized, excellent image segmentation performance is obtained, the performance of the image segmentation models of a plurality of clients is improved, and the image segmentation effect is improved.

Inventors

TANG XIAOYING
LIN LI
LIU YIXIANG
WU JIEWEI

Assignees

南方科技大学

Dates

Publication Date: 20260512
Application Date: 20230221

Claims (9)

1. An image segmentation method, applied to a first client, the first client being communicatively connected to a plurality of second clients, the image segmentation method comprising: acquiring real-time parameters of a second segmentation model corresponding to a plurality of second clients; obtaining a batch standardization layer according to a preset first segmentation model; dynamically sequencing the first segmentation model and the second segmentation model according to the first segmentation model and the real-time parameters and circularly distilling knowledge to obtain a general model; performing personalized training on the model of the first client according to the general model, the batch standardization layer and the real-time parameters to obtain an image segmentation model; obtaining an image to be segmented, and passing the image to be segmented through the image segmentation model to obtain a segmentation result of the image to be segmented; The real-time parameters include performance parameters of the second segmentation model, and the dynamic sequencing and the cyclic knowledge distillation are performed on the first segmentation model and the second segmentation model according to the first segmentation model and the real-time parameters to obtain a general model, which includes: Acquiring sample image data, and inputting the sample image data into the first segmentation model to obtain performance parameters and weak supervision segmentation loss of the first segmentation model; Dynamically sequencing the first segmentation model and the second segmentation model according to the performance parameters of the first segmentation model and the second segmentation model; Determining a teacher model of the first segmentation model according to the dynamic sequencing result; obtaining knowledge distillation loss of the first segmentation model according to the first segmentation model and the teacher model; Training the first segmentation model according to the knowledge distillation loss and the weak supervision segmentation loss until the performance parameters of the first segmentation model meet a first preset condition; and obtaining a general model according to the first segmentation model after training is finished.
2. The image segmentation method as set forth in claim 1, wherein the inputting the sample image data into the first segmentation model results in a performance parameter of the first segmentation model, comprising: Acquiring a real label of the sample image data; Inputting the sample image data into the first segmentation model to obtain predicted image data and normalized exponential probability vectors; obtaining a set similarity measurement function according to the predicted image data and the real label; Obtaining a prediction entropy of the first segmentation model according to the normalized exponential probability vector, wherein the prediction entropy is used for representing uncertainty of the first segmentation model; and obtaining the performance parameters of the first segmentation model according to the set similarity measurement function and the uncertainty.
3. The image segmentation method as set forth in claim 1, wherein the inputting the sample image data into the first segmentation model results in a weak supervised segmentation loss of the first segmentation model, comprising: obtaining a corresponding sparse label according to the sample image data; inputting the sample image data into the first segmentation model to obtain predicted image data, predicted probability and image selected characteristics, wherein the image selected characteristics are image characteristics of a preset layer of the sample image data passing through the first segmentation model; performing tree affinity generation on the sample image data and the image selected features to obtain a low-order affinity matrix and a high-order affinity matrix; The low-order affinity matrix and the high-order affinity matrix pass through a cascade filter to obtain a soft pseudo tag corresponding to the sample image data; Obtaining tree energy loss according to the prediction probability and the soft pseudo tag; obtaining partial cross entropy loss according to the predicted image data and the sparse label; Obtaining a gating conditional random field loss according to the prediction probability, a preset source mask and a preset target mask; And obtaining the weak supervision segmentation loss of the first segmentation model according to the tree energy loss, the partial cross entropy loss and the gating conditional random field loss.
4. The image segmentation method according to claim 1, wherein the performing personalized training on the model of the first client according to the generic model, the batch normalization layer and the real-time parameter to obtain an image segmentation model comprises: Initializing a model of the first client according to the general model and the batch standardization layer to obtain a third segmentation model; Obtaining a similarity weight matrix according to the third segmentation model and the real-time parameters; Obtaining a teacher model of the third segmentation model according to the third segmentation model and the similarity weight matrix; obtaining knowledge distillation loss and weak supervision segmentation loss of the third segmentation model according to the third segmentation model and the teacher model; training the third segmentation model according to the knowledge distillation loss and the weak supervision segmentation loss until the performance parameters of the third segmentation model meet a second preset condition; and obtaining the image segmentation model according to the third segmentation model after training is finished.
5. The image segmentation method as set forth in claim 4, wherein the real-time parameters include statistics of the second segmentation model, and the obtaining a similarity weight matrix according to the third segmentation model and the real-time parameters includes: obtaining statistics of the third segmentation model according to the third segmentation model; obtaining optimal transmission distances between the third segmentation model and a plurality of second segmentation models according to statistics of the third segmentation model and the second segmentation model; Obtaining the similarity between the third segmentation model and the second segmentation models according to the optimal transmission distances; sequentially carrying out normalization and moving average update on the similarity to obtain a similarity weight; And obtaining a similarity weight matrix according to the similarity weights.
6. The image segmentation method according to claim 4, wherein the obtaining the teacher model of the third segmentation model according to the third segmentation model and the similarity weight matrix includes: obtaining batch standardized layer parameters and other layer parameters of the third segmentation model according to the third segmentation model; Obtaining approval layer parameters of the teacher model according to the batch standardization layer parameters of the third segmentation model; obtaining other layer parameters of the teacher model according to the other layer parameters of the third segmentation model and the similarity weight matrix; and obtaining the teacher model corresponding to the third segmentation model according to the approval layer parameters and other layer parameters of the teacher model.
7. An image segmentation apparatus for use with a first client communicatively coupled to a plurality of second clients, comprising: the data communication module is used for acquiring real-time parameters of a plurality of second segmentation models corresponding to the second clients; the distribution processing module is used for obtaining a batch standardization layer according to a preset first segmentation model; The general training module is used for carrying out dynamic sequencing and cyclic knowledge distillation on the first segmentation model and the second segmentation model according to the first segmentation model and the real-time parameters to obtain a general model; the personalized training module is used for carrying out personalized training on the model of the first client according to the general model, the batch standardization layer and the real-time parameters to obtain an image segmentation model; the image segmentation module is used for acquiring an image to be segmented, and passing the image to be segmented through the image segmentation model to obtain a segmentation result of the image to be segmented; The real-time parameters include performance parameters of the second segmentation model, and the dynamic sequencing and the cyclic knowledge distillation are performed on the first segmentation model and the second segmentation model according to the first segmentation model and the real-time parameters to obtain a general model, which includes: Acquiring sample image data, and inputting the sample image data into the first segmentation model to obtain performance parameters and weak supervision segmentation loss of the first segmentation model; Dynamically sequencing the first segmentation model and the second segmentation model according to the performance parameters of the first segmentation model and the second segmentation model; Determining a teacher model of the first segmentation model according to the dynamic sequencing result; obtaining knowledge distillation loss of the first segmentation model according to the first segmentation model and the teacher model; Training the first segmentation model according to the knowledge distillation loss and the weak supervision segmentation loss until the performance parameters of the first segmentation model meet a first preset condition; and obtaining a general model according to the first segmentation model after training is finished.
8. An electronic device comprising a memory storing a computer program and a processor implementing the image segmentation method according to any one of claims 1 to 6 when the computer program is executed by the processor.
9. A computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the image segmentation method of any one of claims 1 to 6.

Description

Image segmentation method, device, electronic equipment and storage medium Technical Field The present invention relates to the field of image processing, and in particular, to an image segmentation method, an image segmentation device, an electronic device, and a storage medium. Background Image segmentation is a representative task supporting computer-aided image content analysis, and for medical images, image segmentation can not only identify lesion categories, but also locate specific areas, playing an important role in clinical diagnosis. In the prior art, a federal learning method is generally adopted for image segmentation, and the federal learning can help users to realize common modeling on the basis of ensuring the security of data privacy, so that the performance of a model is enhanced. However, because of unavoidable distribution differences of data among stations, the performance of the image segmentation method based on federal learning is generally deteriorated when different data distributions of different stations are encountered, and the data differences among the stations cause the stations to deviate, so that the image segmentation effect is poor. Disclosure of Invention The application provides an image segmentation method, an image segmentation device, electronic equipment and a storage medium, which can effectively improve the performance of an image segmentation model of each site. In a first aspect, the present application provides an image segmentation method applied to a first client, where the first client is communicatively connected to a plurality of second clients, the image segmentation method including: acquiring real-time parameters of a second segmentation model corresponding to a plurality of second clients; obtaining a batch standardization layer according to a preset first segmentation model; dynamically sequencing the first segmentation model and the second segmentation model according to the first segmentation model and the real-time parameters and circularly distilling knowledge to obtain a general model; performing personalized training on the model of the first client according to the general model, the batch standardization layer and the real-time parameters to obtain an image segmentation model; And obtaining an image to be segmented, and passing the image to be segmented through the image segmentation model to obtain a segmentation result of the image to be segmented. According to the image segmentation method provided by the embodiment of the application, at least the following beneficial effects are that in the training process of the image segmentation model, a batch standardization layer is obtained firstly according to a preset first segmentation model so as to keep individuation of local data distribution, then the segmentation models of a first client and a plurality of second clients are dynamically sequenced, and the sequenced models are accumulated through a circulating knowledge distillation mode so as to obtain a general model with common knowledge of each client, the models of the first client are individuated and trained according to real-time parameters of the batch standardization layer, the general model and the second segmentation model so as to obtain the image segmentation model corresponding to the first client. According to some embodiments of the first aspect of the present application, the real-time parameters include performance parameters of the second segmentation model, and the dynamic ordering and cyclic knowledge distillation of the first segmentation model and the second segmentation model according to the first segmentation model and the real-time parameters, to obtain a generic model, includes: Acquiring sample image data, and inputting the sample image data into the first segmentation model to obtain performance parameters and weak supervision segmentation loss of the first segmentation model; Dynamically sequencing the first segmentation model and the second segmentation model according to the performance parameters of the first segmentation model and the second segmentation model; Determining a teacher model of the first segmentation model according to the dynamic sequencing result; obtaining knowledge distillation loss of the first segmentation model according to the first segmentation model and the teacher model; Training the first segmentation model according to the knowledge distillation loss and the weak supervision segmentation loss until the performance parameters of the first segmentation model meet a first preset condition; and obtaining a general model according to the first segmentation model after training is finished. According to some embodiments of the first aspect of the present application, the inputting the sample image data into the first segmentation model, to obtain the performance parameter of the first segmentation model, includes: Acquiring a real label of the sample image data; Inputting the sample im