CN-121999231-A - Lightweight image segmentation method and system based on NeuralODE neural network architecture

CN121999231ACN 121999231 ACN121999231 ACN 121999231ACN-121999231-A

Abstract

The invention discloses a lightweight image segmentation method and a lightweight image segmentation system based on NeuralODE neural network architecture, which comprise the steps of obtaining image data to be segmented, extracting an initial feature map through an initial convolution layer, generating an intermediate feature map through a separable convolution layer, inputting the intermediate feature map into an enhanced neural ordinary differential equation block, executing multi-step continuous feature evolution, introducing a preamble output feature weighting fusion at each step to obtain an evolved feature map, generating a main feature map and a derivative feature map, splicing the main feature map with a channel dimension, outputting a compressed feature map, decoupling teacher model probability distribution into two types of probability distribution by adopting a decoupling knowledge distillation strategy, calculating corresponding loss and constructing total distillation loss to adjust network parameters, inputting the feature map output by a trained network model into a segmentation pre-measurement head, and executing pixel-level prediction operation on the feature map through the segmentation pre-measurement head to generate a segmentation result of the image to be segmented. The invention improves the segmentation precision and gives consideration to the deployment efficiency and the segmentation performance.

Inventors

MA PENGFEI
HE TAO
XIONG JIANLONG
CAO YUE

Assignees

四川大学

Dates

Publication Date: 20260508
Application Date: 20260409

Claims (10)

1. The lightweight image segmentation method based on NeuralODE neural network architecture is characterized by comprising the following steps of: S1, obtaining image data to be segmented, inputting the image data to be segmented into an initial convolution layer, and extracting features of the image data to be segmented through the initial convolution layer to obtain an initial feature map; S2, inputting the initial feature map into a separable convolution layer, and performing feature deepening and downsampling on the initial feature map through the separable convolution layer to generate an intermediate feature map; S3, inputting the intermediate feature map into an enhanced neural ordinary differential equation block, and executing multi-step continuous feature evolution operation on the intermediate feature map by the enhanced neural ordinary differential equation block, wherein in each step of feature evolution process, the output features of the previous step are introduced and weighted fusion is carried out, so that a post-evolution feature map is obtained; S4, inputting the evolving feature map to a high-level parameter compression module, wherein the high-level parameter compression module carries out convolution operation on the evolving feature map through a main feature convolution unit to generate a main feature map, then carries out linear transformation on the main feature map through a derivative feature generation unit to generate a derivative feature map, finally carries out splicing on the main feature map and the derivative feature map in a channel dimension through a feature splicing unit to output a compressed feature map, and the high-level parameter compression module replaces standard point convolution operation in a network through reducing parameter quantity; S5, training a network model corresponding to the compressed feature map by adopting a decoupling knowledge distillation strategy, decoupling probability distribution output by a teacher model into target class probability distribution and non-target class probability distribution, respectively calculating target class knowledge distillation loss and non-target class knowledge distillation loss, constructing total distillation loss based on the target class knowledge distillation loss and the non-target class knowledge distillation loss, and adjusting network model parameters according to the total distillation loss; S6, inputting the feature image output by the trained network model to a segmentation pre-processing head, and performing up-sampling and pixel-level classification on the feature image through the segmentation pre-processing head to generate a segmentation result with the same resolution as the image to be segmented.
2. The method for lightweight image segmentation based on NeuralODE neural network architecture according to claim 1, wherein the updating rule of the enhanced neural ordinary differential equation block in S3 to perform multi-step continuous feature evolution operation on the intermediate feature map satisfies: , wherein, Represent the first The feature states in the step feature evolution process, Represent the first The feature states in the step feature evolution process, Represent the first A leachable time step in the evolution of the step feature, The activation function is represented as a function of the activation, Represent the first A learnable linear transformation matrix in the step feature evolution process, An initial feature map is shown as such, Representing feature extraction of input features and capturing semantic information of an image.
3. The method for segmenting a lightweight image based on NeuralODE neural network architecture according to claim 1, wherein in the step S3, in each step of feature evolution process, the calculation process of introducing the output features of the previous step and performing weighted fusion satisfies the following conditions: , wherein, Represent the first The activation value after weighted fusion in the evolution process of the step characteristics, The activation function is represented as a function of the activation, Represent the first The input features in the step feature evolution process, The weight coefficient is represented by a number of weight coefficients, Represent the first And outputting the characteristics after the activation function processing in the characteristic evolution process.
4. The method for segmenting a lightweight image based on NeuralODE neural network architecture according to claim 1, wherein when the main feature convolution unit performs convolution operation on the post-evolution feature map to generate a main feature map in S4, the number of channels of the main feature map satisfies: , wherein, The number of channels representing the main feature map, The number of channels representing the output feature map of the high-level parameter compression module, The compression ratio is represented by the number of compression ratios, Representing a round-up function.
5. The method for lightweight image segmentation based on NeuralODE neural network architecture according to claim 1, wherein the calculation process for constructing total distillation loss based on the target class knowledge distillation loss and the non-target class knowledge distillation loss in S5 is as follows: , wherein, Indicating the total distillation loss, A weight coefficient representing the target class knowledge distillation loss, Indicating the knowledge of the target class of distillation losses, Weight coefficients representing non-target class knowledge distillation losses, Representing non-target class knowledge distillation losses.
6. The lightweight image segmentation method based on NeuralODE neural network architecture according to claim 1, wherein when the target class knowledge distillation loss is calculated in S5, the calculation process of the target class knowledge distillation loss satisfies: , wherein, Indicating the knowledge of the target class of distillation losses, A set of target classes is represented and, Object class representing teacher model output The probability value of a class is determined, Object class representing student model output Probability values for classes.
7. The method for lightweight image segmentation based on NeuralODE neural network architecture as set forth in claim 1, wherein the step S3 includes the substeps of: S31, inputting the intermediate feature map to an input interface of an enhanced neural ordinary differential equation block, starting a feature evolution function of the enhanced neural ordinary differential equation block, and setting the total step number of feature evolution and an initial feature state; s32, at the first In the step feature evolution process, calculating the first feature through a feature evolution updating rule based on the initial feature state, a preset leachable linear transformation matrix and a leachable time step Feature states after feature evolution are performed, and meanwhile, the first step is The output characteristics processed by the activation function in the step characteristic evolution process are stored in a temporary cache unit; S33, at the first In the process of the evolution of the step features, Fetch the first from temporary cache Unit Output characteristics after being processed by an activation function in the step characteristic evolution process are matched with the first one The input features of the step feature evolution are weighted and fused according to a preset weight coefficient, and then the first step of the weighted fusion is combined Step learning linear transformation matrix and step learning time length, and calculating the first step through characteristic evolution updating rule Feature states after step feature evolution and the first step Updating the output characteristics processed by the activating function to a temporary cache unit in the characteristic evolution process; S34, repeatedly executing the step S33 until the feature evolution operation of the preset total steps is completed, and outputting the feature state after the feature evolution of the last step as an evolving feature map.
8. The method for lightweight image segmentation based on NeuralODE neural network architecture as set forth in claim 1, wherein the S4 includes the substeps of: s41, inputting the evolving feature map to a main feature convolution unit, determining the number and the size of convolution kernels according to a preset compression ratio by the main feature convolution unit, carrying out convolution operation on the evolving feature map, and carrying out feature extraction on different areas of the evolving feature map through sliding convolution check in the convolution operation process to generate a main feature map; s42, inputting the main feature map to a derivative feature generation unit, wherein the derivative feature generation unit adopts a linear transformation kernel with a preset size to respectively execute linear transformation operation on each channel of the main feature map, and generates a derivative feature map without cross-channel feature interaction in the linear transformation process of each channel; S43, inputting the main feature map and the derivative feature map to a feature splicing unit, wherein the feature splicing unit determines a splicing sequence of channel dimensions, stacks the main feature map and the derivative feature map in the channel dimensions according to the sequence, enables the number of channels of the spliced feature map to meet the requirement of a preset output channel number, and generates a compressed feature map; s44, verifying the channel dimension of the compressed feature map, and outputting the compressed feature map to the step S5 after confirming that the channel number meets the subsequent network processing requirement.
9. The method for lightweight image segmentation based on NeuralODE neural network architecture as set forth in claim 1, wherein the step S5 includes the substeps of: s51, acquiring a pre-trained teacher model and a student model to be trained, and respectively inputting image data of a training data set into the teacher model and the student model to obtain probability distribution output by the teacher model and probability distribution output by the student model; s52, respectively extracting target class probability distribution and non-target class probability distribution from probability distribution output by the teacher model and probability distribution output by the student model, and carrying out normalization processing on the non-target class probability distribution to ensure that the probability sum of the non-target class probability distribution is 1; s53, calculating target class knowledge distillation loss by adopting a cross entropy loss function based on target class probability distribution, and calculating non-target class knowledge distillation loss by adopting the cross entropy loss function based on normalized non-target class probability distribution; And S54, carrying out weighted summation on the target class knowledge distillation loss and the non-target class knowledge distillation loss according to a preset weight coefficient to obtain total distillation loss, feeding back the total distillation loss to the student model by using a back propagation algorithm, adjusting parameters of the student model, and repeatedly executing the steps S51-S54 until the training of the student model reaches the preset iteration times.
10. The lightweight image segmentation system based on NeuralODE neural network architecture is characterized by being applied to the lightweight image segmentation method based on NeuralODE neural network architecture according to claim 1, and comprises an image data input and initial feature extraction unit, a segmentation unit and a segmentation unit, wherein the image data input and initial feature extraction unit is used for receiving externally input image data to be segmented, carrying out convolution operation on the image data to be segmented through a built-in initial convolution layer, extracting basic feature information of an image, generating an initial feature map, and transmitting the initial feature map to an intermediate feature deepening and downsampling unit; the intermediate feature deepening and downsampling unit is connected with the image data input and initial feature extraction unit, receives the initial feature map, performs multi-round feature deepening processing on the initial feature map through a built-in separable convolution layer, simultaneously performs downsampling operation to reduce the spatial resolution of the feature map, generates an intermediate feature map, and transmits the intermediate feature map to the multi-step continuous feature evolution and fusion unit; the multi-step continuous feature evolution and fusion unit is connected with the intermediate feature deepening and downsampling unit, receives the intermediate feature map, executes multi-step continuous feature evolution operation through a built-in enhanced neural ordinary differential equation block, invokes output features of the previous step in each step of evolution process to perform weighted fusion, generates a post-evolution feature map, and transmits the post-evolution feature map to the high-level feature compression and splicing unit, the high-level feature compression and splicing unit is connected with the multi-step continuous feature evolution and fusion unit, receives the post-evolution feature map, generates a main feature map through a built-in main feature convolution subunit, generates a derivative feature map through a derivative feature generation subunit, the method comprises the steps of obtaining a main feature image, obtaining a derivative feature image, obtaining a channel dimension, obtaining a compressed feature image, transmitting the compressed feature image to a network model training and parameter adjustment unit through a feature splicing subunit, obtaining a network model training and parameter adjustment unit, connecting the compressed feature image with a high-level feature compression and splicing unit, receiving the compressed feature image, decoupling teacher model output probability distribution into target class and non-target class probability distribution through a decoupling knowledge distillation strategy, calculating two classes of losses, constructing total distillation loss, adjusting network model parameters through back propagation based on the total distillation loss, obtaining a trained network model, transmitting the trained network model to a segmentation result generation unit, connecting the network model training and parameter adjustment unit, receiving the trained network model, inputting the feature image processed by a preamble unit to the network model, inputting the feature image output by the network model to a segmentation pre-measuring head, executing pixel-level prediction operation on the feature image through the segmentation pre-measuring head, obtaining a segmentation result of the image to be segmented, and outputting the segmentation result to an external storage or display device.

Description

Lightweight image segmentation method and system based on NeuralODE neural network architecture Technical Field The invention relates to the technical field of image segmentation, in particular to a lightweight image segmentation method and system based on NeuralODE neural network architecture. Background In the image segmentation process, along with the improvement of the requirements of practical applications such as medical image analysis, intelligent monitoring and the like on the model deployment efficiency, a lightweight neural network architecture becomes a research and development focus. Although the traditional convolutional neural network can realize higher segmentation precision, the traditional convolutional neural network has huge parameter and calculation amount and is difficult to adapt to resource-limited equipment, and the architecture based on the transducer strengthens the global feature modeling capability, but further aggravates the calculation resource consumption. The subsequent lightweight schemes reduce parameters by simplifying the convolution structure, but the characteristic expression capability is insufficient. In recent years, a neural ordinary differential equation (NeuralODE) provides a new direction for lightweight design by means of continuous characteristic evolution characteristics, so that the problems of poor characteristic evolution stability, network parameter redundancy and the like of the existing segmentation model still exist, the model weight and segmentation performance cannot be fully balanced, and the development of a novel architecture with efficient characteristic processing and parameter optimization is promoted. The prior art has two obvious disadvantages in this field. On the one hand, the feature evolution stability is insufficient, and when the model based on NeuralODE is used for executing multi-step feature evolution operation, the output features of the preamble steps are not effectively introduced for fusion, so that feature information is easy to lose or distort in the evolution process, continuous and robust feature expression is difficult to form, and the accuracy of a final segmentation result is affected. On the other hand, the high-level parameter compression efficiency is low, most lightweight models only aim at the low level of the network to simplify parameters, the high-level network still depends on the traditional convolution structure, the efficient compression of parameters is not realized through the collaborative generation of main features and derivative features and channel splicing, high-level parameter redundancy is caused, the storage and calculation burden of the models is increased, and the deployment capacity of the models in a resource limited scene is restricted. Disclosure of Invention In order to overcome the defects and shortcomings in the prior art, the invention provides a lightweight image segmentation method and system based on NeuralODE neural network architecture. S1, obtaining image data to be segmented, inputting the image data to be segmented into an initial convolution layer, and extracting features of the image data to be segmented through the initial convolution layer to obtain an initial feature map; S2, inputting the initial feature map into a separable convolution layer, and performing feature deepening and downsampling on the initial feature map through the separable convolution layer to generate an intermediate feature map; S4, inputting the evolving feature map into a high-level parameter compression module, firstly, carrying out convolution operation on the evolving feature map through a main feature convolution unit to generate a main feature map, then carrying out linear transformation on the main feature map through a derived feature generation unit to generate a derived feature map, finally, carrying out channel dimension splicing on the main feature map and the derived feature map through a feature splicing unit to output a compressed feature map, S5, training a network model corresponding to the compressed feature map by adopting a decoupling knowledge distillation strategy, decoupling the probability distribution output by a teacher model into a target class probability distribution and a non-target class probability distribution, respectively calculating target class knowledge distillation loss and non-target class knowledge distillation loss, constructing a non-target class knowledge distillation loss and a non-target class total distillation loss based on the target class probability distribution and the non-target class knowledge distillation loss, s6, inputting the feature image output by the trained network model to a segmentation pre-measurement head, and carrying out up-sampling and pixel level classification on the feature image by the segmentation pre-measurement head to generate a segmentation result with the same resolution as the image to be segmented. Further, the updating rule of