Search

CN-122023958-A - Plant fiber electron microscope image generation method and related equipment based on diffusion model

CN122023958ACN 122023958 ACN122023958 ACN 122023958ACN-122023958-A

Abstract

The embodiment of the application provides a plant fiber electron microscope image generation method and related equipment based on a diffusion model, belonging to the field of computer vision and image generation. The method comprises the steps of collecting and labeling an image data set containing fiber categories and magnification, preprocessing and enhancing the data, constructing a conditional noise prediction network, processing continuous magnification conditions through a Fourier coding layer based on logarithmic coordinates by adopting a UNet architecture, simultaneously deeply fusing multi-mode condition information through a feature map affine modulation mechanism, training the network based on a correction flow frame to learn a vector field from noise to a clean image, and carrying out iterative denoising by utilizing a classifier-free multi-condition guiding technology in an reasoning stage to generate a high-quality fiber electron microscope image conforming to the designated categories and the magnification. The application has the advantages of high generation efficiency, accurate continuous condition control, strong multi-condition fusion depth and the like, and can provide an efficient data enhancement tool for fiber science research and industrial application.

Inventors

  • He Sizhe
  • ZHOU JIAO
  • HUANG YAN
  • LI HAILONG
  • XU YONG

Assignees

  • 华南理工大学

Dates

Publication Date
20260512
Application Date
20251219

Claims (10)

  1. 1. The plant fiber electron microscope image generation method based on the diffusion model is characterized by comprising the following steps of: The method comprises the steps of data preparation, namely acquiring a data set containing plant fiber electron microscope images and labels, wherein the labels at least comprise fiber types and magnification; data preprocessing, namely dividing the data set into a training set and a testing set, and preprocessing an image; constructing a noise prediction network based on a UNet network architecture, wherein the input of the network comprises a noise image, a diffusion time step, a fiber class condition and an amplification factor condition; model training, namely training the noise prediction network based on a correction flow frame by taking a vector field from a noise-added image to an original clean image as a target, and storing network parameters; And (3) image generation, namely loading a trained network, carrying out multi-round iterative denoising by combining no classifier guidance from random noise, and generating a plant fiber electron microscope image which accords with the specified fiber category and magnification condition.
  2. 2. The method of claim 1, wherein the step of data preparation comprises: Preparing a plant fiber sample, wherein the plant fiber sample comprises the steps of pretreatment, dissociation, cleaning and freeze drying; And observing the sample by using a scanning electron microscope, and respectively acquiring images at a plurality of different magnifications in a plurality of areas to construct an original image set.
  3. 3. The method of claim 1, wherein the step of preprocessing the data comprises: Randomly cutting the image, and updating the magnification marking value of the corresponding image according to the proportional relation between the cutting area and the original image; scaling the cut image to a uniform size, and carrying out random horizontal overturning according to a preset probability; And carrying out normalization processing on the pixel values of the image.
  4. 4. The method of claim 1, wherein the constructed noise prediction network is embodied as: the noise prediction network accepts four inputs, a noisy image Fiber class label Magnification value Diffusion time step ; The device is provided with a category embedding module for labeling discrete fiber categories Mapping to category feature vectors ; The device is provided with a magnification coding module, adopts continuous Fourier coding based on logarithmic coordinates, and adopts a magnification value Mapping to a magnification feature vector ; A time step mapping module is arranged to step the diffusion time Mapping to time step feature vectors ; The category feature vector is set Feature vector of magnification And time step feature vector Splicing to obtain uniform condition characterization ; The UNet network comprises an encoder, a middle block and a decoder, wherein each layer of the encoder and the decoder is provided with a corresponding characteristic modulation module; The characteristic modulation module is characterized by the aforementioned conditions For input, predicting by at least one multi-layer perceptron to obtain scaling parameters for affine transformation of the current layer output feature map Bias parameter To achieve deep modulation of the network profile by the condition information.
  5. 5. The method of claim 4, wherein the log-based continuous fourier code is calculated as follows: Wherein, the Is a feature vector Is used in the manufacture of a printed circuit board, Is a preset or randomly sampled frequency weight parameter.
  6. 6. The method according to claim 1, wherein the specific step of model training comprises: forward noise addition procedure for clean images Adding noise by linear interpolation Obtaining a noise-added image Wherein , Obeying a standard gaussian distribution; loss function calculation, wherein the training target of the noise prediction network is a prediction vector field Make it approach to the true vector field The loss function is defined as the L2 norm between the two: ; Random conditional discarding, in which the input fibre class conditions are independently conditioned with a certain probability during training Or magnification condition And setting zero so that the network learns the condition generation and the unconditional generation simultaneously.
  7. 7. The method according to claim 1, wherein the specific step of image generation comprises: A1 sampling from a standard Gaussian distribution to obtain an initial noise image ; A2, setting the target fiber class And target magnification And sets class guidance weights Magnification guidance weight ; A3, for each step in the denoising process, calculating the following output: Unconditional output: ; Category condition output: ; and (5) amplifying the condition output: ; A4, correcting the model output by using the classifier-free guidance to obtain a final denoising direction: ; a5, updating the image according to the corrected denoising direction: wherein The step length is the denoising step length; A6, iteratively executing the steps A3 to A5 until a final generated image is obtained 。
  8. 8. An electronic device comprising a memory storing a computer program and a processor implementing the method of any of claims 1 to 7 when the computer program is executed by the processor.
  9. 9. A computer readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the method of any one of claims 1 to 7.
  10. 10. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the method of any one of claims 1 to 7.

Description

Plant fiber electron microscope image generation method and related equipment based on diffusion model Technical Field The application relates to the field of computer vision and image generation, in particular to a plant fiber electron microscope image generation method based on a diffusion model and related equipment. Background The microscopic morphology of plant fibers, such as length, diameter, degree of fibrillation, etc., is a key factor in determining its physicochemical properties and properties of the final product (e.g., paper, textile, composite material). Scanning electron microscopy (Scanning Electron Microscope, SEM) is a major tool for observing these microstructures, which is indispensable in basic research and industrial applications. However, obtaining SEM images through physical experiments has inherent bottlenecks of high cost, cumbersome flow, long period, and the like, resulting in serious scarcity and uneven distribution of image data under specific types or specific experimental conditions. The data wasteland greatly restricts the application and development of modern research methods driven by data, in particular to the deep learning technology in the field of plant fiber science. In order to alleviate the shortage of data, the traditional method adopts affine transformation, color dithering and other technologies to enhance the data, but the generated images have limited diversity, and it is difficult to create new samples with essential characteristic variation. Although the generation of depth generation models such as an antagonism network is advanced, the problems of unstable training, mode collapse, insufficient detail fidelity and the like are often faced when the scientific images with high details and complex structures such as fiber electron microscope images are processed. The diffusion model is taken as a representative of a new generation model, and a significant breakthrough in image quality is realized. However, the method is still challenging to directly apply the method to plant fiber electron microscope image generation (1) the traditional diffusion model has a large number of sampling iteration steps and low generation speed, and is difficult to meet the practical requirement on rapid data amplification, (2) for continuous numerical conditions such as 'magnification', an effective coding and control mechanism is lacking, images with specified scale and detail level are difficult to accurately generate, and (3) the method is difficult to carry out deep cooperative control on multi-mode conditions such as discrete categories, continuous parameters and the like, and a mechanism for flexibly adjusting influence of each condition is lacking in an inference stage, so that good balance between the fidelity and diversity of generated results cannot be achieved. Disclosure of Invention The embodiment of the application mainly aims to provide a plant fiber electron microscope image generation method, electronic equipment, a storage medium and a program product based on a diffusion model, which aim to solve the problems of difficult acquisition and data scarcity of plant fiber SEM images, can quickly and controllably generate high-fidelity synthetic images meeting the specified fiber category and magnification conditions, and provide an efficient data support tool for fiber science research, industrial quality detection and deep learning model training. In order to achieve the above object, an aspect of an embodiment of the present application provides a plant fiber electron microscope image generation method based on a diffusion model, the method including: The method comprises the steps of data preparation, namely acquiring a data set containing plant fiber electron microscope images and labels, wherein the labels at least comprise fiber types and magnification; data preprocessing, namely dividing the data set into a training set and a testing set, and preprocessing an image; constructing a noise prediction network based on a UNet network architecture, wherein the input of the network comprises a noise image, a diffusion time step, a fiber class condition and an amplification factor condition; model training, namely training the noise prediction network based on a correction flow frame by taking a vector field from a noise-added image to an original clean image as a target, and storing network parameters; And (3) image generation, namely loading a trained network, carrying out multi-round iterative denoising by combining no classifier guidance from random noise, and generating a plant fiber electron microscope image which accords with the specified fiber category and magnification condition. In some embodiments, the step of data preparation includes: Preparing a plant fiber sample, wherein the plant fiber sample comprises the steps of pretreatment, dissociation, cleaning and freeze drying; The sample is observed by using a scanning electron microscope (i.e. a scanning ele