CN-116309947-B - Simulated indigo carmine dyeing method based on cyclic countermeasure generation network

CN116309947BCN 116309947 BCN116309947 BCN 116309947BCN-116309947-B

Abstract

The invention discloses a simulated indigo carmine dyeing method based on a cyclic countermeasure generation network, which comprises the following steps of inputting a white light image of an endoscope image into a trained simulated indigo carmine dyeing model to generate a simulated indigo carmine dyeing image, wherein the simulated indigo carmine dyeing model is a TransUNet backbone network based on a Transformer and a CNN, the CNN comprises a generator G A , a generator G B and a discriminator D x , the training method of the simulated indigo carmine dyeing model comprises the steps of building a network training set, then generating a simulated dyeing image P 3 , inputting the simulated dyeing image P 3 into the discriminator D x , judging whether the simulated dyeing image P 3 is the endoscope image dyeing image P 2 by the discriminator D x , obtaining corresponding total loss and inputting the total loss into the generator G A and the generator G B for training. The method for simulating indigo carmine dyeing based on the cyclic countermeasure generation network provided by the invention can simulate indigo carmine dyeing of an endoscope image, reduce medical cost and avoid repeated endoscope detection and drug use.

Inventors

SONG HUIHUI
DING XINLAN
SUN ZIZHENG

Assignees

南京信息工程大学

Dates

Publication Date: 20260505
Application Date: 20230320

Claims (6)

1. A simulated indigo carmine dyeing method based on a cyclic countermeasure generation network is characterized by comprising the following steps of: Inputting the white light image of the endoscope image into a trained simulated indigo carmine dyeing model to generate a simulated indigo carmine dyeing image; the simulated indigo carmine dyeing model is a TransUNet backbone network based on a transducer and CNN, wherein the CNN comprises a generator Generator and method for generating a digital signal Distinguishing device , The training method of the simulated indigo carmine dyeing model comprises the following steps: Collecting an endoscopic image dataset, dividing an undyed endoscopic image into white light images And an endoscopic image stained image by indigo carmine staining For white light image of endoscope image And endoscopic image staining image Cutting to obtain a network training set; during downsampling, the endoscope image white light image in the network training set is processed Feed-in generator Encoding, namely, white light image of the endoscope image The bottom layer feature of (a) is converted into a feature vector, then a transducer is used for carrying out feature extraction on the feature vector to obtain a feature result, and finally a generator is used Up-sampling and decoding the characteristic result by a decoder of (a) to generate a simulated dyeing image The method comprises extracting features of image by CNN to obtain feature image of input image, fusing the obtained feature image with feature image of the same size in decoder to obtain fused feature image, linear projection transforming the fused feature image to obtain two-dimensional mark vector with size of n_patch, D (i.e. feature vector), wherein n_patch=8, D is selected according to input image, extracting features of feature vector by transducer, sending the feature vector into 12 transducer layers after obtaining feature vector, and reshaping to obtain the final product , And The height and width of the feature map at this time are represented as 16 times of the input image downsampled; Will simulate a stained image Input discriminator In (1), by a discriminator Judging the simulated dyeing image Whether or not to dye the image for the endoscope image Obtain the corresponding total loss and input to the generator Sum generator Training; Will simulate a stained image Feed-in generator Coding is carried out, and the simulated dyeing image is obtained The bottom layer feature of (a) is converted into a feature vector, then a transducer is used for carrying out feature extraction on the feature vector to obtain a feature result, and finally a generator is used Up-sampling and decoding the feature result by a decoder to generate an analog white light image Wherein the decoder upsampling process uses a ReLU activation function and has a size of The convolution kernel of (2) performs an upsampling operation, the formula of which is as follows: ; Wherein the method comprises the steps of 、、、 For the feature of the downsampling of the image, 、、、 As a feature of the up-sampling of the image, In order to downsample the convolution function, For the transform feature extraction function, Is an up-sampling convolution function; will simulate white light image Input discriminator In (1), by a discriminator Judging analog white light image Whether or not it is an endoscopic white light image Obtain the corresponding total loss and input to the generator Sum generator Training to obtain a trained generator with network parameter weights Sum generator 。
2. A method for dyeing a simulated indigo carmine based on a cyclic countermeasure generation network as claimed in claim 1, wherein the method comprises the steps of And endoscopic image staining pattern The cutting is specifically performed by adopting a central cutting method, namely, taking the midpoint of the image as the center, cutting the periphery by the selected size to obtain the cutting size of 。
3. A simulated indigo carmine dyeing method based on a cyclic countermeasure generation network as claimed in claim 1, characterized in that the total loss To combat losses And cycle coincidence loss The sum of, i.e 。
4. A simulated indigo carmine dyeing method based on a cyclic challenge generating network as claimed in claim 3, characterized by a challenge loss The calculation is as follows: ; the countering loss is a loss function of the discriminator for Y is the distribution from the real data The sample obtained by the middle sampling is taken, The larger is equivalent to The larger the discriminator is, the more accurately the true sample can be identified as the true sample X is from a particular distribution The resulting sample of (a) is then processed, A false sample generated by the generator, The larger the The smaller, i.e. the more correctly the arbiter can distinguish between false samples, Refers to the difference information between the true sample and the false sample obtained by the arbiter D under the action of the sample y, Refers to the resulting false samples from the arbiter D Difference information between the real sample and the real sample; Loss of cyclic uniformity The sum is calculated as follows: 。
5. a simulated indigo carmine dyeing device based on a cyclic countermeasure generation network, performing the simulated indigo carmine dyeing method based on a cyclic countermeasure generation network as claimed in any one of claims 1 to 4, characterized by comprising: The image acquisition module is used for acquiring a white light image of the endoscope image; A simulated indigo carmine dyeing model which is a TransUNet backbone network based on a Transformer and CNN, wherein the CNN comprises a generator Generator and method for generating a digital signal Distinguishing device , The training method of the simulated indigo carmine dyeing model comprises the following steps: Collecting an endoscopic image dataset, dividing an undyed endoscopic image into white light images And an endoscopic image stained image by indigo carmine staining For white light image of endoscope image And endoscopic image staining image Cutting to obtain a network training set; during downsampling, the endoscope image white light image in the network training set is processed Feed-in generator Encoding, namely, white light image of the endoscope image The bottom layer feature of (a) is converted into a feature vector, then a transducer is used for carrying out feature extraction on the feature vector to obtain a feature result, and finally a generator is used Up-sampling and decoding the characteristic result by a decoder of (a) to generate a simulated dyeing image ; Will simulate a stained image Input discriminator In (1), by a discriminator Judging the simulated dyeing image Whether or not to dye the image for the endoscope image Obtain the corresponding total loss and input to the generator Sum generator Training; Will simulate a stained image Feed-in generator Coding is carried out, and the simulated dyeing image is obtained The bottom layer feature of (a) is converted into a feature vector, then a transducer is used for carrying out feature extraction on the feature vector to obtain a feature result, and finally a generator is used Up-sampling and decoding the feature result by a decoder to generate an analog white light image ; Will simulate white light image Input discriminator In (1), by a discriminator Judging analog white light image Whether or not it is an endoscopic white light image Obtain the corresponding total loss and input to the generator Sum generator Training to obtain a trained generator with network parameter weights Sum generator 。
6. A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the simulated indigo carmine dyeing method based on a cyclic challenge generating network as claimed in any one of claims 1-4.

Description

Simulated indigo carmine dyeing method based on cyclic countermeasure generation network Technical Field The invention relates to a simulated indigo carmine dyeing method based on a cyclic countermeasure generation network, and belongs to the technical field of image processing. Background Stomach cancer is one of common malignant tumors in China, the incidence rate is the first of digestive system tumors, the death rate can be well reduced by detecting early treatment, if stomach cancer can be detected through a gastroscope in the early stage and is limited on a small piece of mucosa, diseased tissues can be removed through gastroscope operation, no knife or chemotherapy is needed, pain is small, recovery is good, and natural survival rate is greatly improved. It is important to improve the endoscope detection effect and help doctors to find focus in time. Digestive endoscopy is the most dominant means of screening for early cancers of the digestive tract. The conventional enhancement assisting method for the general digestive endoscopy comprises an indigo carmine dyeing technology, the normal mucosa surface of the digestive tract is smooth, when inflammation, ulcer and canceration occur, the mucosa surface is changed, if the lesion range is small or the lesion is difficult to be found under the ordinary endoscope in the early stage, indigo carmine dyeing agent needs to be sprayed on the mucosa, part of the indigo carmine with smooth surface can not be precipitated, and the indigo carmine is completely deposited between grooves of folds of the gastric mucosa, and the fine concave-convex change of the gastric mucosa and the three-dimensional structure are displayed under the gastroscope. However, indigo carmine staining technology not only increases medical cost, patients need multiple gastroscopies, and the requirements on the professional quality of doctors are too high, but also can not meet the requirements of clinical treatment for doctors in underdeveloped areas, and multiple endoscopic detection and drug use can cause serious secondary injury to the patients. Disclosure of Invention The invention aims to overcome the defects of the prior art, and provides a simulated indigo carmine dyeing method based on a cyclic countermeasure generation network, which can simulate indigo carmine dyeing of an endoscope image, reduce medical cost and avoid repeated endoscope detection and drug use. In order to solve the technical problems, the invention adopts the following technical scheme: a simulated indigo carmine dyeing method based on a cyclic countermeasure generation network, comprising the steps of: Inputting the white light image of the endoscope image into a trained simulated indigo carmine dyeing model to generate a simulated indigo carmine dyeing image; the simulated indigo carmine dyeing model is a TransUNet backbone network based on a transducer and CNN, wherein the CNN comprises a generator Generator and method for generating a digital signalDistinguishing device, The training method of the simulated indigo carmine dyeing model comprises the following steps: Collecting an endoscopic image dataset, dividing an undyed endoscopic image into white light images And an endoscopic image stained image by indigo carmine stainingFor white light image of endoscope imageAnd endoscopic image staining imageCutting to obtain a network training set; during downsampling, the endoscope image white light image in the network training set is processed Feed-in generatorEncoding, namely, white light image of the endoscope imageThe bottom layer feature of (a) is converted into a feature vector, then a transducer is used for carrying out feature extraction on the feature vector to obtain a feature result, and finally a generator is usedUp-sampling and decoding the characteristic result by a decoder of (a) to generate a simulated dyeing image; Will simulate a stained imageInput discriminatorIn (1), by a discriminatorJudging the simulated dyeing imageWhether or not to dye the image for the endoscope imageObtain the corresponding total loss and input to the generatorSum generatorTraining; Will simulate a stained image Feed-in generatorCoding is carried out, and the simulated dyeing image is obtainedThe bottom layer feature of (a) is converted into a feature vector, then a transducer is used for carrying out feature extraction on the feature vector to obtain a feature result, and finally a generator is usedUp-sampling and decoding the feature result by a decoder to generate an analog white light image; Will simulate white light imageInput discriminatorIn (1), by a discriminatorJudging analog white light imageWhether or not it is an endoscopic white light imageObtain the corresponding total loss and input to the generatorSum generatorTraining to obtain a trained generator with network parameter weightsSum generator。 White light image of endoscope imageAnd endoscopic image staining patternThe cutting is specifically performed by a