CN-121987130-A - Automatic foreign matter detection and positioning system in digestive system department image

CN121987130ACN 121987130 ACN121987130 ACN 121987130ACN-121987130-A

Abstract

The application discloses an automatic detection and positioning system for foreign matters in an image of a digestive system, which comprises a data acquisition module, a refraction correction module, a spectrum enhancement module, a profile positioning module and a foreign matter positioning module, wherein the data acquisition module is used for acquiring multispectral image data in an digestive tract through an endoscope acquisition terminal and acquiring real-time liquid level depth information corresponding to the multispectral image data, the refraction correction module is used for inputting the multispectral image data and the real-time liquid level depth information into a preset refraction correction model, the refraction correction model is used for processing the multispectral image data, the spectrum enhancement module is used for carrying out self-adaptive spectrum enhancement processing on restored image data to generate enhanced image data, the foreign matter judgment module is used for extracting texture contrast difference characteristics of a plurality of candidate areas from the enhanced image data to obtain foreign matter target probability distribution, and the profile positioning module is used for fitting the foreign matter edges in the enhanced image data according to the foreign matter target probability distribution to generate a foreign matter positioning result with marking information.

Inventors

WEI ZEHAO
YANG SHUO
ZHANG ZHENG

Assignees

江苏大学附属医院

Dates

Publication Date: 20260508
Application Date: 20260128

Claims (8)

1. An automatic foreign matter detection and positioning system in gastroenterology image, which is characterized by comprising: the data acquisition module is used for acquiring multispectral image data in the digestive tract through the endoscope acquisition terminal and acquiring real-time liquid level depth information corresponding to the multispectral image data; The refraction correction module is used for inputting the multispectral image data and the real-time liquid level depth information into a preset refraction correction model, and processing the multispectral image data by utilizing the refraction correction model, wherein the refraction correction model fuses a physical information neural network of a light refraction law, and the refraction correction model carries out refraction path reverse calculation and position mapping on the multispectral image data according to the real-time liquid level depth information to output restored image data; The spectrum enhancement module is used for carrying out self-adaptive spectrum enhancement processing on the restored image data, and carrying out weight redistribution on pixel values of different wave bands by utilizing a preset spectrum conversion matrix to generate enhanced image data; The foreign matter judging module is used for extracting texture contrast difference characteristics of a plurality of candidate areas from the enhanced image data, inputting the texture contrast difference characteristics into a preset classification model for foreign matter judgment and obtaining foreign matter target probability distribution; and the contour positioning module is used for fitting the foreign object edges in the enhanced image data according to the foreign object target probability distribution to generate a foreign object positioning result with marking information.
2. The system for automatically detecting and locating a foreign object in an image of a digestive system according to claim 1, wherein the acquiring multispectral image data and real-time liquid level depth information specifically comprises: collecting a plurality of narrow-band images covering a preset wavelength range through a multispectral imaging unit of an endoscope to form multispectral image data; And measuring the distance between the lens of the endoscope and the surface of liquid in the digestive tract by using an integrated depth sensor module of the endoscope, wherein the distance is real-time liquid level depth information.
3. The automated system for detecting and locating a foreign object in an image of a digestive system of claim 1, wherein the training process of the refraction correction model comprises: constructing a training data set, wherein a sample of the training data set comprises a multispectral image acquired in a fluid environment with a known depth and a corresponding refraction-free reference image thereof; Constructing a neural network model, and integrating a physical equation describing refraction and classification of light rays at a fluid interface into a loss function of the neural network model as constraint conditions; Training the neural network model fused with the physical constraint by using the training data set until the neural network model can accurately predict the refraction effect according to the input depth information and output a corrected image to obtain the refraction correction model.
4. The system for automatically detecting and locating a foreign object in an image of a digestive system according to claim 1, wherein the adaptive spectral enhancement processing for the restored image data comprises the following specific steps: Inquiring a predefined lookup table according to the real-time liquid level depth information to obtain estimated absorption coefficients of the target fluid to each spectrum channel at the current depth; constructing a spectrum compensation matrix according to the estimated absorption coefficient; And multiplying the multispectral vector of each pixel point of the restored image data with the spectrum compensation matrix to realize spectrum reconstruction and obtain the enhanced image data.
5. The automatic detecting and locating system for foreign matter in an image of digestive system according to claim 1, wherein the characteristic of texture contrast difference is input to a predetermined classification model for foreign matter determination, and the specific steps are as follows: inputting the texture contrast difference features into a feature fusion layer of the classification model to obtain fused high-dimensional feature vectors; inputting the high-dimensional feature vector to a full-connection layer of the classification model for calculation; Processing the calculation result of the full-connection layer through a Softmax function in an output layer of the classification model, and outputting a confidence coefficient score of the candidate region belonging to a preset foreign object class; Based on the confidence scores of all candidate regions, the alien target probability distribution characterizing alien spatial distribution probabilities is generated.
6. The automated inspection and localization system of foreign objects in an image of a digestive system of claim 1, wherein the specific method for extracting texture contrast difference features of a plurality of candidate regions from the enhanced image data comprises: performing multi-scale sliding window scanning on the enhanced image data to divide the candidate areas; for each candidate region, calculating a gray level co-occurrence matrix of the candidate region, and extracting contrast, energy and homogeneity characteristics from the gray level co-occurrence matrix; and calculating the difference value between each candidate region and the adjacent neighborhood tissue thereof on the contrast, energy and homogeneity characteristics, and generating the texture contrast difference characteristics.
7. The automated system for detecting and locating a foreign object in an image of a digestive system of claim 4, wherein the look-up table records estimated absorption coefficients for each spectral band for different fluid types at different depths.
8. The automated system for detecting and localizing foreign objects in an image of a digestive system of claim 4, wherein the refraction correction module is further configured to identify a fluid class in a current digestive tract by a clustering algorithm based on global color moment characteristics of the multispectral image data, and to select a corresponding estimated absorption coefficient from the lookup table based on the fluid class.

Description

Automatic foreign matter detection and positioning system in digestive system department image Technical Field The application relates to the technical field of gastroenterology, in particular to an automatic foreign matter detection and positioning system in an image of the gastroenterology. Background Auxiliary diagnosis of foreign matters such as fishbone, button cell or calculus in the gastroenterology is mainly dependent on manual observation or a general target detection algorithm at present, but detection accuracy and reliability are extremely limited in a real clinical environment filled with bile, blood and gastric juice. Due to the influence of the refractive index difference of liquid and air, the semi-submerged or total submerged foreign matter can generate serious geometric distortion and edge distortion in the image, so that the conventional algorithm is difficult to identify the actual physical contour of the foreign matter; meanwhile, the strong absorption effect of the turbid fluid on the specific spectrum can cause the loss of color information on the target surface, the visual contrast between foreign matters and surrounding biological tissues is greatly reduced, missed diagnosis or misdiagnosis is easily caused, and dynamic visual displacement caused by continuous peristalsis of the digestive tract and liquid level fluctuation is caused, so that the accurate space coordinate positioning cannot be realized by the existing static detection model, and emergency intervention by doctors is difficult to effectively assist. Therefore, how to eliminate the refraction distortion and the color cast interference generated by the complex fluid environment and realize the accurate reduction and positioning of the foreign matters becomes a technical problem to be solved in clinical diagnosis. Disclosure of Invention The present application aims to solve at least one of the technical problems in the related art to some extent. Therefore, an object of the present application is to provide an automatic detecting and positioning system for foreign matters in images of digestive system department, which can eliminate refraction distortion and color cast interference generated by complex fluid environment, and realize accurate recovery and positioning of foreign matters. In order to achieve the above objective, an embodiment of a first aspect of the present application provides an automatic detection and positioning system for foreign objects in an image of a digestive system, including: the data acquisition module is used for acquiring multispectral image data in the digestive tract through the endoscope acquisition terminal and acquiring real-time liquid level depth information corresponding to the multispectral image data; The refraction correction module is used for inputting the multispectral image data and the real-time liquid level depth information into a preset refraction correction model, and processing the multispectral image data by utilizing the refraction correction model, wherein the refraction correction model fuses a physical information neural network of a light refraction law, and the refraction correction model carries out refraction path reverse calculation and position mapping on the multispectral image data according to the real-time liquid level depth information to output restored image data; The spectrum enhancement module is used for carrying out self-adaptive spectrum enhancement processing on the restored image data, and carrying out weight redistribution on pixel values of different wave bands by utilizing a preset spectrum conversion matrix to generate enhanced image data; The foreign matter judging module is used for extracting texture contrast difference characteristics of a plurality of candidate areas from the enhanced image data, inputting the texture contrast difference characteristics into a preset classification model for foreign matter judgment and obtaining foreign matter target probability distribution; and the contour positioning module is used for fitting the foreign object edges in the enhanced image data according to the foreign object target probability distribution to generate a foreign object positioning result with marking information. In addition, the system for automatically detecting and positioning the foreign matters in the images of the digestive system according to the application can also have the following additional technical characteristics: In one embodiment of the present application, the acquiring multispectral image data and real-time liquid level depth information specifically includes: collecting a plurality of narrow-band images covering a preset wavelength range through a multispectral imaging unit of an endoscope to form multispectral image data; And measuring the distance between the lens of the endoscope and the surface of liquid in the digestive tract by using an integrated depth sensor module of the endoscope, wherein the distance is