US-12626491-B2 - Method for detecting object of esophageal cancer in hyperspectral imaging

US12626491B2US 12626491 B2US12626491 B2US 12626491B2US-12626491-B2

Abstract

A method for detecting objects in hyperspectral imaging is revealed. First obtaining a hyperspectral imaging information by a reference image. Then converting an input image according to the hyperspectral imaging information to get a hyperspectral image. A plurality of hyperspectral eigenvalues is obtained after image analysis of the hyperspectral image. Next getting a plurality of dimensionality reduction eigenvalues by a principal component analysis (PCA). Then performing convolution operation on the dimensionality reduction eigenvalues to get a value of a convolution matrix for extracting a feature image from an image of an object to be detected in the input image. Generating an anchor box and a prediction box in the feature image to get a bounded image. Lastly matching and comparing the bounded image with a sample image to determine whether the input image is a target object image. Thereby the method provides assistance for physicians in gastrointestinal image diagnosis.

Inventors

Hsiang-Chen Wang
Ting-Chun MEN
Yu-Ming TSAO
Yu-Lin Liu

Assignees

NATIONAL CHUNG CHENG UNIVERSITY

Dates

Publication Date: 20260512
Application Date: 20230207
Priority Date: 20220304

Claims (11)

1 . A method for detecting objects in hyperspectral imaging, comprising, in a host: obtaining hyperspectral imaging information relating to a reference hyperspectral image, the reference hyperspectral image having been converted from a reference image captured by an endoscope; obtaining an input image captured by the endoscope; converting the input image based on the hyperspectral imaging information to obtain a hyperspectral image of the input image; executing an image analysis of the hyperspectral image of the input image to obtain a plurality of hyperspectral eigenvalues from the hyperspectral image of the input image; executing a principal component analysis (PCA) to simplify the plurality of hyperspectral eigenvalues and obtain a plurality of dimensionality reduction eigenvalues from the simplified hyperspectral eigenvalues; obtaining a value of a convolution matrix based on the dimensionality reduction eigenvalues, a kernel, and at least one convolutional layer, wherein the kernel includes a plurality of feature weight parameters, and the value of the convolution matrix is obtained by multiplying the plurality of feature weight parameters of the kernel with the plurality of dimensionality reduction eigenvalues; extracting a plurality of feature images from the input image based on the value of the convolution matrix; executing zooming, cropping, and arrangement of the plurality of feature images to connect the plurality of feature images and form a joint image; generating a plurality of grid cells based on the joint image; setting at least one anchor box on the input image based on the plurality of grid cells and extracting a plurality of positioning parameters corresponding to the plurality of feature images; setting at least one prediction box on the input image based on the plurality of positioning parameters; obtaining at least one bounded image from the plurality of feature images based on the prediction box; comparing the at least one bounded image with at least one sample image to generate a comparison result; and determining whether the input image is a target object image or not based on the comparison result.
2 . The method as claimed in claim 1 , wherein the host compares the bounded image with the sample image to generate the comparison result by an object detection algorithm YOLOv5.
3 . The method as claimed in claim 1 , wherein the hyperspectral imaging information includes a plurality of color matching functions, a correction matrix, and a conversion matrix each corresponding to the input image.
4 . The method as claimed in claim 1 , wherein the host reads the sample image from a database for comparing with the at least one bounded image.
5 . The method as claimed in claim 1 , wherein the host sets the plurality of grid cells on the input image for positioning the anchor box based on the grid cells, and the anchor box corresponds to at least one ratio between a length and a width.
6 . The method as claimed in claim 1 , a plurality of prediction boxes is generated by the host respectively based on a plurality of anchor boxes of different target sizes.
7 . The method as claimed in claim 1 , wherein the host aligns a central coordinate of the at least one prediction box based on a central coordinate of the anchor box and approaches a bounding coordinates of the at least one prediction box based on an aspect ratio of the anchor box.
8 . The method as claimed in claim 7 , wherein the host sets the at least one prediction box in a plurality of scales including ⅛, 1/16, 1/32, and a combination thereof based on an object detection algorithm YOLOv5.
9 . The method as claimed in claim 1 , wherein in the principal component analysis (PCA), the host extracts a maximum variance based on a hyperspectral vector to which the hyperspectral eigenvalues correspond and then the dimensionality reduction eigenvalues are generated.
10 . The method as claimed in claim 1 , wherein the image analysis extracts a plurality of feature points in the hyperspectral image to obtain the plurality of hyperspectral eigenvalues.
11 . The method as claimed in claim 1 , wherein an object to be detected in the input image is bounded by the at least one prediction box.

Description

FIELD OF THE INVENTION The present invention relates to an image recognition method, especially to a method for detecting objects of the esophageal cancer in hyperspectral imaging. BACKGROUND OF THE INVENTION The esophagus is a tubular organ which connects the pharynx to the stomach for sending food ingested through the mouth to the stomach. The normal esophageal mucosa includes multiple layers of squamous epithelial cells with thickness of 200-5001 μm. The multiple layers consist of epithelium (EP), lamina propria mucosae (LPM), muscularis mucosae (MM), submucosa (SM), and muscularis propria (MP) from top to bottom. Esophageal cancer is the eighth most common cancer worldwide. Carcinoma is a malignancy that develops from epithelial cells. Cancer, also called malignant tumor, has certain impact on physiological functions and further includes sarcoma, lymphoma, leukemia, melanoma, carcinosarcoma, malignant glioma, etc. Sarcoma is a type of cancer that arises in body's connective tissues, which include fibrous tissue, fat, muscle, blood vessels, bones, and cartilage. Lymphoma and leukemia are hematologic malignancies while melanoma develops in skin cells. Carcinosarcomas are malignant tumors that consist of a mixture of epithelial cancer and connective tissue cancer. As to malignant glioma, it is a type of nerve tissue cancer. In esophageal cancer, malignant cells not only infiltrate in epithelial tissue of esophagus but also in connective tissue at advanced stage. Most of medical techniques for disease diagnosis available now depend on a single type of indicator or a piece of information such as temperature, blood pressure, and body scan images. For example, in order to detect serious diseases such as cancer, the most common medical device used now is image-based equipment including X-ray, computer tomography (CT) scan, nuclear magnetic resonance (NMR) imaging, etc. Various combinations of these techniques are useful in disease diagnosis in some degrees. Yet early detection of the serious diseases by the respective techniques is not so accurate, reliable, effective and economical while being used alone. Moreover, most of the devices are invasive and having larger volume such as those using X-ray, CT, and NMR. Thus, more compact and accurate devices such as endoscope have been developed and used to observe lesions on different systems such as gastrointestinal system. Furthermore, detection of esophageal cancer at early stage is not easy. Besides nearly no symptoms, a part of people with subtle changes such as a bit change in colors of the tissue are unable to be identified even using endoscopic examination. Thus, a certain number of early-stage lesions are not diagnosed and thus the treatment is delayed. ∘ In order to detect lesions which are not easy to spot, several techniques including lugol chromoendoscopy, narrow band image (NBI), and magnifying endoscopy have been developed. However, the endoscopic examination is complicated and labor-intensive so that medical staff working in endoscopy must be licensed. Physicians need to detect lesions and interpret the image during the operation. Even both the endoscopy and the examination process have been improved over years, the problems such as human errors in operations and the endoscopic images are hard to interpret for physicians are still there. Thus, there is a room for improvement and there is a need to provide a novel method for detecting objects in hyperspectral imaging by which an input image is classified into a target object image or not in order to avoid difficulties in endoscopic image recognition and interpretation. A host is used to perform convolution operation in a convolutional neural network (CNN) on an input image to get a feature image. Then a bounded image containing an object to be detected is calculated and compared with a sample image so as to classify the input image into a target object image or not. SUMMARY OF THE INVENTION A primary object of the present invention to provide a method for detecting objects in hyperspectral imaging which obtains feature images by convolutional neural network computation of hyperspectral images. Then a bounded image containing an object to be detected is obtained by object detection. Next whether an input image is classified into a target object image or not is determined by comparing the bounded image with a sample image. Thereby the present method provides assistance for physicians in endoscopic image diagnosis. In order to achieve the above object, a method for detecting objects in hyperspectral imaging according to the present invention which includes a plurality of steps run by a host. First run a step of obtaining a hyperspectral imaging information. The host gets a reference image. After the reference image converted into a reference hyperspectral image, a hyperspectral imaging information is obtained according to the reference hyperspectral image. Then run a step of getting a plurality of dimensi