CN-121998909-A - Real-time abnormality detection method for pathological section scanner

CN121998909ACN 121998909 ACN121998909 ACN 121998909ACN-121998909-A

Abstract

The invention provides a real-time anomaly detection method of a pathological section scanner, which embeds anomaly detection links of pathological images of various fields captured by the pathological section scanner into scanning data streams, realizes closed loop of imaging-analysis-decision, can immediately intercept quality problems, avoids continuation of invalid scanning, is a fundamental advantage which cannot be realized by a post detection scheme, obtains strong characteristic expression capability of the pathological images through fine adjustment of a general vision basic model (DINOv) at the front edge, and combines a dynamic search strategy to enable a system to intelligently select the most relevant normal reference, overcomes the inherent high variability problem of the pathological images and obviously reduces false alarm rate.

Inventors

ZHENG ZHONGXI
ZHANG JIE
Zhang Chukang
ZHANG JINSHENG
XIA MIN

Assignees

武汉现代病理工程研究院有限公司

Dates

Publication Date: 20260508
Application Date: 20251224

Claims (10)

1. A method for detecting a pathological section scanner in real time, comprising: Retraining the universal vision basic model based on the pathological image training set to obtain a pathological image feature extraction model; Constructing a normal pathology image reference set, extracting global semantic feature vectors of each normal pathology image based on the pathology image feature extraction model, and constructing a global feature retrieval library; Extracting global semantic feature vectors of the current visual field pathological image to be detected based on the pathological image feature extraction model, and taking the global semantic feature vectors as query vectors; Matching the query vector to the global feature search library to find k best matched normal pathology images, wherein k is a positive integer, serving as normal pathology reference sample images of the current visual field pathology image to be detected; Extracting local semantic feature vector sequences of k normal pathology reference sample images based on the pathology image feature extraction model, and constructing a temporary memory bank of the current field of view to be detected; extracting a local semantic feature vector sequence of a current visual field pathological image to be detected based on the pathological image feature extraction model; Calculating the abnormal score of the current visual field pathological image to be detected based on the local semantic feature vector sequence of the current visual field pathological image to be detected and the local semantic feature vector sequence in the temporary memory bank; And determining whether the current visual field pathological image to be detected is abnormal or not based on the abnormal score of the current visual field pathological image to be detected.
2. The method for detecting a real-time abnormality of a pathological section scanner according to claim 1, wherein retraining the general visual basic model based on the pathological image training set to obtain a pathological image feature extraction model comprises: Collecting multi-source heterogeneous pathology full-section images to form a pathology full-section image set, wherein the pathology full-section images set cover pathology full-section images of different dyeing batches, pathology full-section images scanned by different brands/models scanners and pathology full-section images generated by different laboratory preparation processes; Based on an adaptive threshold algorithm and a lightweight semantic segmentation model, identifying an actual tissue region in each pathological full-section image, and generating a binary mask image; calculating the proportion of the actual tissue area in each pathology full-section image to the whole pathology full-section image visual field area based on the binary mask image, and screening each pathology full-section image based on the proportion; sampling each screened pathological full-section image in a multi-scale level manner to obtain sampled image blocks; All the sampled image blocks form a pathological image training set; Retraining the universal visual basic model based on the pathological image training set to obtain a pathological image feature extraction model, wherein the universal visual basic model is an open source model trained based on natural images.
3. The method for detecting a real-time abnormality of a pathological section scanner according to claim 2, wherein the multi-scale level sampling is performed on each of the screened pathological full-section images to obtain a sampled image block, and the method comprises: Based on the different objective magnification of the microscope, each pathology whole-section image is fixedly sampled from three scale levels: For the low-power global context scale, sampling 512 x 512 pixel image blocks from each pathology full-slice image; for the mesoscopic tissue structure scale, an image block of 512×512 pixels is sampled from each pathological full-slice image; For the high-power mirror cell detail scale, an image block of 512 x 512 pixels is sampled from each pathology whole-slice image.
4. The method for detecting real-time abnormalities of a pathological section scanner according to claim 1, wherein said constructing a normal pathological image reference set, extracting global semantic feature vectors of each normal pathological image based on the pathological image feature extraction model, constructing a global feature search library, comprises: screening and labeling a panoramic normal pathology image set, wherein the normal pathology image set covers all common tissue types, all staining schemes and normal pathology images of all production batches in a target application scene; inputting each normal pathology image in the normal pathology image set into the pathology image feature extraction model to obtain a global semantic feature vector of each normal pathology image; The global semantic feature vector and associated metadata of each normal pathology image are stored in a global feature retrieval library, wherein the metadata comprises the tissue type, the staining method, the acquisition mechanism code and the image sample number of each normal pathology image.
5. The method of claim 4, wherein storing the global semantic feature vector and associated metadata for each normal pathology image in a global feature search library comprises: Packaging and deploying Milvus a vector database by adopting a Docker container and depending on the environment; Creating a global feature data table in a Milvus vector database, wherein the global feature data table comprises a main field and an auxiliary field, the global semantic feature vector of each normal pathological image is stored under the main field, and the metadata of each normal pathological image is stored under the auxiliary field.
6. The method for real-time anomaly detection of a pathological section scanner according to claim 5, wherein matching the query vector to the global feature search library to find the k best matched normal pathological images comprises: Packaging the query vector of the current visual field pathological image to be detected as a query request, and initiating the query request to the global feature retrieval library, wherein the query request carries a filtering condition; Screening the auxiliary fields in the global feature data table based on the filtering conditions to obtain screened candidate global semantic feature vectors; Calculating the similarity between the query vector and each candidate global semantic vector, wherein the similarity is cosine similarity; And sequencing the similarity from large to small, returning k candidate global semantic feature vectors with the highest similarity with the query vectors, and acquiring k normal pathological images corresponding to the k candidate global semantic feature vectors, wherein the k value is dynamically adjusted according to the application scene.
7. The method for detecting the real-time abnormality of the pathological section scanner according to claim 1, wherein the step of extracting the local semantic feature vector sequences of the K normal pathological reference sample images based on the pathological image feature extraction model to construct a temporary memory bank of the current field of view to be detected comprises the steps of: Inputting each normal pathology reference sample image into the pathology image feature extraction model, wherein the pathology image feature extraction model uniformly divides each normal pathology reference sample image into n multiplied by n image blocks, and calculates the local semantic feature vector of each image block, wherein n is a positive integer; And storing the local semantic feature vectors of the k normal pathology reference sample images into a temporary memory bank M of the current visual field to be detected, wherein the temporary memory bank M of the current visual field to be detected comprises k multiplied by n local semantic feature vectors.
8. The method for detecting a pathological section scanner according to claim 7, wherein extracting a local semantic feature vector sequence of a current visual field pathological image to be detected based on the pathological image feature extraction model comprises: Inputting the current visual field pathological image to be detected into the pathological image feature extraction model, and outputting n multiplied by n local semantic feature vectors of the current visual field pathological image to be detected; The calculating the abnormal score of the current visual field pathological image to be measured based on the local semantic feature vector sequence of the current visual field pathological image to be measured and the local semantic feature vector sequence in the temporary memory bank comprises the following steps: Calculating the similarity between any local semantic feature vector of the current visual field pathology image to be detected and each local semantic feature vector in the temporary memory bank M, and taking the minimum similarity between any local semantic feature vectors as the abnormal score of the image block corresponding to any local semantic feature vector; Traversing each local semantic feature vector of the current visual field pathological image to be detected to obtain the abnormal score of each image block; taking the largest first p% of the abnormal scores of all the image blocks, taking an average value, taking the average value as the global abnormal score of the current visual field pathological image to be detected, wherein p is a positive integer of 1-10.
9. The method for detecting real-time abnormality of a pathological section scanner according to claim 1, wherein the determining whether the current visual field pathological image to be detected is abnormal based on the abnormality score of the current visual field pathological image to be detected includes: If the abnormal score of the current visual field pathological image to be detected is larger than the preset score threshold, the current visual field pathological image to be detected is abnormal, otherwise, the current visual field pathological image to be detected is normal.
10. The method for detecting real-time abnormalities of a pathological section scanner according to claim 1, wherein said determining whether the current visual field pathological image to be detected is abnormal based on the abnormality score of the current visual field pathological image to be detected further comprises: Performing anomaly detection on each visual field pathological image to be detected, which is continuously captured by a pathological section scanner in a preset time period; When the proportion of the pathological images of the field of view to be detected, which is detected to be abnormal, reaches a preset proportion threshold value in a preset time period, a control instruction is sent to a control system of the pathological section scanner, so that the control system of the pathological section scanner controls the pathological section scanner to pause scanning; And automatically recording the visual field coordinates, the global anomaly score, the key frame images and the time stamps of the visual field pathological images to be detected with the detected anomalies into a scanning log and a quality control database.

Description

Real-time abnormality detection method for pathological section scanner Technical Field The invention relates to the technical field of digital pathology and computer vision, in particular to a real-time abnormality detection method of a pathological section scanner. Background The full-slide scanner divides a glass pathological section into thousands to tens of thousands of continuous high-resolution fields of view through a high-precision automatic microscope platform to shoot each frame, and generates a complete digital full-slide image through an image stitching technology. Currently, quality control of the scanning process is mainly dependent on whole image evaluation after scanning is completed or manual spot inspection by operators. This approach has significant drawbacks: (1) Hysteresis, the inability to find problems (e.g., tissue folds, bubbles, focal length loss, staining contamination, etc.) in the scanning process in real time, resulting in a large number of invalid scans, wasting time and memory resources, and possibly contaminating subsequent slice images due to the inability to find hardware problems (e.g., lens stains) in real time. (2) The method is non-intelligent and low in efficiency, relies on manual experience for spot check, is low in efficiency and poor in consistency, and is easy to miss due to fatigue in a high-flux scanning scene. Closed loop control cannot be realized, namely when an abnormality is detected later, the scanning task is completed completely or partially, an operator cannot stop scanning of an abnormal area in time, the success rate of primary scanning is reduced, and a time-consuming rescanning process is required. Therefore, the prior art lacks a solution that can perform intelligent analysis immediately after each view image is captured, accurately judge whether the view has quality abnormality in second-level time, and can realize real-time decision and intervention in linkage with a scanner control system. Disclosure of Invention Aiming at the technical problems in the prior art, the invention provides a real-time abnormality detection method of a pathological section scanner, which can solve the problem of abnormality detection lag caused by integral image evaluation or manual spot check of operators after the existing scanning is completed. According to a first aspect of the present invention, there is provided a real-time abnormality detection method of a pathological section scanner, comprising: Retraining the universal vision basic model based on the pathological image training set to obtain a pathological image feature extraction model; Constructing a normal pathology image reference set, extracting global semantic feature vectors of each normal pathology image based on the pathology image feature extraction model, and constructing a global feature retrieval library; Extracting global semantic feature vectors of the current visual field pathological image to be detected based on the pathological image feature extraction model, and taking the global semantic feature vectors as query vectors; matching the query vector to the global feature search library to find k best matched normal pathology reference sample images, wherein k is a positive integer; Extracting local semantic feature vector sequences of K normal pathology reference sample images based on the pathology image feature extraction model, and constructing a temporary memory bank of the current field of view to be detected; extracting a local semantic feature vector sequence of a current visual field pathological image to be detected based on the pathological image feature extraction model; Calculating the abnormal score of the current visual field pathological image to be detected based on the local semantic feature vector sequence of the current visual field pathological image to be detected and the local semantic feature vector sequence in the temporary memory bank; And determining whether the current visual field pathological image to be detected is abnormal or not based on the abnormal score of the current visual field pathological image to be detected. According to a second aspect of the present invention, there is provided a real-time abnormality detection system of a pathological section scanner, comprising: the training module is used for retraining the universal vision basic model based on the pathological image training set to obtain a pathological image feature extraction model; The first construction module is used for constructing a normal pathological image reference set, extracting global semantic feature vectors of each normal pathological image based on the pathological image feature extraction model and constructing a global feature retrieval library; The first extraction module is used for extracting global semantic feature vectors of the current visual field pathological image to be detected based on the pathological image feature extraction model, and the global semantic fea