CN-121979684-A - Pathological WSI dynamic rendering and computing power scheduling method and system based on multi-mode semantic-space mapping

CN121979684ACN 121979684 ACN121979684 ACN 121979684ACN-121979684-A

Abstract

The invention discloses a pathological WSI dynamic rendering and computational power scheduling method and system based on multi-mode semantic-space mapping, and belongs to the technical field of digital pathology and computer resource scheduling. Aiming at the problems of memory congestion caused by uniform loading of WSI and easy missed diagnosis of calculation power, the scheme inputs a clinical priori text into a large language model, extracts a diagnosis intention label with semantic risk weight, performs cross-modal matching by combining local visual features of a low-resolution panorama to obtain physical tile coordinates, synthesizes semantic risk weight, local image information entropy and rendering cost to generate a two-dimensional space calculation power weight matrix, and a bottom scheduling engine executes memory priority prefetching and asymmetric calculation power distribution of an AI cascade reasoning model according to the memory priority prefetching. The invention optimizes the software and hardware resources and reduces the missed diagnosis rate, and is mainly used for digital pathology reading and AI diagnosis.

Inventors

HUANG QIANG
WANG ZIHAN
CHEN YULING
JIN JIE
KUANG GUOTAO

Assignees

深圳市生强科技有限公司

Dates

Publication Date: 20260505
Application Date: 20260402

Claims (10)

1. A pathological WSI dynamic rendering and computing power scheduling method based on multi-mode semantic-space mapping is characterized by comprising the following steps: Acquiring a clinical priori text of a target patient and a corresponding full-slice image WSI; Inputting the clinical priori text into a large language model for analysis, extracting a diagnosis intention label with initial semantic risk weight, and converting the diagnosis intention label into a pathological semantic feature vector; Extracting a low-resolution panoramic image of the full-slice image WSI, extracting local visual features of the panoramic image through a lightweight segmentation network, performing cross-mode similarity matching on the local visual features and the pathological semantic feature vector, and mapping an area with the similarity meeting the requirement into a two-dimensional physical tile coordinate set under a high-resolution level; Aiming at the image tiles to be processed, calculating the scheduling priority of the image tiles by combining the corresponding semantic risk weights, the local image information entropy and the rendering cost, and generating a two-dimensional space computing force weight matrix according to the scheduling priority of the whole tiles; Inputting the two-dimensional space computational power weight matrix into a bottom layer resource scheduling engine, and executing a memory rendering scheduling strategy and/or a computational power distribution strategy of an artificial intelligent reasoning model based on the scheduling priority.
2. A pathological WSI dynamic rendering and computational power scheduling method according to claim 1, wherein the step of inputting the clinical prior text into a large language model for parsing comprises: extracting medical entities containing anatomical orientations or specific histological orientations through preset prompt words to serve as diagnosis intention labels; Assigning an initial semantic risk weight to the diagnostic intent label and encoding the normalized medical entity as the pathological semantic feature vector.
3. A pathological WSI dynamic rendering and computing power scheduling method according to claim 1, wherein the step of cross-modal similarity matching the local visual features and the pathological semantic feature vectors specifically comprises: Generating candidate tissue connected domains and local visual features of the low-resolution panoramic image by using a lightweight morphological segmentation network; Inquiring a preset cross-modal mapping dictionary, and calculating similarity scores of the pathological semantic feature vectors and local visual features of each candidate tissue connected domain; And selecting a candidate tissue connected domain with similarity score higher than a preset threshold as a target area, and linearly mapping the target area into the two-dimensional physical tile coordinate set under a high-resolution level according to the scaling relationship of the image pyramid level.
4. The method for dynamically rendering and computing power scheduling pathological WSI according to claim 3, wherein the cross-modal mapping dictionary is constructed by marking a typical anatomical or pathological region on a pathological low-resolution image to form a visual prototype library, inputting standardized pathological terms and corresponding marked regions into a text encoder and an image encoder respectively, and performing cross-modal alignment training through visual-language contrast learning to obtain a semantic-visual embedding space for retrieval.
5. The pathological WSI dynamic rendering and computing power scheduling method according to claim 1, wherein calculating the scheduling priority of the image tiles specifically comprises: Carrying out weighted summation calculation on the semantic risk weight corresponding to the image tile and the local image information entropy, and carrying out ratio calculation or weight reduction calculation on the summation result based on the rendering cost of the image tile transferred from a storage medium to a display memory to obtain a final scheduling priority; The local image information entropy is used for representing image complexity after blank and uniform bubble interference are eliminated, and the rendering cost is obtained through comprehensive evaluation of file size, storage reading time delay, network transmission time delay and decoding and carrying time consumption.
6. The method of claim 1, wherein the memory rendering scheduling policy includes forcing image tiles with scheduling priorities greater than a first threshold in the two-dimensional spatial power weighting matrix into a cache region of a system, and executing a delayed load or discard instruction on image tiles with scheduling priorities less than or equal to the first threshold.
7. The method for dynamically rendering and computing power scheduling of pathological WSI according to claim 6, wherein the computing power distribution strategy of the artificial intelligent reasoning model comprises the steps of starting an adaptive cascade deep learning framework, calling a heavy reasoning model for fine recognition analysis on image tiles with scheduling priorities larger than a second threshold value in the two-dimensional spatial computing power weight matrix, and downgrading and calling a lightweight coarse screening model for pre-screening or skipping processing on image tiles with scheduling priorities lower than or equal to the second threshold value.
8. The method for dynamically rendering and computing power scheduling for pathological WSI according to claim 7, wherein the first threshold and the second threshold are dynamic adaptive thresholds, and further comprising dynamically adjusting the first threshold and the second threshold in real time based on a preset reference threshold by combining a full-slice image size, a candidate tile number, a current host residual memory, a graphics processor residual memory and an input/output congestion degree.
9. The pathological WSI dynamic rendering and computing power scheduling method of claim 1, further comprising an anti-hallucination confidence verification and system rollback mechanism: When the two-dimensional physical tile coordinate set obtained by cross-modal matching is empty, the confidence of cross-modal matching is lower than a safety threshold or the system has input and output abnormality, triggering a system back-off mechanism of a bottom resource scheduling engine, stopping scheduling according to the two-dimensional space computing force weight matrix, and backing off a resource allocation mode into a full-image uniform scanning and sequential loading mode.
10. A pathological WSI dynamic rendering and computational power scheduling system based on multi-modal semantic-spatial mapping, comprising: the multi-source data acquisition and analysis module is used for acquiring clinical priori texts and full-slice images, extracting diagnosis intention labels with semantic risk weights through a large language model and converting the diagnosis intention labels into pathological semantic feature vectors; The image topology perception module is used for extracting a low-resolution panoramic image and acquiring local visual characteristics, and mapping the high-risk area into a two-dimensional physical tile coordinate set under high resolution through cross-modal matching; the computing moment array generating module is used for combining the semantic risk weight of the image tiles, the local image information entropy and the rendering cost to calculate the scheduling priority and generate a two-dimensional space computing force weight matrix; and the bottom layer resource scheduling engine is used for receiving the two-dimensional space computing power weight matrix, and executing prefetching scheduling of the memory and dynamic computing power distribution of the artificial intelligent cascade model on the image tiles according to the two-dimensional space computing power weight matrix.

Description

Pathological WSI dynamic rendering and computing power scheduling method and system based on multi-mode semantic-space mapping Technical Field The invention relates to the technical fields of digital pathology, multi-modal Large Language Model (LLM) and computer system resource scheduling, in particular to a method and a system for dynamically optimizing loading priority of a super-resolution pathology image (WSI) Tile (Tile) and calculation power distribution of a bottom GPU by utilizing priori knowledge of clinical texts. Background Digital pathology full-slice images (white SLIDE IMAGE, WSI) have ultra-high resolution characteristics, typically up to the level of hundred thousand by hundred thousand pixels, with single file volumes of several GB to several tens of GB. To accommodate computer processing, a multi-level "image tile pyramid (TILE PYRAMID)" architecture is typically employed for storage, rendering, and AI analysis. However, existing digital pathology treatment schemes face the following serious bottlenecks in practical clinical applications: first, in front-end rendering and memory scheduling, existing WSI film reading systems (views) generally use a hard-coded strategy of "Sliding Window" or "uniform loading per field of view" when loading images. The system cannot predict the anatomical region of the doctor which is most concerned currently, so that high-magnification images of a large number of non-critical regions (such as large-area blank glass and normal adipose tissues) are blindly pushed into a memory and a video memory, and serious I/O congestion, rendering jam and even memory overflow (OOM) occur. Secondly, in the aspect of bottom calculation power distribution, when the conventional AI auxiliary diagnosis model performs full-slice scanning, GPU (graphics processing Unit) reasoning calculation power with the same magnitude is often distributed to all organization areas, for example, a ViT or ResNet large model with huge parameters is uniformly used. In fact, the diagnostic emphasis of a doctor is often hidden in the patient's pathological application form or clinical text. The existing AI system ignores the precious text priori knowledge, consumes a large amount of calculation force in the low-risk area, not only results in long overall reporting time, but also is extremely easy to miss micro focus in the high-risk area because the calculation force is uniformly spread. Disclosure of Invention The embodiment of the invention provides a pathological WSI dynamic rendering and computing power scheduling method and system based on multi-mode semantic-space mapping, which aim at solving the problems that in the prior art, when an ultra-large resolution pathological image is processed, an indiscriminate uniform loading and computing power sharing strategy is adopted, so that system I/O congestion, memory and GPU computing power are seriously wasted, and tiny focus is easily missed due to computing power dispersion. The core technology of the invention mainly analyzes clinical texts through a large language model to extract diagnostic intention, combines a low-magnification panorama to generate a two-dimensional space calculation force weight matrix, and dynamically guides the memory of a bottom layer system to perform asymmetrical cascade scheduling of the large/small AI model according to the demand. In a first aspect, the present invention provides a pathological WSI dynamic rendering and computational power scheduling method based on multi-modal semantic-spatial mapping, the method comprising the steps of: Acquiring a clinical priori text of a target patient and a corresponding full-slice image WSI; Inputting a clinical priori text into a large language model for analysis, extracting a diagnosis intention label with initial semantic risk weight, and converting the diagnosis intention label into a pathological semantic feature vector; Extracting a low-resolution panoramic image of the full-slice image WSI, extracting local visual features of the panoramic image through a lightweight segmentation network, performing cross-mode similarity matching on the local visual features and pathological semantic feature vectors, and mapping an area with the similarity meeting the requirement into a two-dimensional physical tile coordinate set under a high-resolution level; aiming at the image tiles to be processed, calculating the scheduling priority of the image tiles by combining the corresponding semantic risk weights, the local image information entropy and the rendering cost, and generating a two-dimensional space computing force weight matrix according to the scheduling priority of the whole tiles; Inputting the two-dimensional space calculation force weight matrix into a bottom layer resource scheduling engine, and executing a memory rendering scheduling strategy and/or a calculation force allocation strategy of an artificial intelligent reasoning model based on the scheduling priority. Furth