Search

CN-122019796-A - Dynamic spectrum multi-mode Gaussian adaptive clustering dynamic granularity retrieval method

CN122019796ACN 122019796 ACN122019796 ACN 122019796ACN-122019796-A

Abstract

The invention provides a dynamic spectrum multi-mode Gaussian self-adaptive clustering dynamic granularity retrieval method, which comprises the steps of respectively carrying out differential pretreatment after receiving multi-mode data, mapping the multi-mode data into unified semantic feature vectors through a large model system, then carrying out double-model verification to ensure that core semantic elements are accurate, calculating knowledge spectrum side weights based on semantic similarity, resource association strength and length Cheng Yuyi association degree three-dimension, introducing time sequence attenuation factors, automatically recalculating the side weights during resource updating, constructing a dynamic knowledge spectrum, clustering the feature vectors by adopting a GMM model with customized parameters by the semantic feature vectors, carrying out EM iterative optimization and feature correction, evaluating semantic block importance by combining a graph attention network, generating a three-level structure, quantifying global scene complexity and local semantic complexity, dynamically adapting retrieval granularity through a gating function, finally combining association logic of the dynamic knowledge spectrum, and outputting retrieval results balanced in accuracy, efficiency and calculation force.

Inventors

  • HOU GUANGQI
  • SONG PING
  • ZHANG XUEYING

Assignees

  • 天津中科虹星科技有限公司

Dates

Publication Date
20260512
Application Date
20251230

Claims (9)

  1. 1. A dynamic spectrum multi-mode Gaussian adaptive clustering dynamic granularity retrieval method is characterized by comprising the following steps: step 1, collecting multi-modal data and preprocessing, mapping the preprocessed multi-modal data into 256-dimensional original feature vectors through a large-model cross-modal attention mechanism, and obtaining a semantically accurate feature vector set from the original feature vectors through double-model verification to output effective feature vectors; Step 2, calculating semantic similarity of multi-mode data, field association strength with time sequence attenuation and length Cheng Guanlian degrees derived by GAT multi-hop based on the effective feature vector output in the step 1, deducing a three-dimensional dynamic side weight Em, constructing a dynamic knowledge graph, and outputting the association relation between the side weight and the graph; step 3, clustering the effective feature vector output in the step 1 and the association relation between the edge weight and the atlas output in the step 2 by adopting a Gaussian Mixture Model (GMM), carrying out EM iterative optimization and feature correction, and evaluating the importance of the semantic blocks by combining a graph attention network to generate a three-level structure of 'coarse-granularity semantic clusters-medium-granularity semantic blocks-fine-granularity feature units'; and 4, quantifying the output of the steps 1,2 and 3 through global and local two-dimensional complexity, adapting the input retrieval granularity through a gating function, and finally outputting a scene structured retrieval result by combining Em score ordering and timeliness filtering.
  2. 2. The method for dynamic particle size retrieval of dynamic spectrum multi-modal Gaussian adaptive clustering of claim 1, wherein in step 1, the multi-modal data includes text data, image data, audio data, and table data, and preprocessing the multi-modal data includes: text data, extracting word embedding and span context semantic features through a large model built-in mechanism long-range attention mechanism; extracting text information through OCR transcription, and simultaneously retaining a target detection frame, a visual feature vector and gradient change features; Audio data is converted into text through voice recognition, and frequency spectrum features and key voice segment marks are synchronously extracted; And analyzing the form data into the structural text and the numerical characteristics.
  3. 3. The method for dynamically searching the dynamic granularity of the multi-modal Gaussian adaptive clustering of claim 1, wherein in the step 1, the preprocessed multi-modal data is mapped into 256-dimensional original feature vectors through a large-model cross-modal attention mechanism, the original feature vectors are subjected to double-model verification to obtain a semantically accurate feature vector set, and the outputting of the effective feature vectors comprises the following steps: Splitting the multi-mode data into independent semantic units according to semantic integrity, wherein each semantic unit comprises a plurality of core semantic elements, and uniformly mapping the core semantic elements of different modes into the same feature space through a large-model cross-mode attention mechanism, and converting the core semantic elements into 256-dimensional uniform semantic cross-mode mapped original feature vectors; and checking the core semantic element coincidence rate and the core semantic element feature vector coincidence rate respectively by utilizing the double models, and outputting the effective feature vector subjected to coincidence rate screening.
  4. 4. The method for dynamic particle size retrieval by the dynamic spectrum multi-modal Gaussian adaptive clustering is characterized in that in the step 2, a semantic similarity, domain association strength with time sequence attenuation and a long Cheng Guanlian-degree three-dimensional weight collaborative side weight calculation formula derived by GAT multi-hop are designed: Em=α×Sim(Sm,Sn)+β×Rel(Rm,Rn)+(1-α-β)×LongSim(Lm,Ln) em is taken as a comprehensive edge weight, is a weighted fusion result of three dimensional similarity, alpha and beta are core adjustment coefficients, alpha+beta is less than or equal to 1, sim (Sm, sn) is semantic similarity, rel (Rm, rn) is field association strength, longSim (Lm, ln) is Cheng Guanlian degrees long; f m 、f n is a 256-dimensional unified semantic feature vector obtained by preprocessing data Sm and Sn respectively, and I f m ||、||f n I is an L2 norm of the two vectors respectively; Rel(R m ,R n )＝τ×(w 1 ·Corr+w 2 ·Freq+w 3 ·Adapt) τ is a time sequence attenuation factor, w 1 、w 2 、w 3 is a weighting coefficient, w 1 +w 2 +w 3 =1 is satisfied, corr is a domain correlation score, freq is a frequency normalization score used, and Adapt is a scene adaptation score; and (3) iteratively calculating final eigenvectors of nodes corresponding to the data Sm and Sn after GAT multi-hop propagation based on a GAT layer propagation rule: l is the number of propagation layers, alpha mn is the attention weight of GAT automatic learning, and W (l) is the first layer learnable parameter.
  5. 5. The method for dynamic particle size retrieval by dynamic spectrum multi-mode Gaussian adaptive clustering according to claim 1, wherein the step 3 specifically comprises the following steps: taking the effective feature vector obtained in the step 1 as training input data of a Gaussian Mixture Model (GMM); In the training process, K-Means pre-clustering is firstly carried out to obtain an initial cluster center Meanwhile, setting initial mixing weight K as the number of clusters obtained by pre-clustering, wherein the calculation mode is as follows: Using an initial covariance matrix As covariance of samples in each cluster, then starting an EM iterative optimization flow, and calculating posterior probability of the ith feature vector belonging to the kth cluster according to a multi-element gaussian distribution probability density function in step E: M step, based on posterior probability updating mixing weight pi k , cluster center mu k and covariance matrix sigma k , repeating the iterative process until parameter variation <1e-5, stopping convergence; after the clustering is completed, a characteristic correcting mechanism is started; evaluating the importance of semantic blocks by combining a graph attention network, taking a coarse-granularity semantic block B k output by a Gaussian Mixture Model (GMM) as a node, and initializing node characteristics: s i is the traditional importance score of the position i, the edge is established according to the semantic similarity or k neighbor relation, the edge weight is the corresponding semantic similarity, then multi-hop propagation is carried out through the L-layer graph annotation force network, and the propagation rule is as follows: Alpha mn is the attention weight, ultimately characterized by the final characteristics of the node As the total semantic block importance score S (B m ), top-N core semantic blocks are retained in descending order of score; On the basis, three-level structured granularity is generated, coarse granularity is K semantic clusters obtained after GMM convergence, the middle granularity is to split each coarse granularity semantic cluster into sub-semantic blocks according to core semantic element categories, and fine granularity is a key feature unit in the sub-semantic blocks.
  6. 6. The method for dynamic particle size retrieval by dynamic spectrum multi-mode Gaussian adaptive clustering according to claim 1, wherein the step 4 specifically comprises the following steps: Firstly, comprehensively representing semantic complexity through global and local double dimensions, wherein a global complexity formula is as follows: complexity = 0.3 x modality score +0.5 x element score +0.2 x response score The local semantic complexity is realized by calculating attention score entropy and characteristic gradient amplitude, and the attention score entropy formula is as follows: H (C k ) is the attention score entropy of the kth coarse-grained semantic cluster, C k is the kth semantic cluster generated by GMM clustering and contains feature vectors corresponding to a plurality of semantic units, and p i is the normalized posterior probability of i: gamma ik is the posterior probability of the feature vector belonging to the kth cluster, which satisfies the following condition Ln p i is a natural log operation on p i ; Based on the two-dimensional complexity quantization result, the gating function calculation formula is as follows: ρ(x)=σ(W·[Complexity,Entropy,Gradient]+b) w and b are learnable references, sigma is a Sigmoid function, a fine granularity retrieval proportion rho (x) epsilon (0, 1) is output, and different granularity switchable strategies are adapted for scenes with different complexity, so that corresponding granularity in a three-level semantic structure is automatically matched.
  7. 7. A dynamic spectrum multi-mode Gaussian adaptive clustering dynamic granularity retrieval device is characterized by comprising The first processing unit is used for collecting the multi-mode data and preprocessing, mapping the preprocessed multi-mode data into 256-dimensional original feature vectors through a large-model cross-mode attention mechanism, checking the original feature vectors through double models to obtain a feature vector set with accurate semantics, and outputting effective feature vectors; The second processing unit is used for calculating semantic similarity of multi-mode data, field association strength with time sequence attenuation and length Cheng Guanlian degrees derived by GAT multi-hop based on the effective feature vector output by the first processing unit, deducing a three-dimensional dynamic side weight Em, constructing a dynamic knowledge graph and outputting the association relation between the side weight and the graph; The third processing unit is used for clustering the association relation between the effective feature vector output by the first processing unit and the side weight and the atlas output by the second processing unit by adopting a Gaussian Mixture Model (GMM), carrying out EM iterative optimization and feature correction, and evaluating the importance of the semantic blocks by combining a graph attention network to generate a three-level structure of 'coarse-granularity semantic clusters-medium-granularity semantic blocks-fine-granularity feature units'; And the fourth processing unit is used for quantifying the complexity of the first processing unit, the second processing unit and the third processing unit output through global and local double dimensions, adapting the retrieval granularity by a gating function, combining Em score sorting and timeliness filtering, and finally outputting a scene structured retrieval result.
  8. 8. An electronic device comprising a processor and a memory communicatively coupled to the processor for storing instructions executable by the processor, wherein the processor is configured to perform the method of any of claims 1-6.
  9. 9. A computer-readable storage medium, storing a computer program, wherein the computer program is configured to implement the method of any one of claims 1-6 when executed by a processor.

Description

Dynamic spectrum multi-mode Gaussian adaptive clustering dynamic granularity retrieval method Technical Field The invention belongs to the technical fields of computer technology, artificial intelligence and knowledge graph, and particularly relates to a dynamic graph multi-mode Gaussian self-adaptive clustering dynamic granularity retrieval method. Background In the era of digital transformation acceleration and dual driving of popularization of artificial intelligence technology, multi-mode data such as text, image, audio and table have become the main stream form of information production, transmission and storage, and are widely applied to key fields such as academic scientific research, medical diagnosis, enterprise operation, digital content service and the like. Such data often contains rich cross-modal associated information, such as supporting relation of paper text and experimental data form in academic research, complementary information of medical record text and medical image in medical scene, and corresponding relation of product description and technical parameter form in enterprise business, and the value of the data is maximized to mine highly-dependent and efficient cross-modal retrieval technology. Cross-modal retrieval is used as a core support technology for breaking different types of data barriers and realizing multi-source information fusion and utilization, becomes a key premise of upper-layer application landing such as intelligent retrieval, deep data analysis, personalized recommendation and the like, and directly determines the data utilization efficiency and the service intelligent level. At present, as the data scale spans from millions to billions, the data form is more complex and diversified, the application scene of long-sequence data and multi-document cross-domain data is also continuously expanded, and the industry has higher and higher requirements on the accuracy, the correlation integrity, the instantaneity and the computational adaptability of cross-mode retrieval. However, existing cross-modal retrieval techniques are limited to several problems: 1. the multi-mode data semantic representation system is inconsistent to cause mode splitting, the traditional knowledge graph side weight is static and stiff, cross-document and cross-section long Cheng Yinxing association is difficult to capture, and the timeliness of the adaptive resources is insufficient; 2. The retrieval granularity is fixed, the scene adaptation lacks quantization standard, the coarse granularity is easy to miss key details, the fine granularity calculation power consumption is overlarge, and the retrieval accuracy, efficiency and calculation power requirements cannot be balanced. In the existing multi-mode retrieval technology, a Gaussian Mixture Model (GMM) mostly adopts general parameter clustering, multi-mode data characteristics are not adapted, knowledge graph matching depends on static weights, long-range association capturing capacity is weak, gating adjustment lacks quantification basis, and granularity switching is blind. Therefore, there is an urgent need in the art for a new multi-modal retrieval scheme that can dynamically perceive semantics, adaptively adjust granularity, and effectively capture long-range associations. Disclosure of Invention In view of the above, the invention aims to overcome the defects of the prior art, and provides a dynamic particle size retrieval method for multi-mode Gaussian adaptive clustering of dynamic patterns, which realizes the efficient fusion of multi-mode data, the dynamic balance of retrieval accuracy and calculation power consumption, and ensures the correlation integrity and scene adaptation flexibility of a long Cheng Yuyi. In order to achieve the above purpose, the technical scheme of the invention is realized as follows: In a first aspect, the invention provides a dynamic spectrum multi-mode Gaussian adaptive clustering dynamic granularity retrieval method, which comprises the following steps: step 1, collecting multi-modal data and preprocessing, mapping the preprocessed multi-modal data into 256-dimensional original feature vectors through a large-model cross-modal attention mechanism, and obtaining a semantically accurate feature vector set from the original feature vectors through double-model verification to output effective feature vectors; Step 2, calculating semantic similarity of multi-mode data, field association strength with time sequence attenuation and length Cheng Guanlian degrees derived by GAT multi-hop based on the effective feature vector output in the step 1, deducing a three-dimensional dynamic side weight Em, constructing a dynamic knowledge graph, and outputting the association relation between the side weight and the graph; step 3, clustering the effective feature vector output in the step 1 and the association relation between the edge weight and the atlas output in the step 2 by adopting a Gaussian Mixture Model (GMM),