CN-122019615-A - Data resource accurate processing and sharing method based on multi-mode semantic association analysis

CN122019615ACN 122019615 ACN122019615 ACN 122019615ACN-122019615-A

Abstract

The invention provides a data resource accurate processing and sharing method based on multi-mode semantic association analysis, and relates to the technical field of semantic processing. And constructing a mixed query vector by combining the two, performing multi-granularity matching, performing confidence weighting sequencing on the results by evaluating indexes such as cross-modal semantic consistency, and finally, adaptively adjusting the graph structure according to user feedback. The invention realizes the mining and the utilization of deeper semantic association of the multi-mode data resource, and improves the accuracy, the completeness and the interpretability of the retrieval result.

Inventors

SUN SHUMENG
YAN MIN
LIU CHAO
ZHANG DANDAN
GONG CHUYUN

Assignees

北京亦庄智能城市研究院集团有限公司

Dates

Publication Date: 20260512
Application Date: 20260415

Claims (10)

1. The data resource accurate processing and sharing method based on multi-mode semantic association analysis is characterized by comprising the following steps of: Acquiring a query request to be retrieved and multi-modal input data corresponding to the query request, respectively extracting features of each modal data in the multi-modal input data to obtain feature representations of all modes, and projecting the feature representations of all modes into a unified semantic space based on cross-modal semantic mapping relation to obtain fusion semantic features; Constructing a semantic enhancement graph structure, taking the fused semantic features as node features, taking semantic association relations between data resources as edge connections, and carrying out multi-hop semantic reasoning through a graph propagation mechanism to obtain an expanded semantic representation; Constructing a mixed semantic query vector according to the fusion semantic features and the expanded semantic representations, and performing multi-granularity semantic matching in a data resource base by utilizing the mixed semantic query vector to obtain a candidate retrieval result set; Establishing a semantic confidence evaluation mechanism, and performing confidence weighted sorting on the candidate search result set by calculating the comprehensive confidence of cross-modal semantic consistency, semantic integrity and semantic interpretability between each data resource in the candidate search result set and the mixed semantic query vector to generate an accurate search result; Extracting a semantic preference evolution mode according to the interactive feedback behavior of the user on the accurate search result, and carrying out self-adaptive adjustment on the edge weight of the semantic enhancement graph structure by utilizing the semantic preference evolution mode.
2. The method of claim 1, wherein constructing a semantic enhancement graph structure, using the fused semantic features as node features, using semantic association relationships between data resources as edge connections, and performing multi-hop semantic reasoning through a graph propagation mechanism, and obtaining an extended semantic representation comprises: Respectively taking each data resource and fusion semantic feature in a data resource library as graph nodes, establishing directed edge connection reflecting semantic dependency strength through calculating cross-modal semantic coupling degree and semantic structure similarity among the nodes, and distributing edge weight based on semantic transfer credibility for each edge to construct a semantic enhancement graph structure; In the semantic enhancement graph structure, starting from a query node, carrying out layer-by-layer semantic diffusion along edge weights, carrying out path-dependent weighted fusion on semantic features reaching the node in each hop of propagation, and carrying out dynamic cut-off control on propagation depth according to accumulated path weights to obtain a multi-hop reachable node set and a corresponding path aggregation semantic representation thereof; And carrying out semantic space alignment on the path aggregation semantic representation and the fusion semantic features, and generating an expansion semantic representation through a graph propagation mechanism.
3. The method of claim 2, wherein in the semantic enhancement graph structure, starting from a query node, performing hierarchical semantic diffusion along edge weights, performing path-dependent weighted fusion on semantic features reaching the node in each hop propagation, and performing dynamic truncation control on propagation depth according to accumulated path weights, to obtain a multi-hop reachable node set and a corresponding path-aggregated semantic representation thereof comprises: setting a node corresponding to the fusion semantic features as a query node, and initializing the propagation state and the accumulated path weight of the query node; starting from the query node, performing first jump semantic diffusion along directed edge connection, performing weighted combination on the original semantic features of the reached neighborhood nodes and the semantic features of the query node according to connection edge weights, generating first jump path fusion features, and recording accumulated path weights of all the neighborhood nodes; Judging whether accumulated path weights of all neighborhood nodes corresponding to the first hop path fusion features meet a semantic transfer effectiveness threshold value or not, continuing to perform next hop semantic diffusion on nodes meeting the conditions, performing propagation interception on nodes not meeting the conditions, performing new round of weighted combination on the path fusion features of the previous hop serving as input features of the current hop in the next hop diffusion, and iteratively updating the accumulated path weights until all propagation paths are intercepted or reach a preset propagation layer number to obtain a multi-hop reachable node set; And carrying out global weighted aggregation on the path fusion characteristics of each node in the multi-hop reachable node set according to the final accumulated path weight of each node to generate path aggregation semantic representation.
4. The method of claim 1, wherein constructing a hybrid semantic query vector from the fused semantic features and the expanded semantic representations and performing multi-granularity semantic matching in a data resource base using the hybrid semantic query vector, the obtaining a candidate search result set comprising: calculating the semantic complementarity between the fusion semantic features and the expansion semantic representation, determining the self-adaptive fusion weights of the fusion semantic features and the expansion semantic representation according to the semantic complementarity, and carrying out weighted fusion on the fusion semantic features and the expansion semantic representation according to the self-adaptive fusion weights to construct a hybrid semantic query vector; Projecting the mixed semantic query vector to a plurality of semantic representation spaces with different granularities, generating a local semantic matching vector in the fine granularity semantic representation space, and generating a global semantic matching vector in the coarse granularity semantic representation space; Calculating local semantic similarity by using the local semantic matching vector and the fine granularity characteristic representation of each data resource in the data resource library, calculating global semantic similarity by using the global semantic matching vector and the coarse granularity characteristic representation of each data resource, and carrying out self-adaptive weighted combination on the local semantic similarity and the global semantic similarity to obtain comprehensive matching score; And sorting and screening the data resources in the data resource base according to the comprehensive matching score to obtain a candidate retrieval result set.
5. The method of claim 1, wherein establishing a semantic confidence assessment mechanism, performing confidence weighted ranking on the candidate search result set by calculating a comprehensive confidence of cross-modal semantic consistency, semantic integrity, and semantic interpretability between each data resource in the candidate search result set and the hybrid semantic query vector, the generating a precise search result comprising: Establishing a semantic confidence evaluation mechanism, wherein the semantic confidence evaluation mechanism comprises a cross-modal consistency evaluation module, a semantic integrity evaluation module and a semantic interpretability evaluation module, and the cross-modal consistency evaluation module quantifies the degree of synergy and the degree of conflict between semantic expressions of different modalities by establishing a semantic consistency constraint matrix among modalities to obtain a cross-modal consistency metric value; The semantic integrity evaluation module maps semantic features of each data resource to a node space and calculates coverage rate based on a semantic element dependency graph obtained by decomposing the mixed semantic query vector, and performs compensation evaluation on semantic missing nodes by combining with an implicit semantic path in the expanded semantic representation to obtain a semantic integrity metric value; Extracting a semantic matching attribution chain based on the semantic interpretability evaluation module, quantifying the interpretable intensity according to the semantic transmission transparency, and obtaining a semantic interpretability metric value; Inputting the cross-modal consistency metric value, the semantic integrity metric value and the semantic interpretability metric value into a confidence fusion function according to the semantic confidence assessment mechanism, calculating the comprehensive confidence of each data resource, and sequencing according to the comprehensive confidence to generate a precise search result.
6. The method of claim 5, wherein inputting the cross-modality consistency metric, the semantic integrity metric, and the semantic interpretability metric into a confidence fusion function according to the semantic confidence assessment mechanism, the computing the integrated confidence for each data resource comprises: Analyzing the cross-modal consistency metric value, semantic dependency relationship between the semantic integrity metric value and the semantic interpretability metric value, identifying confidence transfer paths and transfer weights thereof between different metric values, and establishing a correlation structure between metric dimensions; Performing association strength propagation based on the association structure, transmitting confidence influence to the related metric values along the confidence transmission path by each metric value, and performing association enhancement adjustment on each metric value according to the transmission weight to obtain an adjusted cross-mode consistency metric value, an adjusted semantic integrity metric value and an adjusted semantic interpretability metric value; Analyzing the modal distribution characteristics and the semantic complexity of the mixed semantic query vector, and determining the adjusted cross-modal consistency metric value, the adjusted semantic integrity metric value and the adaptive fusion weight of the adjusted semantic interpretability metric value according to the modal distribution characteristics and the semantic complexity; And carrying out weighted combination on the adjusted cross-modal consistency metric value, the adjusted semantic integrity metric value and the adjusted semantic interpretability metric value according to the self-adaptive fusion weight input confidence fusion function to obtain the comprehensive confidence of each data resource.
7. The method of claim 1, wherein extracting a semantic preference evolution mode according to the interactive feedback behavior of the user on the accurate search result, and using the semantic preference evolution mode to adaptively adjust the edge weight of the semantic enhancement graph structure comprises: Collecting interactive feedback behavior sequences of users on all data resources in accurate search results, extracting semantic feature patterns focused by the users from the interactive feedback behavior sequences, and identifying evolution directions and evolution rates of semantic preferences of the users by analyzing semantic feature variation trends of the user interactive behaviors in different time windows to obtain semantic preference evolution patterns; Mapping the semantic preference evolution mode to a semantic enhancement graph structure, identifying a semantic association path related to the semantic preference evolution mode, carrying out enhancement adjustment on the edge weight on the semantic association path according to the evolution direction in the semantic preference evolution mode, determining the adjustment amplitude of the edge weight according to the evolution rate, and completing the edge weight self-adaptive adjustment of the semantic enhancement graph structure.
8. A data resource accurate processing and sharing system based on multi-modal semantic association analysis for implementing the method of any of the preceding claims 1-7, comprising: The feature extraction unit is used for acquiring a query request to be retrieved and corresponding multi-modal input data thereof, respectively extracting features of each modal data in the multi-modal input data to obtain feature representations of all modes, and projecting the feature representations of all modes to a unified semantic space based on a cross-modal semantic mapping relation to obtain fusion semantic features; The semantic fusion unit is used for constructing a semantic enhancement graph structure, taking the fusion semantic features as node features, taking semantic association relations between data resources as edge connection, and carrying out multi-hop semantic reasoning through a graph propagation mechanism to obtain an expanded semantic representation; The graph reasoning unit is used for constructing a mixed semantic query vector according to the fusion semantic features and the expansion semantic representations, and carrying out multi-granularity semantic matching in a data resource base by utilizing the mixed semantic query vector to obtain a candidate retrieval result set; The query construction unit is used for establishing a semantic confidence evaluation mechanism, and performing confidence weighted sequencing on the candidate search result set by calculating the comprehensive confidence of cross-modal semantic consistency, semantic integrity and semantic interpretability between each data resource in the candidate search result set and the mixed semantic query vector to generate an accurate search result; and the matching retrieval unit is used for extracting a semantic preference evolution mode according to the interactive feedback behavior of the user on the accurate retrieval result, and carrying out self-adaptive adjustment on the edge weight of the semantic enhancement graph structure by utilizing the semantic preference evolution mode.
9. An electronic device, comprising: A processor; A memory for storing processor-executable instructions; Wherein the processor is configured to invoke the instructions stored in the memory to perform the method of any of claims 1 to 7.
10. A computer readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the method of any of claims 1 to 7.

Description

Data resource accurate processing and sharing method based on multi-mode semantic association analysis Technical Field The invention relates to a semantic processing technology, in particular to a data resource accurate processing and sharing method based on multi-mode semantic association analysis. Background In the prior art, retrieval and sharing of data resources typically rely on single modality or simply fused semantic matching methods. The conventional method is that feature extraction is respectively carried out on data of different modes such as text, images, audio and the like, similarity calculation is carried out through a vector space model or a shallow fusion network, and matching is carried out in a database so as to return a relevant result. The method can process multi-mode data to a certain extent, but the core of the method is often limited to direct and single-hop semantic comparison between the query request and the data resources, and the method lacks of effective mining and utilization of deep and complex semantic association relations between the data resources. In addition, the existing method generally regards the retrieval process as one-time static matching, and dynamic semantic preference information contained in user interaction feedback cannot be fully considered, so that the retrieval performance of the system is difficult to continuously optimize. The conventional method has obvious defects that the retrieval process is easily restricted by the problem of semantic gap due to lack of deep reasoning of implicit semantic association between data resources. When the user query intention is complex or the semantic expression of the data resource is incomplete, only high-value resources which are difficult to recall in semantic correlation and are not matched in surface features are relied on by direct matching, so that the recall rate and the accuracy of the retrieval result are limited. The existing system usually adopts a fixed matching model or weight, and cannot perform self-adaptive learning and adjustment according to actual use behaviors and feedback of users. The search system lacks individuation capability, is difficult to track and adapt to dynamic evolution of semantic preference of the user, and can lead to gradual increase of deviation between a search result and real demands of the user in the long term, so that the accurate utilization efficiency of shared data resources is affected. Disclosure of Invention The embodiment of the invention provides a data resource accurate processing and sharing method based on multi-mode semantic association analysis, which can solve the problems in the prior art. In a first aspect of the embodiment of the present invention, a method for precisely processing and sharing data resources based on multi-mode semantic association analysis is provided, including: Acquiring a query request to be retrieved and multi-modal input data corresponding to the query request, respectively extracting features of each modal data in the multi-modal input data to obtain feature representations of all modes, and projecting the feature representations of all modes into a unified semantic space based on cross-modal semantic mapping relation to obtain fusion semantic features; Constructing a semantic enhancement graph structure, taking the fused semantic features as node features, taking semantic association relations between data resources as edge connections, and carrying out multi-hop semantic reasoning through a graph propagation mechanism to obtain an expanded semantic representation; Constructing a mixed semantic query vector according to the fusion semantic features and the expanded semantic representations, and performing multi-granularity semantic matching in a data resource base by utilizing the mixed semantic query vector to obtain a candidate retrieval result set; Establishing a semantic confidence evaluation mechanism, and performing confidence weighted sorting on the candidate search result set by calculating the comprehensive confidence of cross-modal semantic consistency, semantic integrity and semantic interpretability between each data resource in the candidate search result set and the mixed semantic query vector to generate an accurate search result; Extracting a semantic preference evolution mode according to the interactive feedback behavior of the user on the accurate search result, and carrying out self-adaptive adjustment on the edge weight of the semantic enhancement graph structure by utilizing the semantic preference evolution mode. Constructing a semantic enhancement graph structure, taking the fused semantic features as node features, taking semantic association relations between data resources as edge connections, and carrying out multi-hop semantic reasoning through a graph propagation mechanism, wherein the obtaining of the expanded semantic representation comprises the following steps: Respectively taking each data resour