CN-122019618-A - Enterprise innovation portrait construction and recommendation method and system based on multi-mode RAG

CN122019618ACN 122019618 ACN122019618 ACN 122019618ACN-122019618-A

Abstract

The invention discloses a method and a system for constructing and recommending enterprise innovation portraits based on multi-mode RAG, and relates to the technical field of artificial intelligence. And constructing an enterprise semantic tree and a comprehensive characterization vector by performing recursive semantic aggregation on the set. And secondly, mapping the semantic tree into an enterprise innovation knowledge graph, and carrying out fact verification and semantic rewriting on the user query based on the graph and an external trusted knowledge source to generate an optimized query instruction. And finally, performing dual-path recommendation, namely performing multi-granularity matching on the vector retrieval path by utilizing an enterprise semantic tree and a unified characterization library, returning a related enterprise list, and generating an interpretable recommendation report which comprises detailed reasons and comparative analysis by using the semantic tree, the characterization library and the optimized query as a combined knowledge source and driving the large language model by the generated reasoning path. The invention realizes the deep description and the interpretable intelligent recommendation of the innovation capability of enterprises.

Inventors

JIANG JIANJIAN
LAI PEIYUAN
DAI QINGYUN

Assignees

广东工业大学
广东技术师范大学

Dates

Publication Date: 20260512
Application Date: 20251230

Claims (9)

1. A method for constructing and recommending enterprise innovation portraits based on multi-mode RAG is characterized by comprising the following steps: Collecting text, images and relation data of enterprises, analyzing the collected multi-source heterogeneous data and converting the multi-source heterogeneous data into a structured semantic object set in a unified format; Performing recursive semantic aggregation on the structured semantic object set, constructing a top semantic vector representing the core innovation capability of an enterprise from bottom to top, generating an interpretable enterprise semantic tree, fusing the top semantic vector through a cross-modal attention mechanism, outputting the fused semantic vector as an enterprise comprehensive characterization vector, and constructing a unified characterization library; Mapping the entity and the relation in the enterprise semantic tree into a structured enterprise innovation knowledge graph, receiving a user natural language query, and carrying out fact verification and semantic rewriting in the enterprise innovation knowledge graph and an external trusted knowledge source based on a large language model to generate an optimized structured query instruction; Based on the optimized structured query instruction, executing dual-path recommendation comprising a vector retrieval path and a generated reasoning path; The vector retrieval path carries out multi-granularity semantic matching on the optimized structured query instruction based on the index constructed by the enterprise semantic tree and the unified characterization library, retrieves and returns a related enterprise list, and the generated reasoning path drives a large language model to generate an interpretable recommendation report containing reasons and comparative analysis by taking the enterprise semantic tree, the unified characterization library and the optimized structured query instruction as a joint enhancement knowledge source.
2. The method for constructing and recommending enterprise innovation portraits based on multi-modal RAG according to claim 1, wherein collecting text, image and relational data of the enterprise, analyzing and converting the collected multi-source heterogeneous data into a structured semantic object set in a unified format, comprises: through a public API, a web crawler and a cooperative data interface, multi-source heterogeneous data of a target enterprise are collected, wherein the multi-source heterogeneous data comprise text data, image data and relationship data; Processing the text data by adopting a multi-level semantic analysis model, extracting basic semantic units corresponding to technical entities, performance indexes, method steps and application scenes, acquiring semantic association of the basic semantic units, dynamically combining the basic semantic units with the association into a composite semantic block based on the semantic association, and taking the composite semantic block as a structural semantic object, wherein the type of the composite semantic block comprises a technical scheme semantic block and an application result semantic block, and recording the logic relationship among the internal units; Analyzing the image data by adopting a visual language model to generate a structural description tuple containing a visual main body, a functional description and related technical item fields as a visual semantic object; mapping the relation data into primary triplet objects with (head entity, relation type, tail entity) as format; and respectively converting all the generated structured semantic objects, visual semantic objects and primary triplet objects into structured semantic object sets in a unified format.
3. The method of claim 1, wherein performing recursive semantic aggregation on the structured semantic object sets, building top-level semantic vectors representing enterprise core innovation capabilities from bottom to top, and generating interpretable enterprise semantic trees comprises: Vectorizing the finest grain semantic objects in the structured semantic object sets of different modes, executing bottom-up iterative semantic abstraction, and clustering semantic object vectors of the current level in each iteration based on semantic similarity; Calling an abstract model for the generated cluster to generate a high-level semantic abstract vector for representing the overall semantic of the cluster and a corresponding natural language abstract description text; Performing context-aware correction on the generated high-level semantic abstract vectors by using a cross-level attention mechanism and utilizing semantic object vectors in the current cluster, lower-level semantic object vectors and other related high-level semantic abstract vectors at the same layer; And repeating the iterative loop until a preset abstraction level is reached, outputting hierarchical enterprise semantic trees of different modes, wherein leaf nodes of the enterprise semantic trees are initial semantic objects, and top-level nodes are top-level semantic vector sets representing the innovation capability of enterprise cores.
4. The method for constructing and recommending enterprise innovation portraits based on multi-modal RAG of claim 3, wherein the fusing the top-level semantic vectors through a cross-modal attention mechanism, outputting as enterprise comprehensive characterization vectors and constructing a unified characterization library comprises: acquiring top-level semantic vector sets of different modes, and inputting the text top-level semantic vector set, the image top-level semantic vector set and the relation top-level semantic vector set into a cross-mode attention fusion layer; In the cross-modal attention fusion layer, interaction, alignment and importance weighting are carried out on top-level semantic vectors of different modes in a semantic space through an attention mechanism, interaction and weighted multi-mode semantic information are integrated, fused enterprise comprehensive characterization vectors are output, and all enterprise comprehensive characterization vectors are stored to form an enterprise unified characterization library.
5. The method for constructing and recommending enterprise innovation portraits based on multi-modal RAG according to claim 1, wherein mapping the entities and relations in the enterprise semantic tree into a structured enterprise innovation knowledge graph comprises: Mapping each level semantic node in the enterprise semantic tree into a corresponding entity in a knowledge graph according to the node type, the level position and the abstract description text, and endowing each entity with a dynamic entity type determined by the enterprise semantic tree and a core attribute which can be traced back to an original data source; generating a longitudinal map relation according to a parent-child node hierarchical structure inherent in the enterprise semantic tree, acquiring association weights among different semantic nodes learned through a cross-level attention mechanism in a recursive semantic aggregation process, and automatically identifying and generating a transverse map relation according to the association weights, wherein the longitudinal map relation comprises 'containing', 'realizing' or 'decomposing into', and the transverse map relation comprises 'collaboration', 'supporting', 'applying to' or 'associating'; and constructing a structured enterprise innovation knowledge graph according to the entity, the longitudinal graph relationship and the transverse graph relationship, and taking the enterprise innovation knowledge graph as an enterprise innovation capability image.
6. The method for constructing and recommending enterprise innovation portraits based on multi-modal RAG according to claim 1, wherein receiving a user natural language query, performing fact verification and semantic rewrite in the enterprise innovation knowledge graph and an external trusted knowledge source based on a large language model, generating an optimized structured query instruction, comprising: After receiving a user natural language query, calling a large language model to analyze and decompose the natural language query into a plurality of atomic fact sub-queries, and carrying out dynamic evidence chain verification comprising internal atlas tracing and external knowledge completion on each atomic fact sub-query; In the internal atlas tracing, the atomic fact sub-query and the enterprise innovation knowledge atlas are interacted in real time, a minimum semantic evidence path capable of supporting or refuting the corresponding atomic fact sub-query is constructed in the enterprise innovation knowledge atlas, the integrity, the matching degree and the attribute consistency of the minimum semantic evidence path are evaluated, and an internal evidence evaluation result is generated; In the external knowledge completion, extracting elements related to timeliness and consensus knowledge in atomic fact sub-queries, and sending the elements to an external trusted knowledge source for retrieval and comparison to obtain an external timeliness evidence result; carrying out weighted fusion and contradiction resolution on the internal evidence evaluation result and the external timeliness evidence result to generate a multidimensional confidence vector corresponding to each atomic fact sub-query, wherein the multidimensional confidence vector comprises a facts confidence, timeliness confidence and evidence conflict identification; based on the multidimensional confidence vector, the large language model is called again, a context-aware semantic rewrite strategy engine is driven to reconstruct the original natural language query, and an optimized structured query instruction, a description correction term and a natural language explanation based on the judgment are output.
7. The method for constructing and recommending enterprise innovation portraits based on multi-modal RAG of claim 1, wherein the vector search path comprises: Constructing a multi-granularity semantic index based on the enterprise semantic tree and the enterprise unified characterization library, storing enterprise comprehensive characterization vectors by the multi-granularity semantic index, associating intermediate semantic vectors generated in a recursive semantic aggregation process, binding all vectors with source nodes in the enterprise semantic tree, and recording a hierarchical relationship; Performing synchronous multi-granularity analysis and vectorization on the optimized structured query instruction to generate a global query vector representing the overall intention of the query and a local query vector representing specific constraints in the query; Performing similarity calculation by using the global query vector and enterprise comprehensive characterization vectors of all enterprises in the multi-granularity semantic index, and completing top-level coarse screening to obtain a preliminary candidate enterprise set; In the preliminary candidate enterprise set, respectively carrying out fine-grained similarity calculation on the local query vector and the intermediate semantic vector stored in the index of the preliminary candidate enterprise; And merging the matching scores calculated by the top coarse screen and the fine granularity similarity, carrying out weighted sorting on the primary candidate enterprises, generating a final relevant enterprise sorting list, and generating a primary structured matching reason according to a matching process.
8. The method for constructing and recommending enterprise innovation portraits based on multi-modal RAGs according to claim 1, wherein the generating an inference path comprises: The related enterprise ordered list is received as a candidate enterprise list, and the enterprise semantic tree, the enterprise unified characterization library and the optimized structured query instruction are fused and packaged in a structured manner to form a joint enhancement knowledge source which is input into a large language model; the enterprise semantic tree provides a logic concept level and entity relation network required by analysis, the unified characterization library provides enterprise comprehensive characterization vectors and relative position relations thereof for quantitative comparison, and the optimized query provides an accurate analysis task framework and constraint conditions; Generating a prompt based on the joint enhancement knowledge source, guiding the large language model to execute inference chain analysis, and identifying and explaining matching situations of candidate enterprises and query intentions on different abstraction levels according to the enterprise semantic tree and the optimized structured query instruction in the inference chain analysis; According to the relationship knowledge of the enterprise comprehensive characterization vector and the enterprise semantic tree in the unified characterization library, carrying out multi-dimensional comparison on the technical similarity, the capability structure difference and the evidence characteristics of candidate enterprises, and generating explanation reasons for each recommendation conclusion by combining knowledge bases; and integrating all outputs of the inference chain analysis according to a preset report template by the large language model to generate an interpretable recommendation report comprising a recommended enterprise list, detailed recommendation reasons, enterprise multidimensional comparison analysis and risk prompt based on analysis.
9. The enterprise innovation portrait construction and recommendation system based on the multi-mode RAG is characterized by being used for realizing the enterprise innovation portrait construction and recommendation method based on the multi-mode RAG as claimed in any one of claims 1 to 8, and comprises a multi-source heterogeneous data acquisition and access module, a semantic analysis and objectification module, a semantic aggregation and characterization learning module, a knowledge graph construction and portrait solidification module, a query intention understanding and purification enhancement module, a multi-granularity intelligent retrieval and matching module and a generation type reasoning and interpretable report generation module; the multi-source heterogeneous data acquisition and access module is responsible for standardized acquisition of multi-source heterogeneous data of enterprises from various internal and external data sources, and comprises texts, images and relation data; The semantic analysis and objectification module analyzes the collected multi-source heterogeneous data and converts the multi-source heterogeneous data into a structured semantic object set in a unified format, wherein the structured semantic object set comprises a structured semantic object, a visual semantic object and a primary triplet object; The semantic aggregation and characterization learning module processes the structured semantic object, the visual semantic object and the primary triplet object in parallel, executes recursive semantic aggregation, constructs a top-level semantic vector representing the core innovation ability of an enterprise from bottom to top, generates an interpretable enterprise semantic tree, fuses the top-level semantic vector through a cross-modal attention mechanism, outputs an enterprise comprehensive characterization vector and constructs a unified characterization library; The knowledge graph construction and image solidification module maps the entities and relations in the enterprise semantic tree into a structured enterprise innovation knowledge graph, and the structured enterprise innovation image is defined as the enterprise innovation image; The query intention understanding and purifying enhancement module receives a user natural language query, decomposes the natural language query into atomic fact sub-queries based on a large language model, performs fact verification in the enterprise innovation knowledge graph and an external trusted knowledge source, and invokes the large language model again to perform semantic rewriting based on a verification result to generate an optimized structured query instruction; The multi-granularity intelligent searching and matching module carries out multi-granularity semantic matching on the optimized structured query instruction based on the index constructed by the enterprise semantic tree and the unified characterization library, and searches and returns a relevant enterprise list; The generation type reasoning and interpretable report generation module takes the enterprise semantic tree, the unified characterization library and the optimized structured query instruction as a joint enhancement knowledge source, and drives a large language model to execute semantic alignment, multidimensional comparison and quantitative analysis based on the joint knowledge source to generate an interpretable recommendation report.

Description

Enterprise innovation portrait construction and recommendation method and system based on multi-mode RAG Technical Field The invention relates to the technical field of artificial intelligence, in particular to a method and a system for constructing and recommending enterprise innovation portraits based on multi-mode RAG. Background Currently, deep evaluation and intelligent recommendation for enterprise innovation capability have become urgent demands in the fields of industry analysis, investment decision-making and the like. However, the technical architecture of the existing enterprise information platform and various industrial atlas systems has obvious limitation in processing the construction of the innovative image of the enterprise, and the requirements of depth, dynamic and intelligent analysis in the professional scene are difficult to meet. The main defects in the prior art are concentrated in four aspects, namely, firstly, the information dimension is single, the structured business financial data is excessively relied on, and the effective semantic analysis and fusion capability on unstructured multi-mode data such as patents, technical reports, product images and the like is lacking, so that the portrait cannot deeply describe the technical kernel. Secondly, the image is static and shallow, and a dynamic knowledge system with clear association between entities and capable of supporting logical reasoning cannot be constructed by adopting a predefined label or keyword list, so that the dynamic process of enterprise innovation is difficult to reflect. And thirdly, the recommendation logic is simple and mechanical, matching is mainly performed based on industry classification or explicit supply chain relation, deep semantic mining such as technology similarity and innovation complementarity is lacked, and the accuracy of recommendation results is insufficient. Finally, man-machine interaction is not intelligent, a user needs to screen through fixed conditions, complex intentions cannot be flexibly expressed by using natural language, and the system is lack of capability in terms of natural language understanding and intention analysis. Although artificial intelligence frontier technologies such as retrieval enhancement generation (RAG) and Large Language Model (LLM) have demonstrated potential in the general field, it remains a blank how to systematically integrate these technologies in a vertical enterprise innovation analysis scenario, designing a set of end-to-end solutions from multi-modal data processing, dynamic knowledge building to generating intelligent recommendations. Therefore, the development of the method for constructing and recommending the enterprise innovation images, which can overcome the defects and realize depth, dynamics and interpretability, has important technical necessity and application value. Disclosure of Invention In order to solve the technical problems, the invention provides a multi-mode RAG-based enterprise innovation portrait construction and recommendation method and system, which realize deep, dynamic depiction and accurate and interpretable intelligent recommendation of enterprise innovation capability. The invention provides a multi-mode RAG-based enterprise innovation portrait construction and recommendation method, which comprises the following steps: Collecting text, images and relation data of enterprises, analyzing the collected multi-source heterogeneous data and converting the multi-source heterogeneous data into a structured semantic object set in a unified format; Performing recursive semantic aggregation on the structured semantic object set, constructing a top semantic vector representing the core innovation capability of an enterprise from bottom to top, generating an interpretable enterprise semantic tree, fusing the top semantic vector through a cross-modal attention mechanism, outputting the fused semantic vector as an enterprise comprehensive characterization vector, and constructing a unified characterization library; Mapping the entity and the relation in the enterprise semantic tree into a structured enterprise innovation knowledge graph, receiving a user natural language query, and carrying out fact verification and semantic rewriting in the enterprise innovation knowledge graph and an external trusted knowledge source based on a large language model to generate an optimized structured query instruction; Based on the optimized structured query instruction, executing dual-path recommendation comprising a vector retrieval path and a generated reasoning path; The vector retrieval path carries out multi-granularity semantic matching on the optimized structured query instruction based on the index constructed by the enterprise semantic tree and the unified characterization library, retrieves and returns a related enterprise list, and the generated reasoning path drives a large language model to generate an interpretable recommendation report