CN-122020672-A - Method and device for generating evaluation data and product

Abstract

The disclosure provides a method, a device, electronic equipment, a storage medium and a computer program product for generating evaluation data, relates to the technical field of computers, in particular to the technical fields of large models, knowledge maps and the like, and can be applied to the generation scene of the evaluation data. The method comprises the steps of structuring business original data, generating a knowledge graph representing the association relation between entities in the business original data, and generating evaluating data of a model to be evaluated according to the knowledge graph through an artificial intelligent large model. The method and the device are based on the structural advantages of the knowledge graph, help to solve the problems of entity confusion, logic fracture, fuzzy relation and the like, superimpose the efficient generation capability of the artificial intelligence large model, and improve the generation quality, the credibility and the generation efficiency of the evaluation data.

Inventors

• LV HONG
• YANG JIAN

Assignees

• 北京百度网讯科技有限公司

Dates

Publication Date

20260512

Application Date

20260212

Claims (20)

1. 1. A method for generating evaluation data comprises the following steps: structuring business original data, and generating a knowledge graph representing the association relation between entities in the business original data; and generating evaluation data of the model to be evaluated according to the knowledge graph through the artificial intelligence large model.
2. 2. The method of claim 1, wherein the structuring process business-native data generates a knowledge graph that characterizes associations between entities in the business-native data, comprising: And adopting a processing mode corresponding to the generation scene of the evaluation data to structuralized process the business original data, and generating a knowledge graph representing the association relation between the entities in the business original data.
3. 3. The method of claim 2, wherein the generation scenario comprises a data query scenario, the business-native data is description data of a database structure, and The step of adopting a processing mode corresponding to the generation scene of the evaluation data to structuralized process the business original data to generate a knowledge graph representing the association relationship between entities in the business original data comprises the following steps: Determining association relations among entities of multiple levels in the description data, and generating multiple triples, wherein the triples represent the association relations between two entities; and combining a plurality of triples to generate a knowledge graph under the data query scene.
4. 4. A method according to claim 3, wherein said determining the association between entities of multiple levels in the description data generates multiple triples, comprising: extracting association relations between a plurality of hierarchical entities and a plurality of hierarchical entities in the description data, wherein the plurality of hierarchical entities comprise data table entities, field entities in a data table, field value entities under fields and constraint rule entities under fields; and generating a plurality of triples according to the association relation.
5. 5. The method of claim 4, wherein the association comprises an association between the data table entity and the field entity, an association between the field entity and the constraint rule entity, an association between the field entity and the field value entity, and an association between the data table entity.
6. 6. The method of claim 4, wherein the combining the plurality of triples to generate a knowledge graph in the data query scenario comprises: And merging the triples comprising the same entity in the triples, taking the data table entity as a primary node, the field entity as a secondary node, and the constraint rule entity and the field value entity as tertiary nodes to generate a knowledge graph under the data query scene.
7. 7. The method according to any one of claims 3-6, wherein the generating the evaluation data of the model to be evaluated according to the knowledge-graph comprises: Grouping field entities in the description data according to service scenes to obtain a plurality of service groups; Combining a plurality of service groups according to service correlation among the service groups to obtain service group combination; And generating evaluation data comprising an evaluation request and a structured query statement according to the relation chain corresponding to the service group combination in the knowledge graph.
8. 8. The method of claim 7, wherein the evaluation data comprises single-round evaluation data, and The generating, according to the relation chain corresponding to the service group combination in the knowledge graph, evaluation data including an evaluation request and a structured query statement includes: And generating single-round evaluation data comprising an evaluation request and a structured query statement according to the relation chain, the evaluation sample and the query type which are required to be covered by the single-round evaluation data and correspond to the service group combination in the knowledge graph.
9. 9. The method of claim 8, wherein the generating single-round profile including the evaluation request and the structured query statement according to the relation chain, the evaluation sample, and the query type required to be covered by the single-round profile corresponding to the service group combination in the knowledge graph includes: Determining a chain type of the relation chain, wherein the chain type comprises a single-table chain which characterizes entities in the relation chain and is covered by one data table and a multi-table chain which is covered by a plurality of data tables in a combined way; and generating the single-round evaluation data according to the relation chain, the evaluation sample, the query type and the evaluation data generation rule corresponding to the chain type.
10. 10. The method of any of claims 7-9, wherein the evaluation data comprises multiple rounds of evaluation data, and The generating, according to the relation chain corresponding to the service group combination in the knowledge graph, evaluation data including an evaluation request and a structured query statement includes: And generating multiple rounds of evaluation data comprising evaluation requests and structured query sentences according to a relation chain corresponding to the service group combination and multiple rounds of query strategies in the knowledge graph, wherein the multiple rounds of query strategies characterize query logic among the multiple rounds of evaluation requests.
11. 11. The method of claim 2, wherein the generation scenario comprises a document question-answer scenario, the business native data is unstructured text data, and The step of adopting a processing mode corresponding to the generation scene of the evaluation data to structuralized process the business original data to generate a knowledge graph representing the association relationship between entities in the business original data comprises the following steps: Extracting triples representing subjects, predicates and objects from the data blocks of the unstructured text data; And combining a plurality of triples to generate a knowledge graph under the document question-answer scene.
12. 12. The method of claim 11, wherein prior to the extracting the triples characterizing subject, predicate and object from the data block of unstructured text data, further comprising: Determining a division mode of the unstructured text data according to the data capacity of the unstructured text data; dividing the unstructured text data by adopting the dividing mode to obtain a plurality of data blocks.
13. 13. The method according to claim 11, wherein the generating the evaluation data of the model to be evaluated according to the knowledge-graph includes: dividing the triples into scene groups according to the service scenes to which the triples belong; for a plurality of scene groups, determining the priority of the business scene corresponding to each scene group according to the correlation between the scene group and the unstructured text data; and generating evaluation data comprising an evaluation request and a standard answer according to the knowledge graph and the priority.
14. 14. The method of claim 13, wherein the dividing the triples into scene groups according to the traffic scenes to which the triples belong respectively comprises: Screening candidate keywords from the entities in the triples according to the occurrence frequency of the entities in the triples; determining a plurality of service scenes according to the candidate keywords, the triples and the fields of unstructured text data; And dividing the triples into a plurality of scene groups according to the correlation between the triples and the business scenes.
15. 15. The method of claim 13, wherein the determining, for the plurality of scene groups, priorities of the service scenes corresponding to the plurality of scene groups according to correlations of the scene groups with the unstructured text data, comprises: for a plurality of scene groups, determining scene keywords corresponding to the scene groups from the candidate keywords to obtain a scene keyword set, and obtaining a scene entity set based on entities in the scene groups; Determining a first degree of overlap between the set of scene keywords and a set of core keywords characterizing a topic of the unstructured text data; Determining a second degree of overlap between the set of scene entities and the set of core keywords; According to the first overlapping degree and the second overlapping degree, determining the correlation between the service scene corresponding to the scene group and the unstructured text data; and determining the priority among the plurality of business scenes according to the correlation corresponding to each of the plurality of business scenes.
16. 16. The method of any of claims 13-15, wherein the evaluation data comprises single-round evaluation data, and And generating evaluation data comprising an evaluation request and a standard answer according to the knowledge graph and the priority, wherein the evaluation data comprises: And generating single-round evaluation data comprising an evaluation request and a standard answer according to the knowledge graph, the priority, the type of the problem required to be covered by the evaluation data and a preset generation strategy of the evaluation data.
17. 17. The method of claim 16, wherein the generating single-round evaluation data including an evaluation request and a standard answer according to the knowledge graph, the priority, the type of the questions to be covered by the evaluation data, and a preset generation policy of the evaluation data comprises: Generating a plurality of initial evaluation data comprising an evaluation request and a standard answer according to the knowledge graph, the priority, the question type and the preset generation strategy; performing an integrity check operation and a semantic deduplication operation on the plurality of initial evaluation data to obtain a plurality of screened evaluation data; And carrying out quality evaluation on the screened evaluation data according to a plurality of preset evaluation dimensions, and determining the single-round evaluation data from the screened evaluation data according to an evaluation result.
18. 18. The method of claim 13, wherein the evaluation data comprises multiple rounds of evaluation data, and And generating evaluation data comprising an evaluation request and a standard answer according to the knowledge graph and the priority, wherein the evaluation data comprises: And generating multiple rounds of evaluation data comprising evaluation requests and standard answers according to the knowledge graph, the priority and multiple rounds of question-answering strategies, wherein the multiple rounds of question-answering strategies represent question-answering logic among the multiple rounds of evaluation requests.
19. 19. An apparatus for generating evaluation data, comprising: The map generation unit is configured to process the business native data in a structuring mode and generate a knowledge map representing the association relation between entities in the business native data; The data generation unit is configured to generate evaluation data of the model to be evaluated according to the knowledge graph through the artificial intelligence large model.
20. 20. An electronic device, comprising: at least one processor, and A memory communicatively coupled to the at least one processor, wherein, The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-18.

Description

Method and device for generating evaluation data and product Technical Field The disclosure relates to the technical field of computers, in particular to the technical fields of large models, knowledge maps and the like, and particularly relates to a method and a device for generating evaluation data, electronic equipment, a storage medium and a computer program product, which can be applied to the generation scene of the evaluation data. Background The evaluation data set is a core support for evaluating the effect of a large model product, but the construction processes of acquisition, labeling and the like of a high-quality data set are time-consuming and labor-consuming, and particularly, the method faces a long-tail scene difficult to acquire, and the screening and the generation of the evaluation data are needed to be realized through technical means. Disclosure of Invention The disclosure provides a method, a device, an electronic device, a storage medium and a computer program product for generating evaluation data. According to a first aspect, a method for generating evaluation data is provided, which comprises the steps of structuring service native data, generating a knowledge graph representing the association relation between entities in the service native data, and generating evaluation data of a model to be evaluated according to the knowledge graph through an artificial intelligence large model. According to a second aspect, an evaluation data generation device is provided, and the evaluation data generation device comprises a map generation unit and a data generation unit, wherein the map generation unit is configured to structurally process business native data and generate a knowledge map representing the association relation between entities in the business native data, and the data generation unit is configured to generate evaluation data of a model to be evaluated according to the knowledge map through an artificial intelligent large model. According to a third aspect there is provided an electronic device comprising at least one processor and a memory communicatively coupled to the at least one processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method as described in any one of the implementations of the first aspect. According to a fourth aspect, there is provided a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform a method as described in any implementation of the first aspect. According to a fifth aspect, there is provided a computer program product comprising a computer program which, when executed by a processor, implements a method as described in any of the implementations of the first aspect. According to the technology, a knowledge graph representing the association relation between entities in the business original data is generated by structuring the business original data, and the evaluation data of the model to be evaluated is generated according to the knowledge graph through the artificial intelligence large model, so that the problems of entity confusion, logic fracture, fuzzy relation and the like are solved based on the structuring advantage of the knowledge graph, the efficient generation capacity of the artificial intelligence large model is superposed, and the generation quality, the reliability and the generation efficiency of the evaluation data are improved. It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification. Drawings The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein: FIG. 1 is an exemplary system architecture diagram to which an embodiment according to the present disclosure may be applied; FIG. 2 is a flow chart of one embodiment of a method of generating profile according to the present disclosure; FIG. 3 is a block diagram of an evaluation data generation system according to the present embodiment; FIG. 4 is a flowchart of an evaluation data generation process in the data query scenario according to the present embodiment; FIG. 5 is a timing chart of an evaluation data generation process in a data query scene according to the present embodiment; FIG. 6 is a flowchart of an evaluation data generation process in the document question-answer scene according to the present embodiment; FIG. 7 is a time chart of an evaluation data generation process in a document question-answer scene according to the present embodiment; Fig. 8 is a schematic diagram of an application scenario of the method for generating an evaluation data according to the present embodiment; FIG. 9