CN-121981595-A - Question and evaluation generation system and method for cognizing intelligent agent by integrating expert preference elements

CN121981595ACN 121981595 ACN121981595 ACN 121981595ACN-121981595-A

Abstract

The invention relates to a question and evaluation generating system and method of a fusion expert preference element cognition intelligent agent, belonging to the technical field of intelligent education, wherein the system comprises a question automatic generating module, a question and evaluation generating module and a question evaluation generating module, wherein the question automatic generating module is used for calling a large language model to generate a question and an evaluation according to a subject information base and an expert preference model; the system comprises a meta-cognition intelligent agent evaluation optimization module, an expert interaction module and a feedback information processing module, wherein the meta-cognition intelligent agent evaluation optimization module is used for calling the meta-cognition intelligent agent to carry out multidimensional evaluation on questions and evaluation according to an expert preference model and generate evaluation results and optimization suggestions, and the expert interaction module is used for obtaining feedback information of the experts on the questions, the evaluation results and the optimization suggestions, and adjusting model parameters of the expert preference model, the large language model and the meta-cognition intelligent agent according to the feedback information if the feedback information indicates that the experts are not approved until the experts are approved to pass. The invention establishes a deep cooperative mechanism between the expert and the artificial intelligence, and guides the artificial intelligence to learn the expert experience and preference through interaction, so as to generate high-quality questions, assessment and evaluation results.

Inventors

LOU PING
ZENG YUHANG
Fan Chuannian
MA KE
Tai Xiu

Assignees

武汉理工大学

Dates

Publication Date: 20260505
Application Date: 20251230

Claims (10)

1. The title and evaluation generation system of the fusion expert preference element cognitive agent is characterized by comprising the following components: the automatic question generation module is used for calling the large language model to generate questions and comments according to the subject information base and the expert preference model; The meta-cognition agent evaluation optimization module is used for calling the meta-cognition agent to carry out multidimensional evaluation on the questions and the evaluation according to the expert preference model and generating an evaluation result and an optimization suggestion; and the expert interaction module is used for acquiring feedback information of the expert on the topic, the evaluation result and the optimization suggestion, if the feedback information indicates that the expert passes the verification, the expert preference model is adjusted according to the feedback information, the feedback information is sent to the big language model and the meta-cognitive agent, so that the big language model and the meta-cognitive agent optimize the topic, the evaluation result and the optimization suggestion after parameter adjustment according to the feedback information until the feedback information indicates that the expert passes the verification or the interactive iteration number reaches a frequency threshold.
2. The system for generating topics and evaluation of fusion expert preference element cognitive agents of claim 1, further comprising an information base construction module, wherein the information base construction module comprises: a content point extraction unit for extracting content points of subjects from the subject data; the relationship construction unit is used for determining the association relationship of each content point and constructing a knowledge graph of the content point according to the association relationship; the attribute labeling unit is used for labeling attribute information for the nodes in the knowledge graph, wherein the attribute information comprises difficulty and importance; and the discipline information base construction unit is used for constructing a discipline information base according to the noted knowledge graph.
3. The system for generating topics and comments for fusion expert preference element cognitive agents of claim 1, wherein said automated question generation module comprises: the content point selection unit is used for determining content points to be examined from the subject information base; the system comprises a question type generating unit, a question type processing unit and a question type processing unit, wherein the question type generating unit is used for determining the multidimensional characteristic of the content point to be checked, and calculating the comprehensive matching degree of the content point to be checked and each question type according to the multidimensional characteristic; and the content generating unit is used for calling a large language model to generate questions and evaluation according to the content points to be examined, the title type and the expert preference model.
4. The system for generating topics and evaluation of fusion expert preference element cognitive agents according to claim 3, wherein the content point selection unit is used for determining a plurality of candidate content points from the subject information base according to a preset content point inspection coverage range, dividing the product of difficulty, importance and weight coefficients of single content points in the candidate content points by the sum of products corresponding to the candidate content points to obtain the selection probability of the single content point, and selecting the content point to be inspected from the candidate content points according to the selection probability.
5. The system for generating topics and comments for fusion expert preference element cognitive agents of claim 3, wherein said automated question generation module further comprises: The RAG unit is used for searching in the subject information base, the verified questions and the evaluation according to the content points to be examined to obtain reference content points, reference questions and reference evaluation; the content generating unit is used for calling a large language model to generate questions and comments according to the content points to be checked, the reference content points, the reference questions, the reference comments, the title type and the expert preference model.
6. The system for generating topics and comments for fusion expert preference element cognitive agents of claim 1, wherein said expert interaction module comprises: the professional auditing unit is used for displaying a professional term library tool and term definitions in the questions and the evaluation in the auditing process of the experts on the questions and the evaluation; the culture value evaluation unit is used for determining the culture value of the title according to the culture target and displaying the culture value in the examination process of the title by the expert; The multi-mode support unit is used for displaying a chart editor, a formula editor and an experimental scene design tool in the process of examining and evaluating the questions and the assessment by the expert; The style adjustment unit is used for displaying a question style template and a style adjustment tool in the examination and evaluation process of the expert on the questions and the evaluation; the content point coverage auditing unit is used for displaying the content point coverage information of the title in the auditing process of the expert on the title and the evaluation; And the multi-round interactive feedback unit is used for acquiring feedback information of the subject, the evaluation result and the optimization suggestion from an expert, adjusting the expert preference model according to the feedback information if the feedback information indicates that the expert is not approved, and sending the feedback information to the large language model and the meta-cognitive intelligent body so that the large language model and the meta-cognitive intelligent body optimize the subject, the evaluation result and the optimization suggestion after parameter adjustment according to the feedback information until the feedback information indicates that the expert is approved or the interactive iteration number reaches a frequency threshold.
7. The system for generating questions and comments for fusion expert preference element cognitive agents of claim 1, wherein said multi-dimensional assessments include a question and answer logic consistency assessment, a content point accuracy assessment, a historical question repeatability assessment, a diversity assessment and a difficulty assessment.
8. The fusion expert preference meta-cognitive agent topic and assessment generation system of claim 7 wherein the meta-cognitive agent assessment optimization module comprises: And the difficulty calibration unit is used for evaluating the difficulty of the title according to the language complexity, the thinking level, the reasoning step and the covered content points of the title.
9. The system for generating topics and evaluations of fusion expert preference element cognitive agents of claim 1, further comprising: The content library management module is used for storing content points, approved questions and evaluation, and displaying the content points, the questions and evaluation inquiry management tools; and the data analysis and feedback module is used for collecting answer information of the students on the approved questions and generating evaluation and optimization suggestions of the questions according to the answer information.
10. The title and evaluation generation method of the fusion expert preference element cognition agent is characterized by being applied to the title and evaluation generation system of the fusion expert preference element cognition agent in any one of claims 1 to 9, and the method comprises the following steps: Calling a large language model to generate questions and comments according to the subject information base and the expert preference model; performing multidimensional evaluation on the questions and the evaluation according to the expert preference model, and generating an evaluation result and an optimization suggestion; And acquiring feedback information of the expert on the title, the evaluation result and the optimization suggestion, if the feedback information indicates that the expert is not approved, adjusting the expert preference model according to the feedback information, and sending the feedback information to the large language model and the meta-cognitive intelligent body so that the large language model and the meta-cognitive intelligent body optimize the title, the evaluation result and the optimization suggestion after parameter adjustment according to the feedback information until the feedback information indicates that the expert is approved or the number of interactive iterations reaches a frequency threshold.

Description

Question and evaluation generation system and method for cognizing intelligent agent by integrating expert preference elements Technical Field The invention relates to the technical field of intelligent education, in particular to a topic and evaluation generation system and method of a cognitive agent integrating expert preference elements. Background In the current educational evaluation system, the creation of questions and evaluation faces multiple challenges, wherein the questions refer to the questions stems, options (part of questions have options and part of questions do not have options) and standard answers, and the evaluation refers to the questions to analyze and score the basis. The traditional method is mainly finished manually, has low efficiency and unstable quality, and is easy to repeat. With the development of artificial intelligence technology, especially the progress of natural language processing and generating model, several test question automatic generating technologies are presented. The techniques are generally based on template filling, rule reasoning or large-scale language models, and can realize batch generation of questions and evaluation to a certain extent, so that the efficiency is improved. However, the existing automatic generation technology also has obvious limitation that the current technology path generally positions the artificial intelligence as a substitute or auxiliary tool, the work flow is generally 'machine generation, manual auditing and correction', and the expert only serves as a final auditor and is not a cooperated creator, so that deep cooperation of the expert and the artificial intelligence cannot be realized. Disclosure of Invention In view of the foregoing, it is necessary to provide a system and a method for generating questions and comments for fusing expert preference element cognition agents, which are used for solving the problem that the existing automatic questions and comments generation technology cannot realize deep collaboration of experts and artificial intelligence. In order to solve the above problems, in a first aspect, the present invention provides a topic and evaluation generation system for fusing expert preference element cognitive agents, including: the automatic question generation module is used for calling the large language model to generate questions and comments according to the subject information base and the expert preference model; The meta-cognition agent evaluation optimization module is used for calling the meta-cognition agent to carry out multidimensional evaluation on the questions and the evaluation according to the expert preference model and generating an evaluation result and an optimization suggestion; and the expert interaction module is used for acquiring feedback information of the expert on the topic, the evaluation result and the optimization suggestion, if the feedback information indicates that the expert passes the verification, the expert preference model is adjusted according to the feedback information, the feedback information is sent to the big language model and the meta-cognitive agent, so that the big language model and the meta-cognitive agent optimize the topic, the evaluation result and the optimization suggestion after parameter adjustment according to the feedback information until the feedback information indicates that the expert passes the verification or the interactive iteration number reaches a frequency threshold. In one possible implementation manner, the system further comprises an information base construction module, wherein the information base construction module comprises: a content point extraction unit for extracting content points of subjects from the subject data; the relationship construction unit is used for determining the association relationship of each content point and constructing a knowledge graph of the content point according to the association relationship; the attribute labeling unit is used for labeling attribute information for the nodes in the knowledge graph, wherein the attribute information comprises difficulty and importance; and the discipline information base construction unit is used for constructing a discipline information base according to the noted knowledge graph. In one possible implementation, the automatic problem generation module includes: the content point selection unit is used for determining content points to be examined from the subject information base; the system comprises a question type generating unit, a question type processing unit and a question type processing unit, wherein the question type generating unit is used for determining the multidimensional characteristic of the content point to be checked, and calculating the comprehensive matching degree of the content point to be checked and each question type according to the multidimensional characteristic; and the content generating unit is used for calling a large language