CN-121998051-A - Expert agent-oriented iterative optimization system and method

CN121998051ACN 121998051 ACN121998051 ACN 121998051ACN-121998051-A

Abstract

The application provides an expert agent-oriented iterative optimization system and method, wherein the system comprises an expert data collection and labeling module, an expert agent development module, an ideal state feedback module, an effect evaluation module and a data production optimization module, wherein the expert data collection and labeling module is used for collecting expert data and conducting layered labeling to form an expert data set, the expert agent development module is used for training a large language model of an expert agent based on the expert data set, the ideal state feedback module is used for establishing a bidirectional interaction channel between the expert agent and the expert to enable the expert to provide expert opinion for output content of the expert agent, the effect evaluation module is used for establishing a comprehensive evaluation index system to evaluate operation effect of the expert agent to obtain an evaluation result, and the data production optimization module is used for conducting optimization adjustment on the expert data set based on the evaluation result and the expert opinion to enable the large language model to conduct iterative optimization training. The technical scheme solves the problems of serious bottleneck, lack of systematic closed-loop optimization mechanism and lack of real-time feedback mechanism in the prior expert system knowledge acquisition.

Inventors

CAI MEIJIE
LIU BAIFENG

Assignees

北京点富科技有限公司

Dates

Publication Date: 20260508
Application Date: 20251218

Claims (10)

1. An expert agent oriented iterative optimization system, the iterative optimization system comprising: The expert data collection and labeling module is used for collecting expert data and carrying out layered labeling on the expert data so as to form an expert data set; The expert agent development module is used for training a large language model of the expert agent based on the expert data set; The ideal state feedback module is used for establishing a bidirectional interaction channel between the expert intelligent bodies and the expert so that the expert provides expert opinion for the output content of the expert intelligent bodies; The effect evaluation module is used for establishing a comprehensive evaluation index system to evaluate the operation effect of the expert intelligent body so as to obtain an evaluation result; And the data production optimization module is used for carrying out optimization adjustment on the expert data set based on the evaluation result and the expert opinion so as to enable the expert agent development module to carry out iterative optimization training on the large language model.
2. The iterative optimization system of claim 1, wherein the expert data is divided into important data and regular data, and wherein the expert data collection and labeling module is configured to assign the important data to an expert for labeling and assign the regular data to a crowd-sourced for labeling.
3. The iterative optimization system of claim 1, wherein the comprehensive assessment index system comprises objective performance indexes and subjective experience indexes, wherein the objective indexes comprise any one or more basic assessment indexes of accuracy, recall, F1 value and AUC, and any one or more professional assessment indexes of sensitivity, specificity, response time, interpretability, generalization capability, adaptability and robustness, and the subjective indexes are feedback information collected according to user satisfaction investigation or expert assessment modes.
4. The iterative optimization system of claim 1, wherein the expert agent development module is configured to iteratively optimize the training data distribution using a gradient-based approach, dynamically adjust the amount of domain-specific data, and adaptively adjust based on its impact on the performance of the downstream task.
5. An expert agent-oriented iterative optimization method is characterized by comprising the following steps: collecting expert data and performing hierarchical annotation on the expert data to form an expert data set; Training a large language model of the expert agent based on the expert dataset; Establishing a bidirectional interaction channel between the expert intelligent body and the expert so that the expert provides expert opinion for the output content of the expert intelligent body; Establishing a comprehensive evaluation index system to evaluate the operation effect of the expert intelligent body to obtain an evaluation result; and optimizing and adjusting the expert data set based on the evaluation result and the expert opinion so as to perform iterative optimization training on the large language model of the expert intelligent body.
6. The iterative optimization method of claim 5, wherein the collecting and hierarchically labeling expert data to form an expert data set comprises assigning the important data to an expert for labeling and assigning the conventional data to crowd sourcing for labeling.
7. The iterative optimization method of claim 5, wherein the training of a large language model of expert agents based on the expert dataset comprises: the training data distribution is iteratively optimized by adopting a gradient-based method, the quantity of the field specific data is dynamically adjusted, and the self-adaptive adjustment is carried out according to the influence of the field specific data on the performance of the downstream task.
8. The iterative optimization method of claim 5, wherein the comprehensive evaluation index system comprises objective performance indexes and subjective experience indexes, wherein the objective indexes comprise any one or more basic evaluation indexes of accuracy, recall, F1 value and AUC, and any one or more professional evaluation indexes of sensitivity, specificity, response time, interpretability, generalization capability, adaptability and robustness, and the subjective indexes are feedback information collected according to user satisfaction investigation or expert evaluation modes.
9. A computer readable storage medium having stored thereon a computer program, which when executed by a processor implements the method of any of claims 5-8.
10. An electronic terminal comprising a memory, a processor and a computer program stored on the memory, characterized in that the processor executes the computer program to implement the method of any of claims 5-8.

Description

Expert agent-oriented iterative optimization system and method Technical Field The application belongs to the technical field of large language models of intelligent agents, and relates to an iterative optimization system and method for an expert intelligent agent, a medium and an electronic terminal. Background In recent years, the application of artificial intelligence technology in the field of medical health is increasingly wide, and especially expert intelligent technology is becoming an important force for promoting the improvement of medical service quality, but the prior art still has a plurality of systematic defects, and the application effect of the artificial intelligent technology in complex medical scenes is severely restricted. However, the existing expert system often has the following disadvantages: 1) Traditional expert systems rely on explicit rule expressions, i.e., manually written by domain experts "if, then" form of generative rules, which, while performing well on certain structured problems, suffer from inherent drawbacks of difficult knowledge acquisition, difficulty in handling uncertainty information, poor adaptability, etc. In the medical health field, the clinical experience of experts often has high degree of situational dependency and implicit features, and is difficult to translate completely into explicit rules. 2) Conventional expert systems typically use a predefined set of rules to simulate the expert's decision making process based on rules and logical reasoning, the knowledge of which is typically encoded in the form of rules, decision trees, or expert experience, with relatively low adaptability, and with little ease to handle complex or uncertain situations. Existing systems are often customized to solve a single problem, with knowledge bases and inference engines tightly coupled, making them difficult to modify or apply to other areas, and once the problem boundaries are slightly expanded, their solution capability drops dramatically. 3) The knowledge base of conventional expert systems is usually static, is manually updated by an expert, is difficult to cope with new situations or unknown problems, and is simple to process data, and usually depends on priori knowledge and rules. Typically run in a fixed environment or scene, with poor adaptability and limited expansion capability for changing environments or new fields. In practical applications, the complexity and dynamics of medical scenarios require that the system have the ability to feed back in real time and optimize continuously, but the prior art has significant drawbacks in this respect. Disclosure of Invention The application provides an expert agent-oriented iterative optimization system and method, a medium and an electronic terminal, which are used for solving the problems of serious bottleneck, lack of systematic closed-loop optimization mechanism and lack of real-time feedback mechanism of the existing expert system knowledge acquisition. In a first aspect, the application provides an iterative optimization system for an expert agent, which comprises an expert data collection and labeling module, an expert agent development module, an ideal state feedback module, an effect evaluation module and a data production optimization module, wherein the expert data collection and labeling module is used for collecting expert data and carrying out layering labeling on the expert data to form an expert data set, the expert agent development module is used for training a large language model of the expert agent based on the expert data set, the ideal state feedback module is used for establishing a bidirectional interaction channel between the expert agent and the expert to enable the expert agent to provide expert comments for output content of the expert agent, the effect evaluation module is used for establishing a comprehensive evaluation index system to evaluate the operation effect of the expert agent to obtain an evaluation result, and the data production optimization module is used for carrying out optimization adjustment on the expert data set based on the evaluation result and the expert comments to enable the expert agent development module to carry out iterative optimization training on the large language model. In the application, the iterative optimization system collects expert data through the expert data collection and labeling module, integrates the expert knowledge, clinical experience and best practice of the expert in the field to construct a high-quality training data set, and trains a large language model of the expert agent based on the high-quality training data set through the expert agent development module. Further, expert opinion is provided for the output content of the expert intelligent body based on the expert through the ideal state feedback module, a comprehensive evaluation index system is established through the effect evaluation module, and an evaluation result is obtained f