CN-121998080-A - Large model answer generation method and system based on factual reasoning

CN121998080ACN 121998080 ACN121998080 ACN 121998080ACN-121998080-A

Abstract

The invention relates to a large model answer generation method and system based on factual reasoning. The method comprises the steps of after an answer draft is generated, conducting a trial process, firstly submitting a question to the answer draft, conducting fact checking and providing evidence based on the question, finally conducting a trial to the answer draft based on the evidence and the question to obtain a trial result, taking the current answer draft as a final draft if the trial result comprises an acceptance instruction, modifying the current answer draft and re-executing the trial process if the trial result comprises a rejection instruction, re-generating the answer draft and conducting the trial process if the trial result comprises a re-planning instruction, and finally optimizing the final draft to obtain and output an answer text. Compared with the prior art, the method has the advantages of remarkably improving the fact accuracy and logic strictness of output content and the like.

Inventors

WANG BINBIN
CHENG DAWEI
GUO NAIWANG
SHEN QUANJIANG
WU YI
YAO YINCHENG
ZHANG LEI
LIU CHANG
ZHENG CHENG
DENG BOWEN

Assignees

国网上海市电力公司
同济大学

Dates

Publication Date: 20260508
Application Date: 20251223

Claims (10)

1. A large model answer generation method based on factual reasoning is characterized by comprising the following steps: S1, carrying out intrinsic complexity analysis on an input problem, and classifying the problem into a simple problem or a complex problem according to an analysis result; S2, directly generating an answer draft aiming at the input problem if the current problem is a simple problem and skipping to execute the step S3; S3, a question is put forward on the answer draft, fact checking is carried out based on the question, evidence is provided, finally, the answer draft is judged based on the evidence and the question, a judging result is obtained, if the judging result comprises an acceptance instruction, the current answer draft is used as a final draft, and a step S4 is executed; And S4, optimizing the final draft to obtain and output an answer text.
2. The method for generating large model answers based on factual reasoning according to claim 1, wherein the specific process of performing the intrinsic complexity analysis on the inputted questions in S1 comprises: presetting an inference hop count threshold; Calling a large language model instance, utilizing a preset reasoning hop number assessment instruction to disassemble a problem into reasoning steps, and counting the reasoning hop number required by the problem; If the number of the reasoning hops required by the problem is smaller than or equal to a preset reasoning hops threshold, classifying the problem as a simple problem, and otherwise classifying the problem as a complex problem.
3. The method for generating large model answers based on factual reasoning according to claim 1, wherein the specific process of multi-view planning, generating and synthesizing to obtain answer draft in S2 comprises: aiming at the input problems, combining domain knowledge and problem analysis dimension to make a structured solution outline; According to the solution outline, a plurality of large language model examples are called in parallel, each example receives the same solution outline and is configured with different view angle instructions, so that a plurality of draft can be generated from multiple views; receiving all generated drafts, identifying through text semantic analysis and integrating all drafts into a consensus draft serving as an answer draft according to consensus content, difference points and unique information among drafts; The multi-view comprises a neutral view, an critical view of the emphasis risk and a quantized view of the emphasis data.
4. The large model answer generation method based on factual reasoning according to claim 1, wherein when the answer draft is challenged in S3, the challenged challenge includes a fact type challenge, a logic type challenge, a causal type challenge, and an countermeasure type challenge, and the challenged challenge is marked with priority.
5. The method for generating a large model answer based on factual reasoning according to claim 4, wherein when the fact is checked and evidence is provided based on the challenge in the step S3, specifically, the large language model itself or an external API is called to perform the fact check, and an evidence report containing a check result is generated, where the check result includes an authenticity binary decision of the fact claim, a confidence quantization value of the authenticity binary decision, source information of the arguments, and a verification conclusion of logical relevance.
6. The method for generating large model answers based on factual reasoning according to claim 5, wherein the result of the step S3 includes a result instruction and a decision reason, the result instruction includes an acceptance instruction, a rejection instruction or a reprofiling instruction, the decision reason adopts a four-segment proof structure of question location, evidence reference, standard comparison and conclusion deduction, and when the result instruction is a rejection, a revised draft is synchronously generated based on the current decision reason and is output together with the result of the decision.
7. The method for generating large model answers based on factual reasoning of claim 1, wherein in the step S3, a mechanism for counting the number of rounds of examination and judging a threshold value is introduced, after the result of examination is obtained, whether the number of rounds of examination is larger than the preset maximum number of rounds of examination is judged, if yes, the current answer draft is used as the final draft, and step S4 is executed, otherwise, the process in the step S3 is continuously executed, the specific content in the result of examination is judged, the number of rounds of examination is one initially, and the number of rounds of examination is triggered to be increased only when the result of examination is a refusal instruction or a rescheduling instruction.
8. A large model answer generation system based on factual reasoning, characterized in that the system works by applying a large model answer generation method based on factual reasoning according to any one of claims 1-7, and the system comprises a router module, a draft generation module, a interrogation module and a stylist module; The router module analyzes the inherent complexity of the input problem, and generates problem type information according to an analysis result, namely classifying the problem into a simple problem or a complex problem; The system comprises a draft generation module, a review module, a router module, a answer draft generation module, a query module and a query module, wherein the input end of the draft generation module is connected with the router module and the review module, and the output end of the draft generation module is connected with the review module; the input end of the examination module is connected with the draft generation module, the output end of the examination module is connected with the draft generation module and the stylist module, and the examination module is used for completing the cyclic examination process, outputting the examination module to the draft generation module for carrying out draft modification or regeneration if a certain draft fails to pass the examination, and outputting the examination module to the stylist module as a final draft if the certain draft passes the examination; And the stylist module is used for optimizing the final draft to obtain and output answer texts.
9. The large model answer generation system based on factual reasoning of claim 8, wherein the draft generation module comprises a first sub-module and a second sub-module, the first sub-module and the second sub-module are of a logic parallel architecture, the first sub-module is started according to the type of the question, the first sub-module is started to generate an answer draft when the question is a complex question, the second sub-module is started to generate the answer draft when the question is a simple question, the first sub-module comprises a planner unit, a multiple proposer unit and a synthesizer unit which are connected in sequence, and the second sub-module comprises a draft generation unit; The planner unit builds a solution outline aiming at the input problem; The multiple proposer unit comprises a plurality of large language model examples, and calls the large language model examples in parallel according to the solution outline, wherein each example receives the same solution outline and is configured with different view angle instructions to generate a plurality of drafts with different view angles; The synthesizer unit integrates all the drafts generated by the multiple proposer units to generate a consensus draft as an answer draft; The draft generation unit directly generates an answer draft for the inputted question.
10. The large model answer generation system based on factual reasoning of claim 8 wherein the questioning module comprises a suspected person unit, a researcher unit and a judge unit which are sequentially interacted, and the output end of the judge unit is in feedback connection with the draft generation module, wherein the questioning information output by the suspected person unit is transmitted to the researcher unit, the evidence report generated by the researcher unit is transmitted to the judge unit together with the questioning information of the suspected person unit, and the judge unit outputs a judging result based on the questioning information of the suspected person unit and the evidence report of the researcher unit.

Description

Large model answer generation method and system based on factual reasoning Technical Field The invention relates to the technical field of large models, in particular to a large model answer generation method and system based on factual reasoning. Background In recent years, artificial intelligence technology represented by a large language model (Large Language Model, LLM) has been rapidly developed and has demonstrated excellent capabilities in the fields of natural language processing, knowledge questions and answers, content generation, and the like. The models learn rich language rules and world knowledge by pre-training on massive text data. However, how to ensure that large language models output the accuracy, reliability and rationality of the reasoning process of content when processing queries that require precise facts and strict logic has become a key challenge in this field. The detection and promotion of the modeling factual reasoning capability is a core technical bottleneck for promoting the application of the modeling factual reasoning capability in serious scenes such as finance, law, scientific research and the like. At present, two main stream technical schemes exist for the detection and application of the factual reasoning capability of a large language model. The first is a direct generation scheme (Direct Generation), which is also the most basic application mode, the system inputs user questions directly into a large language model, the model directly generates answers through one forward calculation by means of parameterized knowledge in the model, and the whole process does not involve external information retrieval or self correction. The second is a search enhancement generation scheme (RETRIEVAL-Augmented Generation, RAG) which, as an improvement over the former, will first search relevant information from an external knowledge base based on user questions before generating answers, and then input these information as additional context into the model along with the original questions, thus guiding the model to generate answers that are richer in content and more realistic. As in the patent of the invention with publication number CN117786091a, a self-inspiring intelligent question-answering implementation method and system based on the scotch-bottom question is disclosed, which is capable of automatically asking and answering the question answer, but as in the search enhancement generation scheme, there are a lot of fact errors which are not self-perceived due to misunderstanding of the context or inherent bias, and the answer is not strict enough. However, both of the above prior art solutions have significant drawbacks in practical use, mainly arising from their "one-time generation" paradigm. First, they generally lack procedural self-review and correction mechanisms, and models cannot actively, iteratively review and correct their own intermediate thoughts and final conclusions in a single generation flow, which makes the model even with external knowledge References (RAGs) possible to generate self-imperceptible fact errors, i.e. "model illusions", due to misunderstanding of context or inherent bias. Secondly, because the reasoning process of the model is like a black box, when an error is input, the prior art is difficult to reveal a reasoning chain in the model, and a specific link where the error occurs cannot be positioned, so that the detection capability can only stay on the surface judgment of a final result, and a specific short plate of the model on multi-step reasoning or causal analysis cannot be deeply exposed. Finally, for complex open questions requiring comprehensive analysis or multi-step deduction, the one-time generation mode is difficult to construct a structured thinking framework, often resulting in answer one-sided, logic jumping or over-simplification, and cannot exhibit strict and comprehensive analysis capability. In summary, the prior art has the remarkable defects of lack of procedural self-correction capability, easiness in generating model illusion, difficulty in detecting, opaque reasoning process, difficulty in positioning defects, insufficient analysis capability on complex problems, difficulty in constructing a structured thinking framework, answer one-sided, logic jump and the like, and lower fact accuracy and logic rigor of output content of the model. Disclosure of Invention The invention aims to overcome the defects of the prior art and provide a large model answer generation method and system based on factual reasoning. The aim of the invention can be achieved by the following technical scheme: According to one aspect of the present invention, there is provided a large model answer generation method based on factual reasoning, the method comprising the steps of: S1, carrying out intrinsic complexity analysis on an input problem, and classifying the problem into a simple problem or a complex problem according to an analysis result; S2,