Search

CN-121979801-A - Probability-fused large language model test case generation method and system

CN121979801ACN 121979801 ACN121979801 ACN 121979801ACN-121979801-A

Abstract

The invention relates to the field of test case generation, and provides a method and a system for generating a large language model test case with fusion probability, wherein the method comprises the steps of obtaining candidate probability and candidate probability parameters of training assertion, and constructing training assertion feature vectors through the candidate probability parameters and uncertainty marks; the method comprises the steps of obtaining a assertion correctness parameter, respectively carrying out multistage correctness probability prediction on a training assertion feature vector by a plurality of reference models to obtain comprehensive prediction probability, obtaining a judging model according to the assertion correctness parameter and the comprehensive prediction probability, generating initial assertion according to assertion generation constraint, calculating pattern coverage number and shannon entropy of the initial assertion, judging coverage of the initial assertion to obtain high coverage assertion, calculating candidate probability parameters and uncertainty marks of the high coverage assertion, constructing an assertion feature vector, inputting the assertion feature vector into the judging model to obtain a judging label, judging and rewriting to obtain target assertion, and generating a test case.

Inventors

  • HOU CHUNYAN
  • GE CHENGFENG

Assignees

  • 天津理工大学

Dates

Publication Date
20260505
Application Date
20260403

Claims (10)

  1. 1. The method for generating the test case of the large language model with the fusion probability is characterized by comprising the following steps of: s1, generating training assertion, obtaining candidate probability of the training assertion, calculating candidate probability parameters according to the candidate probability, adding uncertainty marks, and constructing training assertion feature vectors through the candidate probability parameters and the uncertainty marks; S2, acquiring a assertion correctness parameter, selecting a plurality of reference models, respectively carrying out multi-level correctness probability prediction on the training assertion feature vector by the plurality of reference models to obtain comprehensive prediction probability, obtaining a judgment threshold according to the assertion correctness parameter and the comprehensive prediction probability, and obtaining a judgment model based on the judgment threshold; s3, constructing an assertion generation constraint, generating an initial assertion according to the assertion generation constraint, calculating a pattern coverage number and shannon entropy of the initial assertion, judging the coverage of the initial assertion according to the pattern coverage number and the shannon entropy, and obtaining a high coverage assertion based on the coverage; S4, calculating candidate probability parameters and uncertainty marks of the high coverage assertion, constructing assertion feature vectors according to the candidate probability parameters and uncertainty marks of the high coverage assertion, and inputting the assertion feature vectors into a judgment model to obtain a judgment label; and S5, judging and rewriting the high coverage assertion according to the judgment tag to obtain a target assertion, and taking the target assertion as a test case.
  2. 2. The method for generating the test case of the fusion probability large language model according to claim 1, wherein in step S1, candidate probability of each word block in the training assertion is obtained, first output probability and second output probability are obtained according to the candidate probability, uncertainty measure is calculated based on the first output probability and the second output probability, probability mean is calculated through the candidate probability, difference mean and minimum probability are calculated according to the uncertainty measure, and the minimum probability, probability mean and difference mean are used as the candidate probability parameters.
  3. 3. The method for generating a test case of a fusion probability large language model according to claim 2, wherein in step S1, it is determined whether a potential candidate word corresponding to a minimum probability appears in the training assertion, and when the potential candidate word corresponding to the minimum probability appears in the training assertion, the uncertainty flag takes a value of 1, otherwise, it takes a value of 0.
  4. 4. The method for generating the test case of the fusion probability large language model according to claim 1, wherein in step S2, the correctness parameters of the assertion are obtained according to the correctness of the training assertion, a plurality of models are selected as the reference models, the training assertion is respectively subjected to K-fold cross validation according to training assertion feature vectors by the plurality of reference models to obtain a probability stacking matrix, a second-level discrimination model is selected, and the correct probability prediction is performed by the second-level discrimination model according to the probability stacking matrix to obtain the comprehensive prediction probability.
  5. 5. The method for generating a test case for a fusion probability large language model according to claim 1, wherein in step S2, a judgment threshold interval is determined, and the comprehensive prediction probability and the assertion correctness parameter are used to traverse in the judgment threshold interval, so as to obtain the judgment threshold.
  6. 6. The method for generating the test case of the fusion probability large language model according to claim 1, wherein in step S3, a function type is acquired, a parameter type is determined, the assertion generation constraint is obtained through the function type and the parameter type, the initial assertion is generated according to the assertion generation constraint, and the initial assertion is aggregated and mapped according to the statement position of the initial assertion to obtain a pattern sequence.
  7. 7. The method for generating a test case of a fusion probability large language model according to claim 6, wherein in step S3, the sequence statistics is performed on the pattern sequence to obtain the pattern coverage number, shannon entropy of the pattern sequence is calculated, coverage of initial assertion is judged according to the pattern coverage number and shannon entropy, a coverage threshold is determined, when the coverage is lower than the coverage threshold, assertion overwriting is performed on the initial assertion to obtain the high coverage assertion, otherwise, the initial assertion is used as the high coverage assertion.
  8. 8. The method for generating the test case of the fusion probability large language model according to claim 1, wherein in step S4, a probability average value, a minimum probability and a difference average value of the high coverage assertion are calculated, and the probability average value, the minimum probability and the difference average value of the high coverage assertion are used as candidate probability parameters of the high coverage assertion, so as to obtain an uncertainty flag of the high coverage assertion.
  9. 9. The method for generating a test case of a fusion probability large language model according to claim 1, wherein in step S5, when the judgment tag displays that the high coverage assertion is available, the high coverage assertion is directly used as the target assertion; When the judgment tag shows that the high coverage assertion is not available, maintaining a calling structure and input parameters of the high coverage assertion unchanged, rewriting the high coverage assertion to obtain a rewritten judgment tag of the rewritten assertion, when the rewritten judgment tag shows that the rewritten assertion is available, taking the rewritten assertion as the target assertion, otherwise, regenerating the high coverage assertion, and taking the high coverage assertion of which the judgment tag shows that the high coverage assertion is available as the target assertion.
  10. 10. A fused probability large language model test case generation system for executing the fused probability large language model test case generation method according to any one of claims 1 to 9, comprising: The assertion feature vector module is used for generating a training assertion to obtain candidate probability of the training assertion, calculating candidate probability parameters according to the candidate probability, adding an uncertainty mark, and constructing a training assertion feature vector through the candidate probability parameters and the uncertainty mark; The judgment model module is used for acquiring the assertion correctness parameters, selecting a plurality of reference models, respectively carrying out multi-level correctness probability prediction on the training assertion feature vectors by the plurality of reference models to obtain comprehensive prediction probability, obtaining a judgment threshold according to the assertion correctness parameters and the comprehensive prediction probability, and obtaining a judgment model based on the judgment threshold; The assertion coverage module is used for constructing assertion generation constraint, generating initial assertion according to the assertion generation constraint, calculating pattern coverage number and shannon entropy of the initial assertion, judging coverage of the initial assertion according to the pattern coverage number and the shannon entropy, and obtaining high coverage assertion based on the coverage; the judging tag module is used for calculating the candidate probability parameter and the uncertainty flag of the high coverage assertion, constructing an assertion feature vector according to the candidate probability parameter and the uncertainty flag of the high coverage assertion, and inputting the assertion feature vector into the judging model to obtain a judging tag; And the test case module is used for judging and rewriting the high coverage assertion according to the judgment tag to obtain a target assertion, and taking the target assertion as a test case.

Description

Probability-fused large language model test case generation method and system Technical Field The invention relates to the technical field of test case generation, in particular to a method and a system for generating a large language model test case with fusion probability. Background In the software testing process, the test case is used for verifying whether the behaviors of the program under different input conditions meet expectations or not, and the input coverage range and the result rationality of the test case directly influence the testing effect. Along with the scale expansion of a software system and the improvement of the complexity of function logic, the functions often have the characteristics of multiple parameters, complex branch structures, hidden boundary behaviors and the like, test cases are designed simply by relying on manual experience, program behaviors under different input conditions are difficult to systematically cover, and the test cost and the maintenance difficulty are continuously increased. In actual software testing practice, test cases generally describe program behavior in the form of assertions, that is, by giving function inputs and judging output results, so as to verify whether execution results of a program under specific input conditions meet expectations. Thus, assertions can be viewed as a common representation of test cases, whose quality directly affects the validity and reliability of test results. In practical application, the test case generation mode based on the large model makes the test case construction not completely depend on fixed rules or manual design any more, and provides new possibility for complex function test. However, since the generated results are typically directly involved in the test in the form of assertions, their input coverage and the rationality of the assertions' results have a direct impact on the test results. Therefore, when the method is applied to a real software test scene, how to effectively control the input diversity and correctness of the generated assertion while ensuring the generation efficiency is still a problem which needs to be further researched and solved. Disclosure of Invention The present invention is directed to solving at least one of the technical problems existing in the related art. Therefore, the invention provides a method and a system for generating the test case of the large language model with fusion probability, which are realized. The invention provides a method for generating a large language model test case with fusion probability, which comprises the following steps: s1, generating training assertion, obtaining candidate probability of the training assertion, calculating candidate probability parameters according to the candidate probability, adding uncertainty marks, and constructing training assertion feature vectors through the candidate probability parameters and the uncertainty marks; S2, acquiring a assertion correctness parameter, selecting a plurality of reference models, respectively carrying out multi-level correctness probability prediction on the training assertion feature vector by the plurality of reference models to obtain comprehensive prediction probability, obtaining a judgment threshold according to the assertion correctness parameter and the comprehensive prediction probability, and obtaining a judgment model based on the judgment threshold; s3, constructing an assertion generation constraint, generating an initial assertion according to the assertion generation constraint, calculating a pattern coverage number and shannon entropy of the initial assertion, judging the coverage of the initial assertion according to the pattern coverage number and the shannon entropy, and obtaining a high coverage assertion based on the coverage; S4, calculating candidate probability parameters and uncertainty marks of the high coverage assertion, constructing assertion feature vectors according to the candidate probability parameters and uncertainty marks of the high coverage assertion, and inputting the assertion feature vectors into a judgment model to obtain a judgment label; and S5, judging and rewriting the high coverage assertion according to the judgment tag to obtain a target assertion, and taking the target assertion as a test case. According to the method for generating the test case of the fusion probability large language model, in the step S1, the candidate probability of each word block in the training assertion is obtained, the first output probability and the second output probability are obtained according to the candidate probability, the uncertainty measure is calculated based on the first output probability and the second output probability, the probability mean is calculated through the candidate probability, the difference mean and the minimum probability are calculated according to the uncertainty measure, and the minimum probability, the probability mean and the differ