Search

CN-121979797-A - Intelligent optimization verification method and system for AI test cases

CN121979797ACN 121979797 ACN121979797 ACN 121979797ACN-121979797-A

Abstract

The invention discloses an intelligent optimization verification method and system for an AI test case, and the method specifically comprises the steps of analyzing an implicit dependency relationship by adopting a causal discovery algorithm aiming at an entity association structure in a test case knowledge graph, identifying a redundant test case and an invalid test path to obtain a causal dependency network, traversing an uncovered entity relationship path in the test case knowledge graph based on the causal dependency network to generate a supplementary test case suggestion, dynamically adjusting analysis strategy parameters according to historical optimization data through an adaptive engine to obtain an optimized test case set, generating an input parameter variation sample by utilizing a fuzzy test engine based on the optimized test case set, and expanding a test boundary. The invention can obviously improve the efficiency of the black box test, promote the test coverage rate and accuracy, lighten the work load of a test engineer, improve the user experience and provide an efficient and intelligent solution for the field of software test.

Inventors

  • LIU LIANG

Assignees

  • 广州极尚网络技术有限公司

Dates

Publication Date
20260505
Application Date
20260204

Claims (10)

  1. 1. An intelligent optimization and verification method for an AI test case is characterized by comprising the following steps: Importing the test case set through a multi-format parser, and converting the test case set into structural data by using a natural language processing technology; based on the structured data, establishing a logic dependency relationship among the test steps through entity relationship extraction, and constructing a test case knowledge graph; Aiming at an entity association structure in a test case knowledge graph, analyzing an implicit dependency relationship by adopting a causal discovery algorithm, and identifying a redundant test case and an invalid test path to obtain a causal dependency network; traversing an uncovered entity relationship path in the test case knowledge graph by utilizing a graph reasoning mechanism based on the causal dependency network to generate a supplementary test case suggestion; Based on a causal dependency network and the supplementary test case suggestion, dynamically adjusting analysis strategy parameters according to historical optimization data by a self-adaptive engine integrated with a reinforcement learning model to obtain an optimized test case set; Generating an input parameter variation sample by using a fuzzy test engine based on the optimized test case set, and expanding a test boundary; And transmitting the data in the optimization process to a verification interface, acquiring a manual verification feedback result of the verification interface, and refluxing the result to the self-adaptive engine to perform model parameter optimization.
  2. 2. The method according to claim 1, wherein the importing the test case set by the multi-format parser converts the test case set into the structured data by using a natural language processing technology, and specifically includes: importing test case sets through a multi-format parser, identifying test case files in different formats by adopting a double mechanism of format detection and adapter scheduling, and uniformly converting the files in each format into an intermediate representation form; for natural language description in the intermediate representation form, adopting a transducer-based encoder and a semantic role labeling model to jointly extract test logic elements; Based on the test logic elements, according to a preset conversion rule base, semantic description, parameter entity and assertion conditions in natural language description are respectively converted into an operation sequence, typed parameters and verification expressions, so as to obtain structured data.
  3. 3. The method of claim 1, wherein the building a test case knowledge graph based on the structured data by establishing logical dependencies between test steps through entity relationship extraction specifically comprises: Based on the structured data, identifying key elements and association relations thereof in the testing step by adopting a multi-stage neural network model extraction process; based on key elements and association relations thereof, establishing a test suite, a test case and a test step in a layered manner by adopting an incremental construction strategy through a graph construction engine to form an attribute graph structure; aiming at the constructed attribute graph structure, the dependency path, the dependency strength and the implicit dependency among the testing steps are deeply analyzed by a logic dependency analysis engine to obtain a dependency relationship network; based on the dependency relationship network, verifying and complementing the entity relationship by a knowledge fusion mechanism and combining the structural evidence, the semantic evidence and the statistical evidence to obtain a test case knowledge graph.
  4. 4. The method of claim 1, wherein the analyzing the implicit dependency relationship by using a causal discovery algorithm for the entity association structure in the test case knowledge graph, and identifying the redundant test case and the invalid test path, and obtaining the causal dependency network specifically comprises: Aiming at an entity association structure in a test case knowledge graph, an initial causal graph is constructed through a conditional independence test and greedy equivalent search by adopting a Peter-Clark algorithm-based constraint learning method and a greedy equivalent search algorithm-based score learning method; Based on the initial causal graph, excluding the pseudo causal relationship by a correction framework comprising time aliasing correction, selection aliasing correction and aliasing correction, obtaining a first causal dependency network; Aiming at the first causal dependency network, analyzing the redundancy relation among test cases through a redundancy identification framework comprising structural redundancy identification, functional redundancy identification and statistical redundancy identification to obtain a second causal dependency network; And aiming at the second causal dependency network, identifying an invalid test path through a multi-level invalid path detection framework comprising logic invalidity detection, effect invalidity detection and practice invalidity detection, and obtaining the target causal dependency network.
  5. 5. The method of claim 4, wherein the constructing an initial causal graph by a conditional independence test and greedy equivalence search by adopting a Peter-Clark algorithm constraint learning method and a greedy equivalence search algorithm-based score learning method for the entity association structure in the test case knowledge graph specifically comprises: Based on an entity association structure in the test case knowledge graph, taking each test step entity as a causal node, and taking a relation edge between the entities as an initial causal edge candidate to form a basic graph model; Constructing a three-level condition independence test framework by adopting a Peter-Clark algorithm constraint learning method, wherein a first layer of the condition independence test framework is used for preliminarily screening direct causal relationships by adopting partial correlation-based local independence test, a second layer of the condition independence test framework is used for identifying indirect causal relationships by adopting global independence test based on condition mutual information, and a third layer of the condition independence test framework is used for comprehensively determining causal directions by adopting a Bayesian network-based causal structure learning algorithm; gradually deleting irrelevant edges and determining the direction of the edges of the basic graph model through a condition independence test framework to obtain a primary causal structure; Based on the primary causal structure, a multidimensional scoring function system is designed, a two-stage greedy search strategy is implemented, and an optimal causal structure is found in a Markov equivalent class space, so that an initial causal graph is obtained.
  6. 6. The method of claim 1, wherein the generating the supplemental test case suggestion based on the causal dependency network by traversing an uncovered entity relationship path in the test case knowledge graph using a graph inference mechanism comprises: based on a test logic structure in a causal dependency network, evaluating the coverage condition of the existing test case through a path coverage analysis model according to coverage indexes, and obtaining a coverage analysis result, wherein the coverage indexes comprise node coverage, edge coverage and path coverage; based on the coverage analysis result, adopting a bidirectional breadth first search algorithm to search paths from key cause nodes and key verification nodes of the test case knowledge graph simultaneously, and filtering invalid paths by using a path feasibility evaluation model to obtain uncovered entity relationship paths in the test case knowledge graph; Based on the uncovered entity relationship paths, carrying out priority ranking according to the causal importance dimension, the business importance dimension, the risk importance dimension and the innovation importance dimension, and generating an importance comprehensive score of each path; and sequencing the uncovered entity relationship paths according to the importance comprehensive scores, and generating specific test case suggestions through a graph reasoning mechanism.
  7. 7. The method according to claim 1, wherein the obtaining, based on the causal dependency network and the supplemental test case suggestion, an optimized test case set by dynamically adjusting analysis policy parameters according to historical optimization data through an adaptive engine of an integrated reinforcement learning model, specifically comprises: Constructing a state space based on structural features of a causal dependency network and quality evaluation results of the supplement test case suggestions, wherein the state space comprises a network topology state, a coverage state, a quality state and a resource state; Designing an action space and a reward function based on the parameter adjustment requirement of test case optimization, wherein the action space is used for defining redundancy identification, causal analysis, path discovery and generating an optimized adjustable parameter set, and the reward function comprises coverage reward, efficiency reward, quality reward and stability reward; constructing an adaptive engine of the integrated reinforcement learning model based on the state space, the action space and the reward function; Based on the self-adaptive engine, historical optimization data is input, test case sequence features and causal network structure features are processed through a feature extraction network, and network output parameter adjustment suggestions are generated by adopting a strategy to obtain an optimization test case set.
  8. 8. The method according to claim 1, wherein the generating the input parameter variation sample by using the fuzzy test engine based on the optimized test case set expands the test boundary specifically comprises: based on input parameter characteristics in the optimized test case set, identifying a parameter structure and a constraint relation through a multi-level parameter analysis framework to obtain a parameter analysis result, wherein the multi-level parameter analysis framework comprises a grammar analysis layer, a semantic understanding layer and a context correlation layer; According to the parameter analysis result, constructing a hierarchical operation set comprising a basic variation layer, a semantic variation layer and a combined variation layer through a variation strategy library, and distributing variation priorities for different parameters and strategies based on risk prediction through a priority model to obtain a targeted variation scheme; performing pre-filtering selection, mutation execution operation, post-verification inspection and sample optimization processing on a targeted mutation scheme, and dynamically adjusting mutation parameters by integrating a self-adaptive mutation control mechanism to obtain a parameter mutation sample set; Based on the parameter variation sample set, running a fuzzy test by executing a monitoring frame and capturing abnormal behaviors to obtain a fuzzy test execution result; based on the fuzzy test execution result, the test boundary is expanded through a boundary discovery and evaluation mechanism.
  9. 9. The method according to claim 8, wherein the expanding the test boundary by the boundary discovery and evaluation mechanism based on the fuzzy test execution result specifically comprises: Based on abnormal data in a fuzzy test execution result, establishing direct association between abnormality and specific input parameter variation through basic abnormality classification, noise filtering and abnormality root cause analysis, and determining an effective boundary scene; based on the effective boundary scene, a mode discovery method based on a clustering algorithm is adopted to aggregate similar scenes into mode categories, and a boundary mode knowledge base is constructed; Based on mode knowledge in a boundary mode knowledge base, generating a derived test case by parameter type generalization, operation combination generalization and scene context generalization, and exploring a complex boundary scene of multi-dimensional combination; Based on the derived test cases, evaluating boundary expansion effects according to coverage expansion indexes, wherein the coverage expansion indexes comprise space coverage expansion, state space coverage expansion, code path coverage expansion and combined scene coverage expansion; based on the evaluation result of the boundary expansion effect, generating a boundary expansion scheme through priority ordering and resource allocation decision, and executing test boundary expansion according to the boundary expansion scheme.
  10. 10. An AI test case intelligent optimization verification system, which is characterized by comprising: The data conversion module is used for importing a test case set through the multi-format analyzer and converting the test case set into structured data by utilizing a natural language processing technology; the knowledge graph module is used for establishing a logic dependency relationship among the testing steps through entity relationship extraction based on the structured data, and constructing a test case knowledge graph; The causal analysis module is used for analyzing the implicit dependency relationship by adopting a causal discovery algorithm aiming at the entity association structure in the test case knowledge graph, identifying the redundant test case and the invalid test path and obtaining a causal dependency network; The suggestion supplementing module is used for traversing an uncovered entity relationship path in the test case knowledge graph by utilizing a graph reasoning mechanism based on the causal dependency network to generate a supplementation test case suggestion; the case optimizing module is used for dynamically adjusting analysis strategy parameters according to historical optimization data through an adaptive engine integrated with a reinforcement learning model based on a causal dependency network and supplementing test case suggestions to obtain an optimized test case set; the boundary expansion module is used for generating an input parameter variation sample by using the fuzzy test engine based on the optimized test case set and expanding a test boundary; And the verification feedback module is used for transmitting the data in the optimization process to the verification interface, acquiring a manual verification feedback result of the verification interface and refluxing the result to the self-adaptive engine to perform model parameter optimization.

Description

Intelligent optimization verification method and system for AI test cases Technical Field The invention relates to the technical field of artificial intelligence, in particular to an AI test case intelligent optimization verification method and system. Background In the field of software testing, black box testing is a commonly used testing method, and is widely applied to various stages of software development by virtue of the remarkable advantage that the internal structure of software is not required to be known. The method mainly tests the external behaviors of the software to ensure that the functions of the software accord with expected settings, and plays an important role in guaranteeing the quality of the software. However, with the rapid development of the software industry, the software is increasingly large in scale and the complexity is also drastically increased. This variation presents numerous challenges for black box testing, principally in the following respects: 1. The efficiency problem is that the number of the black box test cases presents explosive growth situations in the current test environment. In the face of such a huge number of test cases, the traditional manual screening and optimizing method not only needs to consume a great deal of time and labor cost, but also is extremely prone to human errors. For example, during manual screening, a test engineer may inadvertently miss some critical test cases or incorrectly preserve redundant test cases, which undoubtedly severely impacts the overall efficiency of the test job. 2. The method has the advantages that a large number of repeated test cases are enriched in the test case set, and the repeated test cases cannot bring additional value to test work, but waste valuable test resources. Meanwhile, unreasonable test case designs, such as improper test path selection, inaccurate assertion condition setting, and the like, can directly lead to the reduction of test coverage rate and accuracy. For example, certain critical business logic may not be fully covered due to unreasonable design of test cases, so that potential defects in the software cannot be found in time, and hidden danger is brought to stable operation of the software. 3. User experience problems test engineers need to put a lot of time and effort into the management and maintenance of test cases in daily work. From the writing, sorting, updating of test cases to storage, each link requires careful handling, which makes it difficult for them to focus more effort on the core work of the test itself, such as in-depth analysis of software functions, exploration of potential defects, etc. Long time, not only the work enthusiasm and creativity of test engineers can be affected, but also the improvement of the whole test work quality is not facilitated. In view of the above problems, there are some related test optimization methods, but most of them have limitations. For example, some methods only focus on simple classification management of test cases, lack in deep analysis and mining of logical relationships between test cases, and cannot effectively identify redundant test cases and invalid test paths, and some methods lack a scientific and reasonable evaluation mechanism when generating supplementary test cases, so that quality of the generated test cases is uneven, and actual test requirements are difficult to meet. Disclosure of Invention The invention aims to provide an intelligent optimization and verification method and system for AI test cases, which can remarkably improve the efficiency of black box test, improve the test coverage rate and accuracy, simultaneously lighten the workload of test engineers, improve the user experience, and provide an efficient and intelligent solution for the field of software test so as to solve at least one of the problems in the prior art. In a first aspect, the present invention provides an intelligent optimization verification method for an AI test case, where the method specifically includes: Importing the test case set through a multi-format parser, and converting the test case set into structural data by using a natural language processing technology; based on the structured data, establishing a logic dependency relationship among the test steps through entity relationship extraction, and constructing a test case knowledge graph; Aiming at an entity association structure in a test case knowledge graph, analyzing an implicit dependency relationship by adopting a causal discovery algorithm, and identifying a redundant test case and an invalid test path to obtain a causal dependency network; traversing an uncovered entity relationship path in the test case knowledge graph by utilizing a graph reasoning mechanism based on the causal dependency network to generate a supplementary test case suggestion; Based on a causal dependency network and the supplementary test case suggestion, dynamically adjusting analysis strategy parameters acco