Search

CN-121996533-A - Test case generation method and device

CN121996533ACN 121996533 ACN121996533 ACN 121996533ACN-121996533-A

Abstract

A method and a device for generating test cases relate to the field of software testing, and the method comprises the steps of obtaining a first test case, inputting the first test case into to-be-tested software inserted with a probe to run so as to obtain constraint information of the first test case, wherein the constraint information indicates the execution condition of constraint passed by the first test case in an execution path of codes related to a target point, the target point comprises code lines with potential vulnerabilities in source codes of the to-be-tested software, and generating a second test case based on the constraint information of the first test case and the first test case, and the constraint information is used for indicating a variation mode of the first test case. The method enables the first test case to mutate against the constraint on the execution path related to the target point, reduces exploration on irrelevant paths which cannot reach the target point, and improves the mutation efficiency of the test case which can trigger the target point.

Inventors

  • WANG YUJIE
  • PAN LINJIE
  • CHANG XIAONING
  • YOU WEI

Assignees

  • 华为云计算技术有限公司
  • 中国人民大学

Dates

Publication Date
20260508
Application Date
20241108

Claims (20)

  1. 1. A method for generating test cases, the method comprising: Acquiring a first test case; inputting the first test case into to-be-tested software inserted with a probe for operation so as to obtain constraint information of the first test case collected by the probe, wherein the constraint information indicates the execution condition of constraint passed by the first test case in an execution path of codes related to a target point, and the target point comprises a code row with potential loopholes in source codes of the to-be-tested software; Generating a second test case based on the first test case and constraint information of the first test case, wherein the constraint information is used for indicating a variation mode of the first test case, and the second test case is used for testing the software to be tested.
  2. 2. The method of claim 1, wherein the method further comprises: Determining a function related to the target point in the source code; Inserting the probe in the source code based on the function; The method for inputting the first test case into the software to be tested after the probe is inserted for operation comprises the following steps: And inputting the first test case into an executable program obtained by compiling the source code inserted with the probe for operation.
  3. 3. The method of claim 2, wherein determining a function in the source code that is associated with the target point comprises: determining a data stream from the source code and the target point, the data stream indicating a variable associated with the target point and a relationship between the variables; The data flow related node is determined according to the data flow, the data flow related node comprises a code row where the operation affecting the variable in the source code is located, and the function related to the target point comprises a function where the code row is located.
  4. 4. The method of claim 3, wherein the probe is further configured to collect coverage information for the first test case, the coverage information indicating code lines covered by the first test case; The method further comprises the steps of: acquiring information of a history constraint, wherein the information of the history constraint indicates the priority of the history constraint; Determining newly added constraints in the first test case based on constraint information of the first test case; the priority of the new constraint is determined, the priority of the new constraint is determined based on coverage rate information of the first test case and the score of the data flow related node, the higher the correlation between the data flow related node and the target point is, the higher the score is, and the priority of the new constraint is higher than the priority of the history constraint.
  5. 5. The method of any of claims 1-4, wherein the constraint information comprises an association between bytes included in a first test case and constraints passed in the keypath; the generating a second test case based on the first test case and constraint information of the first test case includes: Determining a first mutation strategy based on constraint information of the first test case and the association relation, wherein the first mutation strategy indicates a mutation object and/or a mutation mode, the mutation object comprises bytes associated with constraints to be solved in the first test case, and the mutation mode indicates that the mutation object meets the conditions of the constraints to be solved; and mutating the first test case according to the first mutation strategy to generate the second test case.
  6. 6. The method of claim 5, wherein the probe is further configured to collect propagation paths of bytes included in the first test case in a code row covered by the first test case, the association being determined based on the propagation paths.
  7. 7. The method of any one of claims 1-6, wherein the method further comprises: Acquiring description information of the target point, wherein the description information comprises a vulnerability type of the target point and/or a variable related to the target point; and generating a second test case based on the first test case and the description information of the target point.
  8. 8. The method of claim 7, wherein the probe is further configured to collect propagation paths of bytes included in the first test case in a code row covered by the first test case; Generating a second test case based on the first test case and the description information of the target point comprises the following steps: Determining a second mutation strategy based on the description information of the target point, wherein the second mutation strategy indicates a mutation object and/or a mutation mode, the mutation object comprises bytes associated with variables related to the target point, and the mutation mode indicates the operation that the mutation object meets the vulnerability triggering condition of the target point; and mutating the first test case according to the first mutation strategy to generate the second test case.
  9. 9. The method according to claim 7 or 8, wherein the acquiring the description information of the target point includes: determining the vulnerability type of the target point and/or the variables related to the target point based on the source code, or And acquiring static analysis data of the source code, wherein the static analysis data comprises the vulnerability type of the target point and/or variables related to the target point.
  10. 10. A test case generating device, the device comprising: the acquisition module is used for acquiring a first test case; the test module is used for inputting the first test case into to-be-tested software inserted with a probe and running the to-be-tested software to obtain constraint information of the first test case collected by the probe, wherein the constraint information indicates the execution condition of constraint passed by the first test case in a critical path, the critical path comprises an execution path of codes related to a target point, and the target point comprises a code row with potential loopholes in source codes of the to-be-tested software; The generation module is used for generating a second test case based on the first test case and constraint information of the first test case, wherein the constraint information is used for indicating a variation mode of the first test case, and the second test case is used for testing the software to be tested.
  11. 11. The apparatus of claim 10, wherein the apparatus further comprises a determination module; The determining module is used for determining a function related to the target point in the source code; Inserting the probe in the source code based on the function; the test module is specifically configured to, when running in the software to be tested after the first test case input is inserted into the probe: And inputting the first test case into an executable program obtained by compiling the source code inserted with the probe for operation.
  12. 12. The apparatus of claim 11, wherein the determination module, when determining the function associated with the target point in the source code, is specifically configured to: The method comprises the steps of determining a data stream according to a source code and a target point, wherein the data stream indicates a variable related to the target point and a relation between the variable, determining a data stream related node according to the data stream, the data stream related node comprises a code row which influences the operation of the variable in the source code, and the function related to the target point comprises a function which is located by the code row.
  13. 13. The apparatus of claim 12, wherein the probe is further configured to collect coverage information for the first test case, the coverage information indicating code lines covered by the first test case; the acquisition module is further used for acquiring information of historical constraints, wherein the information of the historical constraints indicates priority of the historical constraints; The determining module is further configured to determine a new constraint in the first test case based on constraint information of the first test case, determine a priority of the new constraint, where the priority of the new constraint is determined based on coverage information of the first test case and a score of the data flow related node, the higher the correlation between the data flow related node and the target point is, the higher the score is, and the higher the priority of the new constraint is than the priority of the history constraint.
  14. 14. The apparatus of any of claims 10-13, wherein the constraint information comprises an association between bytes included in a first test case and constraints passed in the keypath; The generation module is specifically configured to, when generating a second test case based on the first test case and constraint information of the first test case: Determining a first mutation strategy based on constraint information of the first test case and the association relation, wherein the first mutation strategy indicates a mutation object and/or a mutation mode, the mutation object comprises bytes associated with constraints to be solved in the first test case, and the mutation mode indicates that the mutation object meets the conditions of the constraints to be solved; and mutating the first test case according to the first mutation strategy to generate the second test case.
  15. 15. The apparatus of claim 14, wherein the probe is further configured to collect propagation paths of bytes included in the first test case in a code row covered by the first test case, the association being determined based on the propagation paths.
  16. 16. The apparatus of any one of claim 10 to 15, The acquisition module is further configured to determine description information of the target point, where the description information includes a vulnerability type of the target point and/or a variable related to the target point; the generation module is further used for generating a second test case based on the first test case and the description information of the target point.
  17. 17. The apparatus of claim 16, wherein the probe is further configured to collect propagation paths of bytes included in the first test case in a code row covered by the first test case; the generation module is specifically configured to determine a second mutation policy based on the description information of the target point when generating a second test case based on the first test case and the description information of the target point, where the second mutation policy indicates a mutation object and/or a mutation mode, the mutation object includes bytes associated with variables related to the target point, the mutation mode indicates an operation that the mutation object meets a vulnerability triggering condition of the target point, and mutate the first test case according to the first mutation policy to generate the second test case.
  18. 18. The apparatus according to claim 16 or 17, wherein the acquiring module, when acquiring the description information of the target point, is specifically configured to: determining the vulnerability type of the target point and/or the variables related to the target point based on the source code, or And acquiring static analysis data of the source code, wherein the static analysis data comprises the vulnerability type of the target point and/or variables related to the target point.
  19. 19. A cluster of computing devices, comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method of any one of claims 1 to 9.
  20. 20. A computer program product containing instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method of any of claims 1 to 9.

Description

Test case generation method and device Technical Field The present application relates to the field of software testing, and in particular, to a method and apparatus for generating test cases. Background With the development of software automation test, in order to guarantee the automated software security and quality test technology, more powerful detection tools and verification means are required for problematic code segments. The fuzzy test is one of the current very popular dynamic test technologies, automatically generates a large number of test cases with randomness, inputs the test cases into the software to be tested in dynamic operation, triggers software abnormality, and discovers software defects (bugs). Thus, fuzzy testing can ensure extremely low false positive rates by providing actual inputs that trigger anomalies. According to different degrees of understanding of the test targets, the fuzzy test can be divided into black box, white box and gray box fuzzy test, wherein the gray box fuzzy test is the software test method with the highest expandability and practicability. Because there is a strong correlation between code coverage and bug, most gray box fuzzing tools are coverage-guided. However, blindly spreading code coverage is inefficient because most of the covered code may not contain bugs. The time spent for exploring irrelevant paths in the test cases generated by the existing scheme is excessive, and the speed of generating the test cases triggering the breakdown of the specific target point is slowed down. Disclosure of Invention The application provides a test case generation method and device, which are used for reducing the exploration cost of test cases in invalid paths and improving the mutation efficiency of the test cases for triggering specific code line loopholes. In a first aspect, the application provides a test case generation method, which comprises the steps of obtaining a test case (marked as a first test case) of a code to be tested (such as software to be tested), wherein the first test case can be an initial test case or one test case selected from a test case set, inputting the first test case into the software to be tested with a probe for operation, obtaining constraint information of the first test case collected by the probe, wherein the constraint information indicates an execution condition of a constraint passed by the first test case in a critical path, the critical path refers to an execution path of the code which is executed by the first test case and is related to a target point, the target point comprises a code line which possibly exists in a source code of the software to be tested, the constraint comprises logic constraint (such as if or switch and the like) in the source code, and the constraint information of the first test case is used for indicating that the first test case is mutated so as to generate a second test case, and the constraint information of the first test case is used for indicating mutation of the first test case, such as to indicate that the mutation of the first test case is solved in the first test case. Through the design, the first test case is guided to be mutated based on the constraint information of the first test case, and in the mutation process, the first test case can be mutated according to the constraint on the execution path related to the target point, so that exploration on irrelevant paths which cannot reach the target point is reduced, and the mutation efficiency of the test case capable of triggering the target point is improved. In one possible design, the method further includes determining a function (related function of the target point) related to the target point in the source code, inserting probes in the function entry locations related to the target point in the source code or in the function, such as before each branch entry of the function, where the probes are used to collect execution cases of test cases on the constraints, if the test cases are executed to the constraint, selecting which branch of the constraint is executed, attribute information of the constraint (such as about a constraint type, whether the constraint contains a variable, a constant, a type of a variable, etc.), and the like, and compiling the source code with the probes inserted therein to obtain an executable program of the software under test, where the first test case is input into the software under test after the probes is executed, where the first test case is input into the executable program. By the design, the probe code is inserted into the function related to the target point in the source code to collect the information of the test case on the program execution path related to the target point, so that the collection and use of irrelevant information in the test process are reduced. In one possible design, determining the function associated with the target point in the source code includes determining a data flo