US-20260127093-A1 - SYSTEMS AND METHODS FOR PREDICTION OF TEST FAILURES

Abstract

Method for prediction of test failures are disclosed. A method may include retrieving, by a test failure prediction computer program, test results for a test; training, by the test failure prediction computer program, a machine learning engine based on one or more hypotheses; receiving, by the test failure prediction computer program, metrics for a code file; providing, by the test failure prediction computer program, the metrics for the code file to the trained machine learning engine; receiving, by the test failure prediction computer program, an output from the trained machine learning engine that provides a probability of failure; and outputting, by the test failure prediction computer program, the output.

Inventors

• Georgios Papadopoulos
• Fanny SILAVONG
• Sean Moran
• Alla Nadein

Assignees

• JPMORGAN CHASE BANK, N.A.

Dates

Publication Date

20260507

Application Date

20251222

Priority Date

20220610

Claims (20)

1. 1 . A computer-implemented method for prediction of test failures, comprising: retrieving, by a test failure prediction computer program stored in memory and executed by a computer processor, historical code files and test results for tests of the historical code files; identifying, by the test failure prediction computer program, one or more hypothesis from metrics for the historical code files; training, by the test failure prediction computer program, a machine learning engine based on the one or more hypothesis for the historical code files and the test results; receiving, by the test failure prediction computer program, a code file and metrics for the code file for evaluation; providing, by the test failure prediction computer program, the metrics for the code file for evaluation to the trained machine learning engine; receiving, by the test failure prediction computer program, a prediction of a failure from the trained machine learning engine; outputting, by the test failure prediction computer program, the prediction; and implementing, by the test failure prediction computer program, a corrective action based on the prediction.
2. 2 . The method of claim 1 , wherein the metric comprises a run length of the tests on the historical code files.
3. 3 . The method of claim 1 , wherein the metric comprises a frequency of changes to the historical code files.
4. 4 . The method of claim 1 , wherein the metric comprises a number of users that made changes to the historical code files.
5. 5 . The method of claim 1 , wherein the metric comprises a number of files in the historical code files, or a date and time of test execution.
6. 6 . The method of claim 1 , wherein the test failure prediction computer program receives a plurality of tests to evaluate the code file to be evaluated with, and ranks the plurality of tests based on their probability of failure.
7. 7 . The method of claim 1 , wherein one of the tests comprises a unit test.
8. 8 . (canceled)
9. 9 . The method of claim 1 , wherein the corrective action comprises an update to the code file to be evaluated.
10. 10 . A system, comprising: a code repository comprising code files for evaluation; a test database comprising historical code files and test results for tests of the historical code files; and an electronic device comprising a computer processor and executing a test failure prediction computer program that is configured to retrieve the historical code files and the test results for tests of the historical code files, to identify one or more hypothesis from metrics for the historical code files, to train a machine learning engine based on the one or more hypothesis for the historical code files and the test results, receives one of the code files for evaluation and metrics for the code file, to provide the metrics for the code file for evaluation to the trained machine learning engine, to receive a prediction of a failure from the trained machine learning engine, to output the prediction; and to implement a corrective action based on the prediction.
11. 11 . The system of claim 10 , wherein the metric comprises a run length of the tests on the historical code files.
12. 12 . The system of claim 10 , wherein the metric comprises a frequency of changes to the historical code files.
13. 13 . The system of claim 10 , wherein metric comprises a number of users that made changes to the historical code files.
14. 14 . The system of claim 10 , wherein the metric comprises a number of files in the historical code files, or a date and time of test execution.
15. 15 . The system of claim 10 , wherein the test failure prediction computer program receives a plurality of tests to evaluate the code file to be evaluated with, and ranks the plurality of tests based on their probability of failure.
16. 16 . The system of claim 10 , wherein one of the tests comprises a unit test.
17. 17 . (canceled)
18. 18 . The system of claim 10 , wherein the corrective action comprises an update to the code file to be evaluated.
19. 19 . A non-transitory computer readable storage medium, including instructions stored thereon, which when read and executed by one or more computer processors, cause the one or more computer processors to perform steps comprising: retrieving historical code files and test results for tests of the historical code files; identifying one or more hypothesis from metrics for the historical code files; training a machine learning engine based on the one or more hypothesis for the historical code files and the test results, wherein the metric comprises a frequency of changes to the historical code files, a number of users that made changes to the historical code files, a number of files in the historical code files, a date and time of test execution, or a run length of the tests on the historical code files; receiving a code file and metrics for the code file for evaluation; providing the metrics for the code file for evaluation to the trained machine learning engine; receiving a prediction of a failure from the trained machine learning engine; outputting the prediction; and implementing a corrective action based on the prediction.
20. 20 . (canceled)

Description

RELATED APPLICATIONS This application claims priority to, and the benefit of, Greek Patent Application No. 20220100486, filed Jun. 10, 2022, the disclosure of which is hereby incorporated, by reference, in its entirety. BACKGROUND OF THE INVENTION 1. Field of the invention Embodiments generally relate to systems and methods for prediction of test failures. 2. Description of the Related Art Testing, such as unit testing, is a fundamental practice of good programming standards. It enables test-driven development and shields the application pipeline from faulty software changes and upgrades. Finding program bugs early in development is cheaper than fixing them afterward - it is estimated that the cost of fixing a bug at a late stage of development can be up to thirty times more expensive than fixing it early on. Running thorough tests, however, requires a substantial amount of computational power and time. SUMMARY OF THE INVENTION Systems and methods for prediction of test failures are disclosed. In one embodiment, a method may include: (1) retrieving, by a test failure prediction computer program, historical code files and test results for tests of the historical code files; (2) training, by the test failure prediction computer program, a machine learning engine based on one or more hypothesis for the historical code files and the test results; (3) receiving, by the test failure prediction computer program, a code file and metrics for the code file for evaluation; (4) providing, by the test failure prediction computer program, the metrics for the code file for evaluation to the trained machine learning engine; (5) receiving, by the test failure prediction computer program, a prediction failure from the trained machine learning engine; and (6) outputting, by the test failure prediction computer program, the prediction. In one embodiment, the hypothesis may be based on a run length of the tests on the historical code files. In one embodiment, hypothesis may be based on a frequency of changes to the historical code files, a number of users that made changes to the historical code files, a number of files in the historical code files, a date and time of test execution, etc. In one embodiment, the test failure prediction computer program receives a plurality of tests to evaluate the code file to be evaluated with, and ranks the plurality of tests based on their probability of failure. In one embodiment, one of the tests may include a unit test. In one embodiment, the method may also include: providing, by the test failure prediction computer program, the prediction to a large language model or a generative machine learning model; receiving, by the test failure prediction computer program, a corrective action; and implementing, by the test failure prediction computer program, the corrective action. In one embodiment, the corrective action may include an update to the code file to be evaluated. According to another embodiment, a system may include: a code repository comprising code files for evaluation; a test database comprising historical code files and test results for tests of the historical code files; and an electronic device executing a test failure prediction computer program that retrieves the historical code files and the test results for tests of the historical code files, trains a machine learning engine based on one or more hypothesis for the historical code files and the test results, receives one of the code files for evaluation and metrics for the code file, provides the metrics for the code file for evaluation to the trained machine learning engine, receives a prediction failure from the trained machine learning engine, and outputs the prediction. In one embodiment, the hypothesis may be based on a run length of the tests on the historical code files, a frequency of changes to the historical code files, a number of users that made changes to the historical code files, a number of files in the historical code files, a date and time of test execution, etc. In one embodiment, the test failure prediction computer program receives a plurality of tests to evaluate the code file to be evaluated with, and ranks the plurality of tests based on their probability of failure. In one embodiment, one of the tests may include a unit test. In one embodiment, the test failure prediction computer program provides the prediction to a large language model or a generative machine learning model, retrieves a corrective action, and implements the corrective action. In one embodiment, the corrective action may include an update to the code file to be evaluated. According to another embodiment, a non-transitory computer readable storage medium, may include instructions stored thereon, which when read and executed by one or more computer processors, cause the one or more computer processors to perform steps comprising: retrieving historical code files and test results for tests of the historical code files; training a machin