US-20260127083-A1 - Hardware Exerciser Having Variable-Length Runtime

US20260127083A1US 20260127083 A1US20260127083 A1US 20260127083A1US-20260127083-A1

Abstract

A hardware exerciser can iteratively perform a plurality of runs of a first test case. For each of the plurality of runs of first test case, at least one performance metric indicating a respective performance of the microprocessor for the respective run can be determined. Responsive to completion of each of the plurality of respective runs of the first test case subsequent to a first run of the first test case, a difference between performance metric(s) determined for a most recent run of the first test case and the performance metric(s) determined for a previous run of the first test case can be determined. For a first most recent run of the first test case, responsive to the difference being greater than a first threshold value, running of the first test case on the microprocessor can be continued.

Inventors

John WIRTH
Larry Leitner
John Alexander Bybel
Rosemary Perez

Assignees

INTERNATIONAL BUSINESS MACHINES CORPORATION

Dates

Publication Date: 20260507
Application Date: 20241107

Claims (20)

1 . A method, comprising: initiating, by a hardware exerciser executed by a processor, a microprocessor to iteratively perform a plurality of runs of a first test case; for each of the plurality of runs of the first test case, determining at least a first performance metric indicating a respective performance of the microprocessor for the respective run of the first test case; responsive to completion of each of the plurality of respective runs of the first test case subsequent to a first run of the first test case, determining a difference between the at least the first performance metric determined for a most recent run of the first test case and the at least the first performance metric determined for a previous run of the first test case; and for a first most recent run of the first test case, responsive to the difference, between the at least the first performance metric determined for the first most recent run of the first test case and the at least the first performance metric determined for a first previous run of the first test case, being greater than a first threshold value, continuing running the first test case on the microprocessor.
2 . The method of claim 1 , further comprising: for a second most recent run of the first test case, responsive to the difference, between the at least the first performance metric determined for the second most recent run of the first test case and the at least the first performance metric determined for a second previous run of the first test case, not being greater than the first threshold value, discontinuing runs of the first test case on the microprocessor.
3 . The method of claim 2 , further comprising: for the second most recent run of the first test case, determining the first most recent run of the first test case to be the second previous run of the first test case.
4 . The method of claim 2 , further comprising: responsive to determining to discontinue runs of the first test case on the microprocessor, automatically generating, by the hardware exerciser, a second test case and initiating the microprocessor to iteratively perform a plurality of runs of the second test case.
5 . The method of claim 1 , further comprising: responsive to determining that a specified duration of time for testing the microprocessor using the test cases has elapsed, discontinuing running the test cases on the microprocessor.
6 . The method of claim 1 , wherein the microprocessor is a microprocessor model.
7 . The method of claim 1 , wherein the microprocessor is a microprocessor chip.
8 . A computer system, comprising: a processor set; one or more computer-readable storage media; and program instructions stored on the one or more storage media to cause the processor set to perform operations comprising: initiating, by a hardware exerciser, a microprocessor to iteratively perform a plurality of runs of a first test case; for each of the plurality of runs of the first test case, determining at least a first performance metric indicating a respective performance of the microprocessor for the respective run of the first test case; responsive to completion of each of the plurality of respective runs of the first test case subsequent to a first run of the first test case, determining a difference between the at least the first performance metric determined for a most recent run of the first test case and the at least the first performance metric determined for a previous run of the first test case; and for a first most recent run of the first test case, responsive to the difference, between the at least the first performance metric determined for the first most recent run of the first test case and the at least the first performance metric determined for a first previous run of the first test case, being greater than a first threshold value, continuing running the first test case on the microprocessor.
9 . The computer system of claim 8 , wherein the operations further comprise: for a second most recent run of the first test case, responsive to the difference, between the at least the first performance metric determined for the second most recent run of the first test case and the at least the first performance metric determined for a second previous run of the first test case, not being greater than the first threshold value, discontinuing runs of the first test case on the microprocessor.
10 . The computer system of claim 9 wherein the operations further comprise: for the second most recent run of the first test case, determining the first most recent run of the first test case to be the second previous run of the first test case.
11 . The computer system of claim 9 , wherein the operations further comprise: responsive to determining to discontinue runs of the first test case on the microprocessor, automatically generating, by the hardware exerciser, a second test case and initiating the microprocessor to iteratively perform a plurality of runs of the second test case.
12 . The computer system of claim 8 , wherein the operations further comprise: responsive to determining that a specified duration of time for testing the microprocessor using the test cases has elapsed, discontinuing running the test cases on the microprocessor.
13 . The computer system of claim 8 , wherein the microprocessor is a microprocessor model.
14 . The computer system of claim 8 , wherein the microprocessor is a microprocessor chip.
15 . A computer program product for variable-length runtime for a hardware exerciser, the computer program product comprising: one or more computer-readable storage media; and program instructions stored on the one or more storage media to perform operations comprising: initiating, by the hardware exerciser, a microprocessor to iteratively perform a plurality of runs of a first test case; for each of the plurality of runs of the first test case, determining at least a first performance metric indicating a respective performance of the microprocessor for the respective run of the first test case; responsive to completion of each of the plurality of respective runs of the first test case subsequent to a first run of the first test case, determining a difference between the at least the first performance metric determined for a most recent run of the first test case and the at least the first performance metric determined for a previous run of the first test case; and for a first most recent run of the first test case, responsive to the difference, between the at least the first performance metric determined for the first most recent run of the first test case and the at least the first performance metric determined for a first previous run of the first test case, being greater than a first threshold value, continuing running the first test case on the microprocessor.
16 . The computer program product of claim 15 , wherein the operations further comprise: for a second most recent run of the first test case, responsive to the difference, between the at least the first performance metric determined for the second most recent run of the first test case and the at least the first performance metric determined for a second previous run of the first test case, not being greater than the first threshold value, discontinuing runs of the first test case on the microprocessor.
17 . The computer program product of claim 16 , wherein the operations further comprise: for the second most recent run of the first test case, determining the first most recent run of the first test case to be the second previous run of the first test case.
18 . The computer program product of claim 16 , wherein the operations further comprise: responsive to determining to discontinue runs of the first test case on the microprocessor, automatically generating, by the hardware exerciser, a second test case and initiating the microprocessor to iteratively perform a plurality of runs of the second test case.
19 . The computer program product of claim 18 , wherein the operations further comprise: responsive to determining that a specified duration of time for testing the microprocessor using the test cases has elapsed, discontinuing running the test cases on the microprocessor.
20 . The computer program product of claim 15 , wherein the microprocessor is a microprocessor model or a microprocessor chip.

Description

BACKGROUND The present invention relates to microprocessors, and more specifically, to hardware exercisers that perform microprocessor verification. Hardware exercisers are useful for verifying microprocessor performance and functionality, both pre-silicon and post-silicon. A hardware exerciser runs test cases on the microprocessor, or microprocessor model, and outputs test results. The test results are analyzed to determine whether the microprocessor meets design expectations. SUMMARY A method includes initiating, by a hardware exerciser executed by a processor, a microprocessor to iteratively perform a plurality of runs of a first test case. The method also can include, for each of the plurality of runs of the first test case, determining at least a first performance metric indicating a respective performance of the microprocessor for the respective run of the first test case. The method also can include, responsive to completion of each of the plurality of respective runs of the first test case subsequent to a first run of the first test case, determining a difference between the at least the first performance metric determined for a most recent run of the first test case and the at least the first performance metric determined for a previous run of the first test case The method also can include, for a first most recent run of the first test case, responsive to the difference, between the at least the first performance metric determined for the first most recent run of the first test case and the at least the first performance metric determined for a first previous run of the first test case, being greater than a first threshold value, continuing running the first test case on the microprocessor. A system includes a processor set and one or more computer-readable storage media. The system also includes program instructions stored on the one or more storage media to cause the processor set to perform operations. The operations correspond to the described method. A computer program product for variable-length runtime for a hardware exerciser includes one or more computer-readable storage media and program instructions stored on the one or more storage media to perform operations. The operations correspond to the described method. This Summary section is provided merely to introduce certain concepts and not to identify any key or essential features of the claimed subject matter. Other features of the inventive arrangements will be apparent from the accompanying drawings and from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 depicts a computing environment according to an embodiment of the present invention. FIG. 2 depicts a microprocessor test environment according to an embodiment of the present invention. FIG. 3 depicts a microprocessor test environment according to an embodiment of the present invention. FIGS. 4A and 4B, together, depict a flowchart illustrating an example of a method of verifying a microprocessor performance. DETAILED DESCRIPTION The arrangements described herein are directed to computer technology, and provide an improvement to computer technology. Specifically, the present arrangements improve microprocessor verification. Typically, using a conventional hardware exerciser, an individual test case is generated and ran a fixed number of times on a microprocessor (e.g., a microprocessor model, a microprocessor integrated circuit chip and/or a microprocessor package). After the test case has been run the fixed number of times, a new test case is generated, and the new test case is ran a fixed number of times. This process repeats until a specified number of test cases are ran the specified number of times, and test results are analyzed. The number of test cases ran on a microprocessor may be in the range of twenty to fifty test cases. During test case runs, there may be parameter values that are missed due to an insufficient number of test runs being specified for each test case. These parameter values may derive from microarchitectural differences in subsequent test case runs due to performance related structures (e.g., structures for branch prediction, cache allocation, etc.) being trained from earlier test case runs. Thus, a conventional hardware exerciser may provide inadequate test coverage for microprocessor verification. A larger number of test case runs can be specified to be ran by the hardware exerciser, but generating test cases and running those test cases is both time intensive and hardware resource intensive for the processing system implementing the hardware exerciser. In this regard, generating and running test cases utilizes a significant amount of processor resources, memory resources and communication resources. Indeed, using a conventional hardware exerciser, generating test cases and running those test cases on a microprocessor model in a simulation farm can take several hours. Microprocessor integrated circuit chips (hereinafter “chips