Search

CN-116324734-B - Generating and updating performance reports

CN116324734BCN 116324734 BCN116324734 BCN 116324734BCN-116324734-B

Abstract

A computer-implemented method for generating and updating performance reports for a system running multiple streams is disclosed that enables performance data for each running stream to be compared to a stream having similar functionality. This is accomplished by linking each run-time flow to one of a plurality of templates describing the functions of the flow. Each template may have an associated performance profile generated for it based on performance data of the run stream linked to the template. A performance report may be generated from a plurality of performance summaries. The performance report may then be updated by running multiple test flows on the standby capability of the system while test performance data is obtained from the test flows. The test performance data may be compared to performance data of the run-time streams on the system linked to the templates.

Inventors

  • J. A. Ruiwei
  • R. Conviri
  • M. Barry
  • A.D. Harrison

Assignees

  • 国际商业机器公司

Dates

Publication Date
20260512
Application Date
20211008
Priority Date
20201021

Claims (17)

  1. 1. A computer-implemented method for generating performance reports for a system configured to run multiple streams, wherein the streams refer to a set of actions that are linked together to achieve a task, comprising: Obtaining performance data associated with each of a plurality of operational flows on a system, wherein each of the plurality of operational flows comprises a connected sequence of nodes, wherein each node of the connected sequence of nodes is configured to perform the task, and wherein each of the plurality of operational flows is associated with one of a plurality of users; obtaining a plurality of templates, wherein each template of the plurality of templates describes a function of a stream; linking each of the plurality of run streams to one of the plurality of templates describing the functionality of the linked run stream; Generating, for each of one or more templates describing the functionality of the linked execution flows, a performance summary based on performance data associated with each of a plurality of execution flows linked to the template, wherein the performance summary is not generated for the template if the number of execution flows linked to the template does not satisfy a condition based on a predetermined number, and Generating a performance report based on the generated one or more performance summaries, wherein generating the performance summary further comprises: Determining, for each of the one or more templates, that a number of run streams linked to the template satisfies a predetermined number-based condition, and For each of the one or more templates, generating the performance summary based on the performance data associated with each of the plurality of run streams linked to the template.
  2. 2. The computer-implemented method of claim 1, wherein obtaining performance data comprises: For each of the plurality of operational flows, obtaining one or more selected from the group consisting of: latency of the run stream; a data rate per second of the operational flow; Cost of the operation flow, and The time the running stream is executed on the system.
  3. 3. The computer-implemented method of claim 1, wherein the performance summary comprises one or more selected from the group consisting of: An average of the performance data of the plurality of run streams linked to the template; a range of the performance data of the plurality of run streams linked to the template, and A list of one or more of the performance data of the plurality of run streams linked to the template.
  4. 4. The computer-implemented method of claim 1, wherein linking each of the plurality of run streams comprises: Determining, for each of the plurality of run streams, a template of the plurality of templates that most closely describes the function of the run stream, and Linking each of the plurality of run streams to one of the plurality of templates based on the determination.
  5. 5. The computer-implemented method of claim 1, wherein generating the performance report further comprises: receiving a performance report query from a user of the plurality of users; Generating the performance report based on the generated one or more performance summaries, wherein each of the one or more performance summaries is based on performance data associated with one or more of the plurality of operational flows, wherein at least one of the one or more of the plurality of operational flows is associated with the user of the plurality of users, and In response to receiving the performance report query, the performance report is sent to the user of the plurality of users.
  6. 6. The computer-implemented method of claim 1, wherein the system is a multi-tenant integrated cloud system, and wherein each of the plurality of run streams is allocated a predetermined processing capacity of the system.
  7. 7. The computer-implemented method of claim 1, further comprising: identifying that a first one of the plurality of run streams cannot be linked to one of the plurality of templates because the first run stream has a first function that the plurality of templates do not describe, and A new template describing the first function of the first operational flow is generated, wherein the new template describes the first function of the first operational flow and is generated by summarizing an existing template.
  8. 8. The computer-implemented method of claim 1, wherein the condition requires linking two or more of the run streams to each of the one or more templates to generate the performance profile.
  9. 9. A computer-implemented method for updating a performance report of a system configured to run multiple flows, the performance report generated for the system using the method of any of claims 1-8, comprising: obtaining a performance report, wherein the performance report is based on one or more performance summaries associated with one or more of the plurality of templates; obtaining a plurality of test streams, wherein each test stream is associated with one of the plurality of templates; Executing the set of the plurality of test flows on the system; obtaining test performance data associated with each of the test flows in the set of the plurality of test flows; Generating, for at least one of the plurality of templates, a test performance summary based on test performance data associated with each of the test flows associated with the template, and The performance report is updated based on the generated one or more test performance summaries.
  10. 10. The computer-implemented method of claim 9, wherein obtaining test performance data comprises: for each test flow in the set of multiple test flows, obtaining one or more selected from the group consisting of: latency of the test flow; a data rate per second of the test stream; Cost of test flow, and Time at which the test flow is executed on the system.
  11. 11. The computer-implemented method of claim 9, wherein executing the set of the plurality of test flows comprises: determining a set of the plurality of test flows based on idle processing capabilities of the system, and Executing a set of the plurality of test flows on the system.
  12. 12. The computer-implemented method of claim 11, wherein determining the set of the plurality of test flows further comprises: Determining idle processing capability of the system; In response to determining that the idle processing capability is above a predetermined capability threshold, increasing an amount of processing capability of the system used by the test flow, and In response to determining that the idle processing capability is below a predetermined capability threshold, the amount of processing capability of the system used by the test flow is reduced.
  13. 13. The computer-implemented method of claim 12, wherein increasing the amount of processing capacity of the system used by the test flow comprises at least one selected from the group consisting of: adding one or more test streams to the set of the plurality of test streams, and Increasing data throughput of one or more of the test flows in the set of the plurality of test flows, an Wherein reducing the amount of processing power of the system used by the test flow comprises at least one of: removing one or more test streams from the set of the plurality of test streams, and Reducing the data throughput of one or more of the test flows in the set of the plurality of test flows.
  14. 14. A system, comprising: Processor, and A computer readable storage medium communicatively coupled to the processor and storing program instructions that, when executed by the processor, cause the processor to perform a method for generating performance reports for a system configured to run multiple streams according to any of claims 1-8.
  15. 15. A system, comprising: Processor, and A computer readable storage medium communicatively coupled to the processor and storing program instructions that, when executed by the processor, cause the processor to perform a method for updating a performance report of a system configured to run multiple streams according to any of claims 9-13.
  16. 16. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions being executable by a processor to cause the processor to perform a method for generating a performance report for a system configured to run a plurality of streams according to any of claims 1-8.
  17. 17. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions being executable by a processor to cause the processor to perform a method for updating a performance report of a system configured to run multiple streams according to any of claims 9-13.

Description

Generating and updating performance reports Background The present disclosure relates to reporting performance of systems, and more particularly, to generating performance reports for systems configured to run multiple flows. Customers are increasingly accessing applications provided by third parties. Third parties host the applications and typically deliver them to customers via the internet. This occurs in the form of software as a service (SaaS), infrastructure as a service (IaaS) and/or platform as a service (PaaS). For many applications, a customer purchases multiple licenses or processing power to enable access to the application. The system capabilities (SYSTEM CAPACITY) required by the customer are typically specified by the capability planning process. Typically, the capacity planning process considers the type of processing required by the customer and then uses the sample performance data to calculate the required capacity. However, the capacity planning process typically involves sampling the performance of only a very limited set of scenarios, as these scenarios need to be set manually for testing (or set with limited automation). In practice, there are many aspects that can vary, such as the complexity of the stream, the message size, and the number of back-end calls required. Because of so many variations, it is time consuming and difficult to obtain accurate capacity demand predictions. Thus, prediction of the required capacity will typically include 100% overhead to account for prediction inaccuracy. In this case, if the predicted outcome is accurate, the customer may eventually pay twice the amount of capacity. Disclosure of Invention According to an embodiment of the present disclosure, a computer-implemented method for generating performance reports for a system configured to run multiple flows is provided. The method includes obtaining performance data associated with each of a plurality of operational flows on a system. The method also includes obtaining a plurality of templates, wherein each template describes a function of the stream. The method also includes linking each of the plurality of run streams to a template of the plurality of templates that describes the functionality of the linked run streams. The method also includes generating, for each of the one or more templates, a performance summary based on performance data associated with each of a plurality of run streams linked to the template. The method also includes generating a performance report based on the generated one or more performance summaries. A concept is presented that links a stream running on a system to a template describing the functions of the stream. This enables similar flows to be grouped and the performance of the flows can then be measured and compared. As a result, performance reports can be generated in which the performance of similar streams can be seen and compared. Thus, the proposed embodiments can provide context to performance data that would otherwise not be available, e.g., like average performance or possible range of flows. Such context information may be useful to the client when evaluating performance and/or to inform the changes to be made to the stream or the system on which the stream is running. By grouping streams running on the system into templates, multiple similar sets of streams can be defined. Performance data for all streams may then be collected and compared. In this way, performance can be compared to other streams having common functions. By generating performance reports from the performance data, customers can easily see how their flows compare to other similar flows and can be notified of the customer's capacity planning process. Thus, some proposed embodiments may involve generating a performance report containing contextualized performance data for each flow running on the system. Linking of streams may be accomplished by linking the streams to templates that describe the functions of the streams. Many streams may be linked to each template and when performance data for each stream is obtained, performance data for streams with similar functionality may be compared. A performance summary may be generated for each template, which may include, for example, a list of average performance, best performance, and all flows and their associated performance for each flow linked to the template. Thus, some of the proposed embodiments may provide the benefit of being able to provide performance data that accurately reflects the actual performance characteristics of the system. In particular, accurate and contextualized performance data may be provided that may greatly improve the capacity planning process and help avoid various performance related problems (e.g., caused by providing insufficient capacity). Such information is particularly useful to new customers planning to use the system, for example by providing an understanding of how much capacity they will need. As a furthe