Search

EP-4740098-A1 - COMPUTER-IMPLEMENTED METHOD FOR ARRANGING ERROR REPORTS IN AT LEAST TWO ERROR REPORT GROUPS OF SIMILAR ERROR SITUATION

EP4740098A1EP 4740098 A1EP4740098 A1EP 4740098A1EP-4740098-A1

Abstract

The invention relates to a computer-implemented method (100) for arranging error reports (10) in at least two error report groups (30) of different error situation (31, 32, 33, 34), and to an associated computer program product (8), group-forming system (5) and system (200), comprising the group-forming system (5) and at least one test system (1).

Inventors

  • Boada, Guillem
  • KAISER, ALEXANDER

Assignees

  • TRUMPF Werkzeugmaschinen SE + Co. KG

Dates

Publication Date
20260513
Application Date
20240624

Claims (15)

  1. 1. Computer-implemented method (100) for grouping error reports (10) into at least two error report groups (30) of different error situations (31, 32, 33, 34), the method (100) comprising at least the following steps (104, 106, 108, 110): - Reading in error reports (10) with at least two error report attachments (11) of different domains (12, 13, 14), - determining a similarity operator (20, 21, 22) for at least two of the different domains (12, 13, 14) to determine a similarity between error report attachments (11) of the same domain (12, 13, 14), - determining similarity values (15) of the error report attachments (11) of the same domain (12, 13, 14) by comparing individual error report attachments (11) of the same domain (12, 13, 14) of different error reports (10) by means of the previously determined similarity operator (20, 21, 22), and - Grouping the error reports (10) into at least two error report groups (30) of different error situations (31, 32, 33, 34) based on the previously determined similarity values (15).
  2. 2. The method (100) of claim 1, wherein the Similarity values (15) are compared with a grouping logic (35) in order to group the error reports (10) into the at least two error report groups (30).
  3. 3. Method (100) according to claim 1 or 2, wherein when determining the similarity values (15), error report attachments (11) of already grouped error reports (10) are compared with read-in error report attachments (11) of not yet grouped error reports (10) by means of the respectively previously determined similarity operator (20, 21, 22).
  4. 4. Method (100) according to one of the preceding claims, wherein the error reports (10) are multimodal and the different domains (12, 13, 14) have different formats (12, 13, 14), in particular different media formats.
  5. 5. The method (100) according to claim 4, wherein the different media formats are from the group comprising image file, video file, audio file and/or text file, in particular error message and/or log file and/or stack trace.
  6. 6. The method (100) according to claim 5, wherein the image file is a screen capture of a test sequence for the error report (10) at an error output time and/or the Video file is a screen video recording of the test run for the bug report (10).
  7. 7. The method (100) according to any one of the preceding claims, wherein the error report attachments (11) of different domains (12, 13, 14) text files comprise different text formats.
  8. 8. Method (100) according to one of the preceding claims, wherein at least one group-based metric is determined for each of the error report groups (30), in particular a ratio of a number of log files (10) of one of the error situations (31, 32, 33, 34) to a total number of log files (10).
  9. 9. The method (100) according to any one of the preceding claims, wherein the method (100) further comprises the step (114) of combining the error reports (10) of at least one of the error report groups (30) into a group error report.
  10. 10. The method (100) according to any one of the preceding claims, wherein the method (100) further comprises the step (116) of determining an error criticality of the error report groups (30) and in particular prioritizing the error report groups (30) according to the determined error criticality.
  11. 11. Method (100) according to one of the preceding claims, wherein machine learning is used in determining a similarity operator (20, 21, 22), in determining the similarity values (15) and/or in grouping the error reports (10).
  12. 12. Computer program product (8) comprising instructions which, when the computer program product is executed by a computer, cause the computer to carry out the method (100) according to one of the preceding claims.
  13. 13. Grouping system (5) comprising a memory (7) and a processor (6) connected to the memory (7), wherein the computer program product (8) according to claim 12 is stored on the memory (7) and the processor (6) is configured to execute the instructions of the computer program product (8).
  14. 14. Grouping system (5) according to claim 13, further comprising at least one interface (9) to at least one test system (1), wherein the at least one test system (1) is configured to generate error reports (10) each having at least two error report attachments (11) of different domains (12, 13, 14).
  15. 15. System (200) comprising the grouping system (5) according to Claim 14 and the at least one test system (1) .

Description

Title: Computer-implemented method for grouping error reports into at least two error report groups of similar error situation Description The invention relates to a computer-implemented method for grouping error reports into at least two error report groups of similar error situations, as well as a computer program product associated therewith, a grouping system, and a system comprising the grouping system and at least one test system. A test system is a system that issues a bug report. The test system can be represented by a computer program that is executed by a corresponding processor, in particular on a production system, a development system, a security system or a quality assurance system. The use of computer programs or software is essential for many technical products because it has become an integral part of these products. This is the case, for example, with machines used in industry, in particular machine tools, which require one or more computer programs to perform their functions. These may, for example, be machine tools that use laser technology for metalworking tasks and chip lithography, which often use particularly complex software applications. The technical products, including their computer programs, can be faulty and issue corresponding error messages. Error messages can also be false alarms, i.e. without a product defect or without a real error in the use of the product, e.g. the machine tool, or in the computer program. In order to provide the products with as few errors as possible, the products are tested in advance. Testing is the process of interacting with the product to check whether it meets the requirements and is free of product defects that could affect its functionality. The goal is to decide whether the product can be released or not. For small products, testing can be successfully performed manually, but as the product increases in size and complexity, especially in the case of the aforementioned machine tools, the size of the system state space grows exponentially and this task becomes no longer feasible manually. In this case it is advisable to carry out automatic testing of the product using a computer-based testing system. For example, automated tests themselves are error-prone, and each failed test requires manual review to verify its result, i.e. to classify its cause as either a product defect or a false alarm. False alarms are due to testing and infrastructure problems, not to defects in the product itself. Developers must examine the defect reports, which consist of error messages, stack traces, and log files, for example, to perform this verification. It is known from the state of the art to automate test procedures. In US 2021/0287109 Al, a computer-implemented method for analyzing test errors using an AI model is described. The method first carries out clustering in which test errors are clustered into individual error clusters. The clustering result is then used to train the AI model in order to automatically identify the cause of new, unclassified errors. The methods known from the state of the art can be used for troubleshooting or identification, but are not as suitable for preventing product defects as would be desirable. For example, it has been found that many error messages, whether false alarms or product defects, are not analyzed in the known test methods. This means that defects can remain in the product and only appear later with the customer. and lead to complications there. Such errors often mean that they have to be resolved through a new test phase, which is very time-consuming. For example, if a machine tool with various product errors is delivered to a customer after a supposedly successful test phase, downtimes caused by product errors can cause major production losses for the customer. The invention is therefore based on the object of increasing the reliability of the products, in particular of proposing a particularly robust computer-implemented method for grouping error reports in order to increase the reliability of the products. The object is achieved by a computer-implemented method for grouping error reports into at least two error report groups of different error situations. The method comprises at least the following steps: - Reading in error reports with at least two error report attachments from different domains, - Determining a similarity operator for each of the different domains to determine a similarity between bug report attachments of the same domain, - Determining similarity values of the error report attachments of the same domain by comparing individual error report attachments of the same domain of different error reports using the previously determined similarity operator, and Grouping the error reports into at least two Error report groups of similar error situations based on the previously determined similarity values. The method according to the invention is therefore based on the error reports being processed