Search

EP-4455958-B1 - AUTOMATIC INSIGHT INTO TICKET SUPPORT PROCESSES VIA XAI EXPLANATION OF PREDICTION MODELS

EP4455958B1EP 4455958 B1EP4455958 B1EP 4455958B1EP-4455958-B1

Inventors

  • GEADA, ROBERT
  • CAUGHEY, Nicholas

Dates

Publication Date
20260506
Application Date
20231013

Claims (11)

  1. A method comprising: receiving an indication of one or more desired statistical parameters to be optimized, the one or more desired statistical parameters being part of a set of statistical parameters relating to performance of a support stack; for each of a plurality of support tickets input to the support stack: analyzing the support ticket using an artificial intelligence, AI, model to generate a set of predicted resolution statistics including predicted values for each of the one or more desired statistical parameters, and analyzing the set of predicted resolution statistics using an explainable artificial intelligence, XAI, algorithm to generate a set of explanations for the set of predicted resolution statistics; and aggregating the set of explanations for each of the plurality of support tickets to generate one or more insights regarding the one or more desired statistical parameters; wherein each explanation in the set of explanations comprises: a plurality of different words that the support ticket is comprised of; and for each of the plurality of different words, an associated cost regarding a desired statistical parameter of the one or more desired statistical parameters; wherein aggregating the set of explanations comprises: for each word among the set of explanations, averaging the associated cost regarding the desired statistical parameter across each explanation where the word occurs.
  2. The method of claim 1, further comprising: training the AI model to predict values for each of the one or more desired statistical parameters based on data in a support ticket.
  3. The method of claim 1, wherein the AI model is trained using a database of previously resolved support tickets and corresponding resolution statistics.
  4. The method of claim 1, wherein the set of statistical parameters comprises: an amount of time required to resolve a support ticket, a number of reassignments required to resolve the support ticket, a personnel cost required to resolve the support ticket, and an indication of whether any terms of a service level agreement, SLA, were breached.
  5. The method of claim 1, wherein generating the set of explanations for the set of predicted resolution statistics of a support ticket comprises: generating a set of synthetic support tickets, each of the set of synthetic support tickets comprising a permutation of the support ticket; querying the AI model with each of the set of synthetic support tickets; and generating the set of explanations for the set of predicted resolution statistics of the support ticket based predicted resolution statistics generated by the AI model for each of the set of synthetic support tickets and the set of predicted resolution statistics.
  6. A system comprising: a memory; and a processing device operatively coupled to the memory, the processing device to: receive an indication of one or more desired statistical parameters to be optimized, the one or more desired statistical parameters being part of a set of statistical parameters relating to performance of a support stack; train an AI model to predict values for each of the one or more desired statistical parameters based on data in a support ticket; for each of a plurality of support tickets input to the support stack: analyze the support ticket using the artificial intelligence, AI, model to generate a set of predicted resolution statistics including predicted values for each of the one or more desired statistical parameters; and analyze the set of predicted resolution statistics using an explainable artificial intelligence, XAI, algorithm to generate a set of explanations for the set of predicted resolution statistics; and aggregate the set of explanations for each of the plurality of support tickets to generate one or more insights regarding the one or more desired statistical parameters; wherein each explanation in the set of explanations comprises: a plurality of different words that the support ticket is comprised of; and for each of the plurality of different words, an associated cost regarding a desired statistical parameter of the one or more desired statistical parameters; wherein to aggregate the set of explanations, the processing device is to: for each word among the set of explanations, average the associated cost regarding the desired statistical parameter across each explanation where the word occurs.
  7. The system of claim 6, wherein the AI model comprises a neural network.
  8. The system of claim 6, wherein the AI model is trained using a database of previously resolved support tickets and corresponding resolution statistics.
  9. The system of claim 6, wherein the set of statistical parameters comprises: an amount of time required to resolve a support ticket, a number of reassignments required to resolve the support ticket, a personnel cost required to resolve the support ticket, and an indication of whether any terms of a service level agreement, SLA, were breached.
  10. The system of claim 6, wherein to generate the set of explanations for the set of predicted resolution statistics of a support ticket, the processing device is to: generate a set of synthetic support tickets, each of the set of synthetic support tickets comprising a permutation of the support ticket; query the AI model with each of the set of synthetic support tickets; and generate the set of explanations for the set of predicted resolution statistics of the support ticket based predicted resolution statistics generated by the AI model for each of the set of synthetic support tickets and the set of predicted resolution statistics.
  11. A non-transitory computer-readable medium having instructions stored thereon which, when executed by a processing device cause the processing device to perform operations comprising the method of any of claims 1 to 5.

Description

TECHNICAL FIELD Aspects of the present disclosure relate to artificial intelligence (AI)-enhanced support ticket analysis, and more specifically to use of an explainable artificial intelligence (XAI) model to explain how and why an AI model that predicts support ticket resolution statistics arrives at its predictions. BACKGROUND Customer support within the IT industry often operates on a ticketing system, where customers create support tickets describing their particular issue with the software/service being offered by a company. Examples of issues can include malfunctions/bugs as well as security vulnerabilities. These support tickets are then triaged by the company (which may hold e.g., a service level agreement (SLA) with the client), which assigns tickets to the company's customer support agents according to the problem domain, complexity, and priority. Once the ticket's issue is resolved, the ticket is marked closed, and statistics about the ticket's resolution are collected. These statistics include the time required to resolve the support ticket, the number of reassignments (e.g., how often the ticket needed to be moved to a different support agent or department) required to resolve the support ticket, the relative difficulty/personnel cost of resolving the support ticket, and whether the terms of the SLA were breached. Document D1 2023/129123 A1 discloses automatically providing issue prediction for trouble ticket, involving utilizing a categorized set of symptoms and identified sets or IT assets and utilizing predicted case classification to resolve trouble ticket. US 2022/058347 A1 relates to a chabot system configured to execute code to perform determining a classification result for an utterance. SUMMARY According to an aspect of the present disclosure there is provided a method as disclosed by claim 1 The AI model may be trained to predict values for each of the one or more desired statistical parameters based on data in a support ticket, and it may be trained using a database of previously resolved support tickets and corresponding resolution statistics. Each explanation in the set of explanations may comprise a plurality of different words that the support ticket is comprised of, and for each of the plurality of different words, an associated cost regarding a desired statistical parameter of the one or more desired statistical parameters. In this case, aggregating the set of explanations may comprise, for each word among the set of explanations, averaging the associated cost regarding the desired statistical parameter across each explanation where the word occurs. The set of statistical parameters may comprise an amount of time required to resolve a support ticket, a number of reassignments required to resolve the support ticket, a personnel cost required to resolve the support ticket, and an indication of whether any terms of a service level agreement (SLA) were breached. Generating the set of explanations for the set of predicted resolution statistics of a support ticket may comprise generating a set of synthetic support tickets, each of the set of synthetic support tickets comprising a permutation of the support ticket, querying the AI model with each of the set of synthetic support tickets, and generating the set of explanations for the set of predicted resolution statistics of the support ticket based predicted resolution statistics generated by the AI model for each of the set of synthetic support tickets and the set of predicted resolution statistics. According to a further aspect of the present disclosure, there is provided a system comprising: a memory; anda processing device operatively coupled to the memory, the processing device to: receive an indication of one or more desired statistical parameters to be optimized, the one or more desired statistical parameters being part of a set of statistical parameters relating to performance of a support stack;train an AI model to predict values for each of the one or more desired statistical parameters based on data in a support ticket;for each of a plurality of support tickets input to the support stack: analyze the support ticket using the artificial intelligence (AI) model to generate a set of predicted resolution statistics including predicted values for each of the one or more desired statistical parameters, andanalyze the set of predicted resolution statistics using an explainable artificial intelligence (XAI) algorithm to generate a set of explanations for the set of predicted resolution statistics; andaggregate the set of explanations for each of the plurality of support tickets to generate one or more insights regarding the one or more desired statistical parameters. The AI model may comprise a neural network, and it may be trained using a database of previously resolved support tickets and corresponding resolution statistics. Each explanation in the set of explanations may comprise a plurality of different words that the support t