EP-4738220-A1 - GENERATIVE AI INDUSTRIAL AUTOMATION AUGMENTED REMOTE SUPPORT SERVICES

EP4738220A1EP 4738220 A1EP4738220 A1EP 4738220A1EP-4738220-A1

Abstract

An industrial remote support system acts as an interactive assistant that leverages generative augmented reality (AR) and artificial intelligence (AI) techniques to provide dynamic support information for industrial assets. The system can suggest solutions to performance problems based on earlier documented solutions, thereby expediting the process of finding resolutions. Users can submit information about the industrial asset for which support is requested via an optical or data scan of the asset using an AR-capable client device. The system enhances a user's prompt with relevant contextual data retrieved from stored documentation as well as relevant past chat histories to assist the system's generative AI model in recommending accurate resolutions to alarm conditions or performance issues described by the user's prompt.

Inventors

MEHROTRA, ABHISHEK
TAYLOR, STEVEN P.
WIANT, JESSICA L.
RAVINDRANATH, APARNA
FLORES, BRITNEY

Assignees

Rockwell Automation Technologies, Inc.

Dates

Publication Date: 20260506
Application Date: 20251022

Claims (15)

A system, comprising: a memory that stores executable components; and a processor, operatively coupled to the memory, that executes the executable components, the executable components comprising: a user interface component configured to receive, from a client device associated with an industrial customer, visual data representing shapes of objects within a field of view of the client device; an asset identification component configured to identify an industrial asset within the field of view of the client device based on first analysis of the visual data; a context retrieval component configured to, based on a result of the first analysis performed by the asset identification component, retrieve contextual data determined to be relevant to the industrial asset from a repository of industrial documentation; and a generative AI component configured to generate a natural language response comprising information about the industrial asset based on second analysis of the result of the first analysis performed by the asset identification component, the contextual data, and a response prompted from a generative AI model, wherein the user interface component is configured to render the natural language response on the client device.
The system of claim 1, wherein the information about the industrial asset comprises at least one of training information explaining how to operate or maintain the industrial asset, issue resolution information describing operational or maintenance actions designed to resolve a performance problem experienced by the industrial asset, an identity of the industrial asset, a function of the industrial asset, operational statistics for the industrial asset, information explaining how to activate software on the industrial asset, or contact information for live technical support personnel capable of providing technical support for the industrial asset.
The system of claim 1 or 2, at least one of: wherein the user interface component is further configured to receive, with the visual data, a natural language prompt requesting a type of information about the industrial asset, and the generative AI component is configured to further perform the second analysis on the natural language prompt; wherein the natural language prompt specifies at least one of a description of a performance issue being experienced by the industrial asset, an identity of an alarm generated by the industrial asset, a request for recommended preventative measures to perform on the industrial asset for mitigating future performance issues, or a request for guidance in performing a maintenance task on the industrial asset; and wherein the industrial documentation stored in the repository of industrial documentation comprises at least one of programming manuals, industrial device manuals, industrial device product specification documents, functional specification documents, knowledgebase articles describing solutions to problems associated with industrial devices or software, or failure code documentation.
The system of one of claims 1 to 3, wherein the generative AI component is configured to, as part of the second, formulate a prompt directed to the generative AI model and designed to obtain, as the response from the generative AI model, information used by the generative AI component to generate the natural language response, and the generative AI component generates the prompt based on the result of the first analysis and the contextual data.
The system of one of claims 1 to 4, wherein the context retrieval component is further configured to retrieve, based on the result of the first analysis, chat history data determined to be relevant to the industrial asset, and the generative AI component is configured to further perform the second analysis on the chat history data.
The system of one of claims 1 to 5, at least one of: wherein the user interface component is configured to render the natural language response as an augmented reality presentation on the client device; wherein the visual data is first visual data, the industrial asset is a first industrial asset, and the executable components further comprise an asset registration component configured to, in response to identification of a second industrial asset based on analysis of second visual data received by the user interface, create an asset record for the second industrial asset and store the asset record as part of asset data maintained for the industrial customer; and wherein the industrial asset is at least one of an industrial controller, an I/O module, a motor drive, a human-machine interface terminal, a contactor, an industrial machine, a component of the industrial machine, or a maintenance tool.
The system of one of claims 1 to 6, further comprising a device interface component configured to generate a control instruction directed to the industrial asset based on a result of the second analysis.
The system of claim 7, wherein the control instruction is at least one of an instruction to modify a setpoint of a controlled industrial process, an instruction to change an operating mode of a device or a machine, or an instruction to change a speed of a controlled industrial process.
A method, comprising: receiving, by a system comprising a processor, visual data from a client device associated with an industrial customer, wherein the visual data represents shapes of objects within a field of view of the client device; in response to the receiving, identifying, by the system, an industrial asset within the field of view of the client device based on first analysis of the visual data; retrieving, by the system based on a result of the first analysis, contextual data determined to be relevant to the industrial asset from a repository of industrial documentation; generating, by the system, a natural language response comprising information about the industrial asset based on second analysis of the result of the first analysis, the contextual data, and a response prompted from a generative artificial intelligence (AI) model; and rendering, by the system, the natural language response on the client device.
The method of claim 9, wherein the information about the industrial asset comprises at least one of training information explaining how to operate or maintain the industrial asset, issue resolution information describing operational or maintenance actions designed to resolve a performance problem experienced by the industrial asset, an identity of the industrial asset, a function of the industrial asset, operational statistics for the industrial asset, information explaining how to activate software on the industrial asset, or contact information for live technical support personnel capable of providing technical support for the industrial asset.
The method of claim 9 or 10, further comprising receiving, by the system in association with the visual data, a natural language prompt requesting a type of information about the industrial asset, wherein the second analysis is performed on the result of the first analysis, the contextual data, the response prompted from a generative AI model, and the natural language prompt.
The method of claim 11, wherein the natural language prompt specifies at least one of a description of a performance issue being experienced by the industrial asset, an identity of an alarm generated by the industrial asset, a request for recommended preventative measures to perform on the industrial asset for mitigating future performance issues, or a request for guidance in performing a maintenance task on the industrial asset.
The method of one of claims 9 to 12, at least one of: further comprising, as part of the second analysis: formulating, by the system based on the result of the first analysis and the contextual data, a prompt directed to the generative AI model and designed to obtain, as the response prompted from the generative AI model, information used by the system to generate the natural language response; and wherein the industrial documentation stored in the repository of industrial documentation comprises at least one of programming manuals, industrial device manuals, industrial device product specification documents, functional specification documents, knowledgebase articles describing solutions to problems associated with industrial devices or software, or failure code documentation.
A non-transitory computer-readable medium having stored thereon instructions that, in response to execution, cause a system comprising a processor to perform operations, the operations comprising: receiving visual data from a client device associated with an industrial customer, wherein the visual data represents shapes of objects within a field of view of the client device; in response to the receiving: identifying an industrial asset within the field of view of the client device based on first analysis of the visual data; retrieving, based on a result of the first analysis, contextual data determined to be relevant to the industrial asset from a repository of industrial documentation; generating a natural language response comprising information about the industrial asset based on second analysis of the result of the first analysis, the contextual data, and a response prompted from a generative artificial intelligence (AI) model; and rendering the natural language response on the client device.
The non-transitory computer-readable medium of claim 14, wherein the information about the industrial asset comprises at least one of training information explaining how to operate or maintain the industrial asset, issue resolution information describing operational or maintenance actions designed to resolve a performance problem experienced by the industrial asset, an identity of the industrial asset, a function of the industrial asset, operational statistics for the industrial asset, information explaining how to activate software on the industrial asset, or contact information for live technical support personnel capable of providing technical support for the industrial asset.

Description

TECHNICAL FIELD The subject matter disclosed herein relates generally to industrial automation systems, and, for example, to digitally assisted technical support for industrial assets. BACKGROUND ART Maintenance and troubleshooting of a plant's industrial control systems and their associated machines and devices are typically carried out by on-site service engineers or machine operators. While some types of routine machine alarm or fault conditions can be easily addressed, unfamiliar alarm conditions or system performance issues require the service personnel to expend considerable time and effort finding resolutions to the problems. These resolution efforts can include referencing device or software manuals or contacting a vendor's customer support personnel for assistance in diagnosing and resolving the condition. The above-described deficiencies of current approaches to resolving industrial alarm conditions and performance issues are merely intended to provide an overview of some of the problems of current technology, and are not intended to be exhaustive. Other problems with the state of the art, and corresponding benefits of some of the various non-limiting embodiments described herein, may become further apparent upon review of the following detailed description. BRIEF DESCRIPTION The following presents a simplified summary in order to provide a basic understanding of some aspects described herein. This summary is not an extensive overview nor is it intended to identify key/critical elements or to delineate the scope of the various aspects described herein. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is presented later. In one or more embodiments, a system is provided, comprising a user interface component configured to receive, from a client device associated with an industrial customer, visual data representing shapes of objects within a field of view of the client device; an asset identification component configured to identify an industrial asset within the field of view of the client device based on first analysis of the visual data; a context retrieval component configured to, based on a result of the first analysis performed by the asset identification component, retrieve contextual data determined to be relevant to the industrial asset from a repository of industrial documentation; and a generative AI component configured to generate a natural language response comprising information about the industrial asset based on second analysis of the result of the first analysis performed by the asset identification component, the contextual data, and a response prompted from a generative AI model, wherein the user interface component is configured to render the natural language response on the client device. Also, one or more embodiments provide a method, comprising receiving, by a system comprising a processor, visual data from a client device associated with an industrial customer, wherein the visual data represents shapes of objects within a field of view of the client device; in response to the receiving, identifying, by the system, an industrial asset within the field of view of the client device based on first analysis of the visual data; retrieving, by the system based on a result of the first analysis, contextual data determined to be relevant to the industrial asset from a repository of industrial documentation; generating, by the system, a natural language response comprising information about the industrial asset based on second analysis of the result of the first analysis, the contextual data, and a response prompted from a generative artificial intelligence (AI) model; and rendering, by the system, the natural language response on the client device. Also, according to one or more embodiments, a non-transitory computer-readable medium is provided having stored thereon instructions that, in response to execution, cause a system to perform operations, the operations comprising receiving visual data from a client device associated with an industrial customer, wherein the visual data represents shapes of objects within a field of view of the client device; and in response to the receiving: identifying an industrial asset within the field of view of the client device based on first analysis of the visual data; retrieving, based on a result of the first analysis, contextual data determined to be relevant to the industrial asset from a repository of industrial documentation; generating a natural language response comprising information about the industrial asset based on second analysis of the result of the first analysis, the contextual data, and a response prompted from a generative artificial intelligence (AI) model; and rendering the natural language response on the client device. To the accomplishment of the foregoing and related ends, certain illustrative aspects are described herein in connection with the following des