EP-4742633-A2 - APPARATUS AND METHOD FOR RECORDING AND REPLAYING INTERACTIVE CONTENT IN AUGMENTED/VIRTUAL REALITY IN INDUSTRIAL AUTOMATION SYSTEMS AND OTHER SYSTEMS

EP4742633A2EP 4742633 A2EP4742633 A2EP 4742633A2EP-4742633-A2

Abstract

A method includes receiving (404) data defining user actions associated with an AR/VR space. The method also includes translating (410) the user actions into associated commands and identifying (412) associations of the commands with visual objects in the AR/VR space. The method further includes aggregating (414) the commands, the associations of the commands with the visual objects, and an AR/VR environment setup into at least one file (120) and transmitting or storing (416) the file(s). Another method includes receiving (504) at least one file (120) containing commands, associations of the commands with visual objects in an AR/VR space, and an AR/VR environment setup. The other method also includes translating (508) the commands into associated user actions and recreating or causing a user device (204, 206, 208) to recreate (512-514) (i) the AR/VR space containing the visual objects based on the AR/VR environment setup and (ii) the user actions in the AR/VR space based on the associations of the commands with the visual objects.

Inventors

DUDEKULA, Subhan Basha
AHAMED, Arif Shuail
KONIKI, RAMESH BABU
DUTTA, Manas
PHILLIPS, MARK

Assignees

Honeywell International Inc.

Dates

Publication Date: 20260513
Application Date: 20180529

Claims (7)

A method comprising: receiving (504) at least one file (120) containing commands, associations of the commands with visual objects in an augmented reality/virtual reality (AR/VR) space, and an AR/VR environment setup; translating (508) the commands into associated user actions by using a grammar reference (116) that associates the user actions with different commands; and associating (412) the commands with the visual objects presented in the AR/VR space; and recreating or causing a user device (204, 206, 208) to recreate (512-514) (i) the AR/VR space containing the visual objects based on the AR/VR environment setup and (ii) the user actions in the AR/VR space based on the associations of the commands with the visual objects.
The method of Claim 1, wherein the at least one file (120) has a portable file format that comprises at least one of an XML or JSON file format.
The method of Claim 1, wherein the user actions comprises: one or more gestures made by a user; one or more voice commands or voice annotations spoken by the user: one or more textual messages provided by the user, and one or more pointing actions taken by the user using at least one pointing
The method of Claim 1, wherein translating (508) the commands into associated user actions by using a grammar reference (116) comprises associating each user input action with corresponding commands based on the grammar reference (116) that represents an action-to-command mapping dictionary.
The method of Claim 1, wherein recreation of the user actions is performed automatically or is driven by a user.
An apparatus comprising: at least one processing device (302) configured to perform the method of any of Claims 1- 5.
A non-transitory computer readable medium containing instructions that when executed cause at least one processing device to perform the method of any of Claims 1-5.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS AND PRIORITY CLAIM This application claims priority under 35 U.S.C. § 119(e) to U.S. Provisional Patent Application No. 62/517,006, U.S. Provisional Patent Application No. 62/517,015, and U.S. Provisional Patent Application No. 62/517,037, all filed on June 8, 2017. These provisional applications are hereby incorporated by reference in their entirety. TECHNICAL FIELD This disclosure generally relates to augmented reality and virtual reality systems. More specifically, this disclosure relates to an apparatus and method for recording and replaying interactive content in augmented/virtual reality in industrial automation systems and other systems. BACKGROUND Augmented reality and virtual reality technologies are advancing rapidly and becoming more and more common in various industries. Augmented reality generally refers to technology in which computer-generated content is superimposed over a real-world environment. Examples of augmented reality include games that superimpose objects or characters over real-world images and navigation tools that superimpose information over real-world images. Virtual reality generally refers to technology that creates an artificial simulation or recreation of an environment, which may or may not be a real-world environment. An example of virtual reality includes games that create fantasy or alien environments that can be explored by users. SUMMARY This disclosure provides an apparatus and method for recording and replaying interactive content in augmented/virtual reality in industrial automation systems and other systems. In a first embodiment, a method includes receiving data defining user actions associated with an augmented reality/virtual reality (AR/VR) space. The method also includes translating the user actions into associated commands and identifying associations of the commands with visual objects in the AR/VR space. The method further includes aggregating the commands, the associations of the commands with the visual objects, and an AR/VR environment setup into at least one file. In addition, the method includes transmitting or storing the at least one file. In a second embodiment, an apparatus includes at least one processing device configured to receive data defining user actions associated with an AR/VR space. The at least one processing device is also configured to translate the user actions into associated commands and to identify associations of the commands with visual objects in the AR/VR space. The at least one processing device is further configured to aggregate the commands, the associations of the commands with the visual objects, and an AR/VR environment setup into at least one file. In addition, the at least one processing device is configured to transmit or store the at least one file. In a third embodiment, a method includes receiving at least one file containing commands, associations of the commands with visual objects in an AR/VR space, and an AR/VR environment setup. The method also includes translating the commands into associated user actions. In addition, the method includes recreating or causing a user device to recreate (i) the AR/VR space containing the visual objects based on the AR/VR environment setup and (ii) the user actions in the AR/VR space based on the associations of the commands with the visual objects. In a fourth embodiment, an apparatus includes at least one processing device configured to receive at least one file containing commands, associations of the commands with visual objects in an AR/VR space, and an AR/VR environment setup. The at least one processing device is also configured to translate the commands into associated user actions. In addition, the at least one processing device is configured to recreate or causing a user device to recreate (i) the AR/VR space containing the visual objects based on the AR/VR environment setup and (ii) the user actions in the AR/VR space based on the associations of the commands with the visual objects. In a fifth embodiment, a non-transitory computer readable medium contains instructions that when executed cause at least one processing device to perform the method of the first embodiment or any of its dependent claims. In a sixth embodiment, a non-transitory computer readable medium contains instructions that when executed cause at least one processing device to perform the method of the third embodiment or any of its dependent claims. Other technical features may be readily apparent to one skilled in the art from the following figures, descriptions, and claims. BRIEF DESCRIPTION OF THE DRAWINGS For a more complete understanding of this disclosure, reference is now made to the following description, taken in conjunction with the accompanying drawings, in which: FIGURE 1 illustrates an example architecture for recording interactive content in augmented/virtual reality according to this disclosure;FIGURE 2 illustrates an example architecture for re