Search

EP-4128829-B1 - DEVICE AND METHOD FOR PROVIDING AUGMENTED REALITY INTERACTION

EP4128829B1EP 4128829 B1EP4128829 B1EP 4128829B1EP-4128829-B1

Inventors

  • SPOERRI, Reto

Dates

Publication Date
20260513
Application Date
20210322

Claims (17)

  1. A communication device (1), comprising an electronic circuit (13) configured to: obtain (S1) location information (21) of a first user (2) of the communication device (1); transmit (S2) a query request via a communication network (3) to a computerized virtual real estate directory system (4), the query request comprising at least the location information (21) of the first user (2); receive (S3) from the virtual real estate directory system (4) via the communication network (3) a query response, the query response comprising one or more virtual real estate objects (41) assigned to a virtual real estate object location (42) within a predefined region (R) defined by the location information of the first user (2), each of the virtual real estate objects (41) comprising identification information (43) and a network address (44) of an augmented reality interaction server (5); transmit (S5) for at least one particular virtual real estate object (41) an interaction request via the communication network (3) to the augmented reality interaction server (5), respectively using the network address (44) of the augmented reality interaction server (5), each interaction request comprising the identification information (43) of the particular virtual real estate object (41); receive (S6) from the augmented reality interaction server via the communication network (3) access authorization for the particular virtual real estate object (41); transmit (S7) first user data to a communication device (1B) of a second user (2B), associated with the particular virtual real estate object (41), via the augmented reality interaction server (5); and receive (S8) second user data from the communication device (1B) of the second user (2B) via the augmented reality interaction server (5).
  2. The communication device (1) of claim 1, wherein the electronic circuit (13) is configured to: transmit a public user identifier of the first user (2) to the augmented reality interaction server (5) comprising one or more of the following: a public name of the first user (2) or an encryption key of the first user (2); and receive, from the augmented reality interaction server (5) a public user identifier of the second user (2B) comprising one or more of the following: a public name of the second user (2B) or an encryption key of the second user (2B).
  3. The communication device (1) of claim 1 or 2, wherein the communication device (1) comprises one or more cameras (12) and a rendering system (11) and wherein the electronic circuit (13) is configured to: determine (S10) visual representation data of the first user (2) using one or more images of the first user (2) captured by the one or more cameras (12); transmit (S11) the visual representation data of the first user (2), included in the first user data, to the communication device (1B) of the second user via the augmented reality interaction server (5); receive (S12), via the augmented reality interaction server (5), from the communication device (1B) of the second user visual representation data of the second user (2B), included in the second user data; and render (S13), using the rendering system (11) and the visual representation data of the second user (2B), augmented reality content comprising a visual representation (54) of the second user (2B).
  4. The communication device (1) of claim 3, wherein the electronic circuit (13) is further configured to: determine a field of view (F) of the first user (2) using the one or more cameras (12); determine from the field of view (F) coordinates (C) of real-world features in the field of view (F); and render (S13) the augmented reality content using the rendering system (11) such that the augmented reality content has a fixed position relative to the coordinates (C) of the real world features.
  5. The communication device (1) of one of claims 3 or 4, wherein the electronic circuit (13) is further configured to: verify (S221) whether the augmented reality content (51) is entirely contained within a bounding volume (V) of the virtual real estate object (41); responsive to positive verification, render (S23) the augmented reality content (51); and responsive to negative verification to either: not render (S232) the augmented reality content (51), or modify (S222) the augmented reality content (51) and render (S231) modified augmented reality content fitting (52) within the bounding volume (V) of the virtual real estate object (41).
  6. The communication device (1) of one of claims 1 to 5, wherein the communication device (1) further comprises a motion tracker (16) and wherein the electronic circuit (13) is further configured to: record user motion data related to body motion of the first user (2), using the motion tracker (16); and transmit the user motion data, included in the first user data, to the augmented reality interaction server (5).
  7. The communication device (1) of one of claims 1 to 6, wherein the communication device (1) further comprises one or more microphones (15) and one or more speakers (14) and wherein the electronic circuit (13) is configured to: record audio data of the first user (2), using the one or more microphones (15); transmit the audio data of the first user (2), included in the first user data, to the augmented reality interaction server (5); receive from the communication device (1B) of the second user audio data of the second user (2B), included in the second user data; and play, on the one or more speakers (14), the audio data of the second user (2B).
  8. The communication device (1) of one of claims 1 to 7, wherein the electronic circuit (13) is further configured to: receive (S15) an augmented reality object (52) from the augmented reality interaction server (5); render (S16) augmented reality content comprising the augmented reality object (52) using the rendering system (11); determine (S17) user interaction data of the first user (2) interacting with the augmented reality object (51); transmit (S18) the user interaction data to the augmented reality interaction server (5); receive (S19) an updated augmented reality object (52) from the augmented reality interaction server (5); and render (S20) the augmented reality content comprising the updated augmented reality object (52) using the rendering system (11).
  9. A method of interaction in augmented reality for a first user (2) of a communication device (1) having an electronic circuit, the method comprising: obtaining (S1) location information (21) of the first user (2) of the communication device (1); transmitting (S2) from the communication device (1) a query request via a communication network (3) to a computerized virtual real estate directory system (4), the query request comprising at least the location information (21) of the first user (2); receiving (S3) in the communication device (1) from the virtual real estate directory system (4) via the communication network (3) a query response, the query response comprising one or more virtual real estate objects (41) assigned to a virtual real estate object location (42) within a predefined region (R) defined by the location information of the first user (2), each of the virtual real estate objects (41) comprising identification information (43) and a network address (44) of an augmented reality interaction server (5); transmitting (S5) from the communication device (1) for at least one particular virtual real estate object (41) an interaction request via the communication network (3) to the augmented reality interaction server (5), respectively using the network address (44) of the augmented reality interaction server (5), each interaction request comprising the identification information (43) of the particular virtual real estate object (41); receiving (S6) in the communication device (1) from the augmented reality interaction server via the communication network (3) access authorization for the particular virtual real estate object (41); transmitting (S7) from the communication device (1) first user data to a communication device (1B) of a second user (2B), associated with the particular virtual real estate object (41), via the augmented reality interaction server (5); and receiving (S8) in the communication device (1) second user data from the communication device (1B) of the second user (2B) via the augmented reality interaction server (5).
  10. The method of claim 9, wherein the method further comprises: transmitting, from the communication device (1), a public user identifier of the first user (2) to the augmented reality interaction server (5) comprising one or more of the following: a public name of the first user (2) or an encryption key of the first user (2); and receiving, in the communication device (1), from the augmented reality interaction server (5) a public user identifier of the second user (2B) comprising one or more of the following: a public name of the second user (2B) or an encryption key of the second user (2B).
  11. The method of one of claim 9 or 10, wherein the communication device (1) comprises one or more cameras (12) and a rendering system (11) and wherein the method further comprises: determining (S10) in the communication device (1) visual representation data of the first user (2) using one or more images of the first user (2) captured by the one or more cameras (12); transmitting (S11) from the communication device (1) of the first user (2) the visual representation data of the first user (2), included in the first user data, to the communication device (1B) of the second user (2B) via the augmented reality interaction server (5); receiving (S12) in the communication device (1) of the first user (2), from the communication device (1B) of the second user (2B), via the augmented reality interaction server (5), visual representation data of the second user (2B), included in the second user data; and rendering (S13), using the rendering system (11) and the visual representation data of the second user (2B), augmented reality content comprising a visual representation (54) of the second user (2B).
  12. The method of claim 11, wherein the method further comprises: determining, in the communication device (1), a field of view (F) of the first user (2) using the one or more cameras (12); determining, in the communication device (1), from the field of view (F) coordinates (C) of real-world features in the field of view (F) relative to a location (21); and rendering, using the rendering system (11), the augmented reality content such that the augmented reality content has a fixed position relative to the coordinates (C) of the real world features.
  13. The method of claim 11 or 12, wherein the method further comprises: verifying (S221), in the communication device (1), whether the augmented reality content (51) is entirely contained within a bounding volume (V) of the virtual real estate object (41); responsive to positive verification, rendering (S23) the augmented reality content (51); and responsive to negative verification either: not rendering (S232) the augmented reality content (51), or modifying (S222), in the communication device (1) the augmented reality content (51) and rendering (S231) modified augmented reality content fitting (52) within the bounding volume (V) of the virtual real estate object (41).
  14. The method of one of claims 9 to 13, wherein the communication device (1) further comprises a motion tracker (16) and wherein the method further comprises: recording, in the communication device (1) user motion data related to body motion of the first user (2), using the motion tracker (16); and transmitting (S8), from the communication device (1), the user motion data, included in the first user data, to the augmented reality interaction server (5).
  15. The method of one of claims 9 to 14, wherein the communication device (1) further comprises one or more microphones (15) and one or more speakers (14) and the method comprises: recording, in the communication device (1), audio data of the first user (2), using the one or more microphones (15); transmitting (S8), from the communication device (1), the audio data of the first user (2), included in the first user data, to the augmented reality interaction server (5); receiving (S8), in the communication device (1), from the communication device (1B) of the second user audio data of the second user (2B), included in the second user data; and playing, on the one or more speakers (14), the audio data of the second user (2B).
  16. The method of one of claims 9 to 15, the method further comprising: Receiving (S15), in the communication device (1) an augmented reality object (52) from the augmented reality interaction server (5); rendering (S16) augmented reality content comprising the augmented reality object (52) using the rendering system (11); determining (S17), in the communication device (1), user interaction data of the first user (2) interacting with the augmented reality object (51); transmitting (S18), from the communication device (1) the user interaction data to the augmented reality interaction server (5); receiving (S19), in the communication device (1) an updated augmented reality object (52) from the augmented reality interaction server (5); and rendering (S20) the augmented reality content comprising the updated augmented reality object (52) using the rendering system (11).
  17. A computer program product comprising a non-transitory computer-readable medium having stored thereon computer program code configured to control an electronic circuit (13) of a communication device (1) such that the communication device (1) performs the steps according to the method of one of claims 9 to 16.

Description

FIELD OF THE INVENTION The present invention relates to a device and method for providing augmented reality interaction. Specifically, the present invention relates to a device, a method, and a computer program product for providing augmented reality interaction. BACKGROUND OF THE INVENTION Augmented Reality (AR) is a technology for enhancing a real-world environment with virtual computer-generated information. As opposed to virtual reality (VR), AR does not replace the real-world with a computer-generated world, but adds computer-generated information to the real-world, often by overlaying virtual content onto real-world objects. The virtual content can encompass not only visual content (e.g. digital images and models) which are overlaid onto real-world objects, enhancing the visual experience of the real-world, but can also encompass auditory content, haptic content, olfactory content, and content involving other sensory modalities. Additionally, augmented reality can include representations of other users, enabling an interaction between users which goes beyond typical video or telephone conferencing by introducing spatial and contextual aspects to the representation and the interaction. Communication devices for playing back augmented reality content are plentiful and varied and undergoing constant development. Communication devices can be fixedly installed or carried or worn by a user. These communication devices include holographic projection systems, video conferencing systems, personal computers, as well as mobile communication devices. The communication devices can comprise various sensors and displays, as well as wearable modules, such as headsets with eyepieces, smart-glasses, or the like, which present augmented reality content to the user in addition to real world content. The communication devices may be supported by additional connected devices for augmented reality interaction, such as headphones for auditory content and gloves for providing precise motion tracking and haptic feedback. For ensuring the privacy of augmented reality interaction, it is essential a given user, or a given server, can control which other users can be interacted with. For large-scale augmented reality environments, in particular ones involving federated systems, it is not practical or desired to centrally control augmented reality interaction. In particular, this is because for a fluid and natural interaction between the users and the servers, a low latency and high data transmission rates connection becomes critical. US 2018/01 59838 A1 discloses an augmented reality system that includes an augmented reality user device. The augmented reality user device includes a display for overlaying virtual objects onto tangible objects in a real scene, a camera, and a global position system sensor. The augmented reality user device includes a processor implementing an object recognition engine, a virtual assessment engine, and a virtual overlay engine. The object recognition engine identifies features of a property. The virtual assessment engine generates a property profile based on the identified features and alternative features. The virtual assessment engine identifies the property based on the geographic location of the user, generates a property token that includes the property profile and the location identifier, sends the property token to a server and receives information related to the property, and presents the information as virtual objects overlaid with the real scene. WO 2020/025142 A1 discloses a way of provisioning applications, for example, augmented reality applications, to a user node automatically or at least semi-automatically based on location of the user node. The user node sends a request comprising location information of the user node to a network node. The network node has mapped geographical coordinates into a plurality of zones. Based on the request comprising the location information, the network node is able to identify one or more zones, and the network node transmits program content provider information associated with the one or more zones to the user node. SUMMARY OF THE INVENTION It is an object of this invention to provide a device and method for providing augmented reality interaction. Specifically, the present invention relates to a device, a method, and a computer program product for providing augmented reality interaction to a user of a communication device. According to the present invention, these objects are achieved through the features of the independent claims. In addition, further advantageous embodiments follow from the dependent claims and the description. According to the present invention, the above-mentioned objects are particularly achieved by a communication device, comprising an electronic circuit configured to obtain location information of a first user of the communication device. The location information can relate to a physical location, e.g. a geographic location de