Search

DE-102024132896-A1 - Method for overlaying an environment with a virtual environment, augmented reality glasses and motor vehicle

DE102024132896A1DE 102024132896 A1DE102024132896 A1DE 102024132896A1DE-102024132896-A1

Abstract

The invention relates to a method for superimposing a real-world environment (14) with a virtual environment displayed on augmented reality glasses (26) worn by a user. For this purpose, based on a similarity criterion, first image data describing a current area (12) are compared with a set (22) of second image data (24), wherein the second image data (24) each comprise at least one street segment and show several buildings (16) in different geographical areas, and a second area is determined. For each of several current buildings (16, 163) in the current area (12), at least one second building (16, 165) in the second area is assigned. When one of the current buildings (16, 163) is detected, a visual representation of at least one second building (16, 165) is extracted and superimposed on the augmented reality glasses (26) in the projection area (30).

Inventors

  • Hendrik Gronau
  • Paul Böckenhoff

Assignees

  • AUDI AKTIENGESELLSCHAFT

Dates

Publication Date
20260513
Application Date
20241111

Claims (15)

  1. A method for superimposing an environment (14) with a virtual environment displayed on a user-worn augmented reality glasses (26), characterized in that a. based on a predefinable similarity criterion relating to at least one street and/or at least one building (16) and b. based on a set (22) of second image data (24) stored in a memory, wherein the second image data (24) each comprise at least one street and show several buildings (16) in different geographical areas, c. to a predefinable current area (12) that relates to an area extending beyond an environment (14) detectable by environmental sensors, a second area is determined from the set (22) of second image data (24) by comparing the first image data describing the current area (12) with the second image data (24), d. several current buildings (16, 163) in the current area (12) are each assigned at least one second building (16, 165) in the second area, e. and when one of the current buildings (16, 163) is detected, f. a direction of view of the user and a position of the detected current building (16, 169) relative to a head position of the user are determined, g. from the position of the detected current building (16, 169) and the direction of view of the user, a projection area (30) on the augmented reality glasses (26) is determined, through which the user perceives exactly the detected current building (16, 169), and h. a visual representation of at least one second building (16, 165), which is extracted from the second image data (24, 241) assigned to the second area and which is assigned to the captured current building (16, 169), is displayed superimposed on the augmented reality glasses (26) in the projection area (30).
  2. Procedure according to Claim 1 , wherein the similarity criterion includes at least one of the following criteria: a. the structures (16, 165) in the second area are on average at least as high as the structures (16, 163) in the predefinable current area (12), b. at least one parameter describing a road alignment in the second area lies within a tolerance range of a parameter describing a road alignment of the current area (12), c. an average number of structures (16) between two successive intersections in the second area is at least as large as in the current area (12), d. a density of structures in the second area is at least as large as a density of structures (16) in the current area (12).
  3. Method according to one of the preceding claims, wherein a relative arrangement of the second structures (16, 165) in the second area corresponds to a relative arrangement of the current structures (16, 163) assigned to the respective second structures (16, 165) in the current area (12).
  4. Method according to one of the preceding claims, wherein two adjacent current structures (16, 163) of the current area (12) are covered by visual representations of second structures (16, 165) adjacent in the second area.
  5. Method according to one of the preceding claims, wherein the actual structure (16, 163) is completely overlaid by the visual representation.
  6. Method according to one of the preceding claims, wherein in all directions perpendicular to the vertical direction of the current structure (16, 163) only the current structure (16, 163) is superimposed with the visual representation of the at least one second structure (16, 165) associated with the current structure (16, 163).
  7. Method according to one of the preceding claims, wherein the second area is located in a specified country and/or in a specified city.
  8. A method according to any of the preceding claims, wherein a route is specified which comprises several route segments, wherein the user moves successively along the route segments and the route segments each represent the current area (12), and wherein the similarity criterion comprises at least one of the following criteria: a. the number of current structures (16, 163) located in the route segment along one side of the road is at most as large as the number of second structures (16, 165) on one side of the road in the respective route segment in the second area, b. the number of structures (16) that fulfill a predefinable purpose in the current area (12) is at most as large as in the second area, c. each current intersection of the current area (12) is assigned a second intersection in the second area assigned, wherein from the respective intersection in the current area (12) at most as many streets branch off as from the respective assigned intersection in the second area, i.e. a current size ratio of two adjacent current structures (16, 163) corresponds to a second size ratio of two adjacent second structures (16, 165) which are assigned to the respective current structures (16, 163).
  9. Method according to one of the preceding claims, wherein the visual representation is semi-transparent, so that the captured current structure (16, 163) remains visible through the superimposed visual representation.
  10. Method according to one of the preceding claims, wherein at least one camera is used to determine how much light and from which direction the light hits the recorded current building (16, 163) and, in the visual representation of the at least one second building (16, 165) associated with the recorded current building (16, 163), the same lighting conditions as for the respective current building (16, 163) are recreated.
  11. Method according to one of the preceding claims, wherein the second structures (16, 165) are structures (16) from the second area at a past time.
  12. Method according to one of the preceding claims, wherein currently detected vehicles are superimposed by second vehicles that are customary for the second area and/or for the past time from which the second structures (16, 165) originate.
  13. Method according to one of the preceding claims, wherein roads (18) and/or traffic infrastructure objects and/or road users and/or moving objects are not superimposed and/or are visible without superimposition.
  14. Augmented reality glasses (26) comprising at least one memory on which image data showing several buildings (16) in each area are stored, and at least one environmental sensor, wherein the augmented reality glasses (26) are configured to perform a method according to one of the preceding claims.
  15. Motor vehicle comprising at least a storage device on which image data showing several buildings (16) in a given area are stored, and at least one augmented reality headset (26), wherein the motor vehicle is equipped to perform a procedure according to one of the Claims 1 until 13 to carry out.

Description

The invention relates to a method for overlaying an environment with a virtual environment, augmented reality glasses and a motor vehicle. Augmented reality glasses (AR glasses), as known from the prior art, are wearable electronic devices. AR glasses can be worn by a user, and digital information can be displayed on the glasses, projected into the user's field of vision, thereby overlaying the real environment. This means, in particular, that a user wearing AR glasses perceives and/or can perceive the digital information displayed via the AR glasses in addition to the real environment, especially visually. This overlay can mean that the real environment is displayed in a background against which the digital information is shown, with both the digital information and the real environment visible and/or perceptible in the background of the digital information. The real environment refers in particular to the user's environment that they can perceive without AR glasses. To display the digital information, the AR glasses can, as is known from the state of the art, include at least a transparent display which can be arranged in the user's field of vision. An example of displaying digital information via augmented reality, for example using AR glasses, is in the EP 2 911 041 A1 This publication describes a method in which an image is generated and displayed using augmented reality to allow the user to see through obstacles in an environment. The user is thus shown a point of interest located behind an obstacle. A disadvantage of the method disclosed in the publication is that a very large number of images must be stored so that at least one image is available that depicts the point of interest from the respective perspective, but without the obstacle in the image. The printed text also describes an overlay of the environment, for example with AR glasses. DE 11 2018 001 422 T5 The publication discloses a method in which a relevance factor is determined between a vehicle's current route and a route associated with a current video, wherein one or more video parameters of the selected video are adjusted based on the determined relevance factor. The display of the at least one selected video is then controlled according to the adjusted video parameters. A disadvantage of the method described in the publication is that a video is played, which means that spontaneous changes to a route, such as a spontaneous turn at an intersection not dictated by the route, cannot be implemented, or only with considerable effort, in the video display. The object of the invention is to provide a method in which an environment is superimposed using AR glasses, whereby the superimposition can be adapted to a current driving situation. The problem is solved by the subject matter of the independent patent claims. Advantageous embodiments of the invention are described by the dependent patent claims, the following description, and the figures. The invention provides a solution comprising a method for superimposing a real-world environment with a virtual environment displayed on augmented reality glasses (hereinafter also referred to as AR glasses) worn by a user. The real-world environment is, in particular, the environment that a user can perceive without using AR glasses, and/or an environment surrounding the user in reality. The method comprises determining a second area based on a predefinable similarity criterion concerning at least one street and/or at least one building and based on a set of second image data stored in a memory, wherein the second image data each show at least one street comprising several buildings in different geographical areas, by comparing the first image data describing the current area (also referred to as current image data) with the second image data from the set of second image data. This means, in particular, that the second area is determined for the predefined current area. The determination of the second area can be based on the predefined similarity criterion. The similarity criterion allows for the comparison of at least one property, in particular a numerical value of the at least one property, of the predefined current area with at least one property and/or corresponding property, in particular a numerical value of the at least one property and/or corresponding property, of the second area, thereby determining how similar these properties and/or numerical values are. The property preferably relates to at least one street and/or at least one building in the respective area, i.e., in the predefined current area and/or in the second area. In other words, a property of at least one street and/or at least one building of the predefined current area can be compared with the property and/or corresponding property of at least one street and/or at least one building of the second area and/or examined for similarity. This can include assigning a numerical value to each of the at least one property of the at l