CN-122015805-A - Offline self-adaptive navigation method and system based on AR (augmented reality) glasses

CN122015805ACN 122015805 ACN122015805 ACN 122015805ACN-122015805-A

Abstract

The application relates to the technical field of travel navigation, in particular to an off-line self-adaptive navigation method and system based on AR (augmented reality) glasses, wherein the method comprises the steps of presetting an off-line map data packet on the local of the AR glasses; the method comprises the steps of starting AR glasses, providing scenic spot overview for users based on data packets and receiving scenic spot selection input of the scenic spot overview, planning an optimal tour path in real time on the basis of the selected scenic spots and the communication relation, carrying out positioning tracking through vision-inertial odometers of the AR glasses, rendering the optimal tour path on the real environment surface in the field of view of the users in real time in an augmented reality guiding mode to navigate, re-planning the path in real time and updating the augmented reality guiding in response to path modification instructions in the navigation process, and correcting the positioning result of the vision-inertial odometers through local beacon signals at preset correction points. And the navigation accuracy is improved by combining the positioning of the vision-inertial odometer and the correction of the local beacon.

Inventors

SONG YUCHAO
Jing Haoxin
WANG YUANQIANG
CHEN MENG

Assignees

浪潮智慧科技有限公司

Dates

Publication Date: 20260512
Application Date: 20260129

Claims (10)

1. An off-line self-adaptive navigation method based on AR glasses is characterized by comprising the following steps: Presetting an offline map data packet on the local of the AR glasses, wherein the offline map data packet comprises spatial data and communication relations of a plurality of scenic spots; after the AR glasses are started, providing scenic spot overview for the user based on the data packet and receiving scenic spot selection input of the user; Based on the selected scenic spots and the communication relation, planning an optimal tour path in real time on the local area of the AR glasses; Positioning and tracking through a visual-inertial odometer of the AR glasses, and rendering the optimal tour path on a real environment surface in the field of view of a user in real time in an augmented reality guiding mode for navigation; in the navigation process, responding to a path modification instruction of a user, and immediately re-planning a path locally and updating the augmented reality guide; and correcting the positioning result of the vision-inertial odometer through a local beacon signal at a preset correction point.
2. The AR glasses-based offline adaptive navigation method according to claim 1, wherein the offline map data packet comprises: A structured file for describing the coordinates of the scenic spots and the connected graph; An image file for scenic spot preview; and an audio file for speech interpretation.
3. The AR glasses-based offline adaptive navigation method according to claim 1, wherein the specific step of providing the user with the scenic spot overview based on the data packet comprises: Analyzing the offline map data packet to obtain preview information of all scenic spots; generating and displaying a panoramic visualization interface containing all scenic spot thumbnails on the optical waveguide display of the AR glasses; simultaneously, broadcasting a voice summary containing the total number of scenic spots and the estimated total tour time through the bone conduction earphone.
4. The AR glasses-based offline adaptive navigation method according to claim 1, wherein the step of receiving the sight selection input thereof is implemented by at least one of: The method comprises the steps of A, tracking head movement of a user, controlling a selection cursor to move in a visual interface and aim at a thumbnail of a target scenic spot, and completing selection through a preset confirmation action; capturing scenic spot names or category keywords spoken by a user through a local voice recognition module, matching the recognition result with scenic spot information in a data packet, and adding the scenic spots successfully matched into a selection list; And C, capturing the stay position of the sight focus of the user in the visual interface through the eye tracking module, and automatically completing the selection of the scenic spot where the focus is located when the stay time exceeds a preset threshold value.
5. The AR glasses-based offline adaptive navigation method according to claim 4, wherein when the mode B is adopted, the method further comprises: And scoring the interest degree of the scenery spots according to the predefined tag library and the user portrait, and automatically recommending one or more tour routes consisting of the scenery spots with high interest degree to the user for the user to confirm.
6. The offline adaptive navigation method based on AR glasses according to claim 4, wherein the step of planning the optimal tour path locally and in real time on the AR glasses based on the selected scenic spots and the connection relation specifically comprises: modeling the plurality of attractions and the communication relation therebetween as a weighted undirected graph, wherein the attractions are nodes of the graph, the communication paths are edges of the graph, and the weights of the edges are determined based on the actual distance or the estimated transit time of the paths; And in the weighted undirected graph, the current position or the first selected scenic spot of the user is taken as a starting point, the last selected scenic spot or the last selected scenic spot is taken as an ending point, and a graph searching algorithm is adopted to solve a shortest path accessing all the selected scenic spots to be used as the optimal tour path.
7. The AR glasses-based offline adaptive navigation method according to claim 4, wherein the steps of performing location tracking through a vision-inertial odometer of the AR glasses and performing navigation by rendering the optimal tour path in real time on a real environment surface in a user's field of view in the form of augmented reality directions comprise: the method comprises the steps that through a camera and an inertial measurement unit integrated by the AR glasses, pose calculation between continuous frames is carried out through a vision-inertial mileage calculation method, and six-degree-of-freedom pose information of the AR glasses in an environment map is output in real time; converting the next travelling direction of the optimal tour path into a three-dimensional virtual guide model, wherein the virtual guide model is in the form of an arrow, a footprint or a light band attached to the ground; and aligning the space coordinates of the virtual guide model with a plane in the real environment determined by the current pose information output by the vision-inertial odometer, and performing fusion rendering through the optical waveguide display module to enable the virtual guide model to be attached to the surface of the real environment.
8. The AR glasses-based offline adaptive navigation method according to claim 7, wherein the steps of re-planning a path and updating the augmented reality instruction in real time locally in response to a path modification instruction of a user are specifically as follows: when an instruction for adding or deleting scenic spots is received, a heuristic optimization algorithm is adopted to conduct local rearrangement on the path sequence based on the current planned path and the communication relation, so that a new optimal tour path is generated, and augmented reality guide rendered on the surface of the real environment is updated immediately.
9. The AR glasses-based offline adaptive navigation method according to claim 1, wherein the step of correcting the positioning result of the vision-inertial odometer by a local beacon signal at a preset correction point specifically comprises: At an entrance or a path key node of a scenic spot, laying a low-power consumption Bluetooth beacon, wherein the beacon continuously broadcasts a beacon signal containing a unique ID of the beacon and preset three-dimensional space coordinates; when the AR glasses move to the coverage range of the effective signals of the beacons, the integrated Bluetooth module receives and analyzes the beacon signals to acquire ID and coordinate information of the beacons; the AR glasses compare the received beacon coordinates with the current self pose calculated through the vision-inertial mileage calculation, and a pose deviation is calculated; and carrying out one-time coordinate translation and rotation correction on the current pose output by the vision-inertial odometer by taking the beacon coordinates as an absolute reference, and applying the correction to a subsequent vision-inertial odometer calculation process to reset the accumulated error.
10. An AR glasses-based offline adaptive navigation system, characterized in that it is configured to implement the method of any one of claims 1-9, where the system includes an AR glasses terminal and a plurality of positioning correction beacons disposed at preset correction points of a scenic spot, for broadcasting the local beacon signal to the AR glasses terminal, and where the AR glasses terminal is integrated with: A memory for pre-storing the offline map data packet; The processing unit is connected with the memory and used for executing scenic spot selection receiving, path planning, re-planning and rendering control; the vision-inertial odometer module is used for executing positioning tracking and outputting pose information; the optical waveguide display module is used for receiving the rendering instruction of the processing unit and presenting the augmented reality guide to the field of view of the user; The audio output device is used for broadcasting the voice summary and the explanation; the input sensing module is used for capturing interaction instructions of users; a wireless communication module for receiving the local beacon signal; the processing unit is configured to model the plurality of scenic spots and the communication relation between the scenic spots as a weighted undirected graph and solve the shortest path accessing all the selected scenic spots by adopting a graph searching algorithm when executing path planning; the processing unit is configured to convert the next travelling direction of the planned path into a three-dimensional virtual guide model, align the space coordinates of the virtual guide model with a real environment plane determined by the current pose information and perform fusion rendering through the optical waveguide display module; The processing unit is configured to locally reorder the current path sequence by adopting a heuristic optimization algorithm when responding to the path modification instruction so as to generate a new optimal tour path and control the optical waveguide display module to immediately update the augmented reality guide.

Description

Offline self-adaptive navigation method and system based on AR (augmented reality) glasses Technical Field The application relates to the technical field of travel navigation, in particular to an off-line self-adaptive navigation method and system based on AR (augmented reality) glasses. Background In the prior art, there are significant limitations to scenic spot oriented augmented reality navigation solutions. Most schemes rely on mobile terminals such as mobile phones, users need to frequently operate the screen to view maps or switch applications, the sightseeing immersion sense is seriously interrupted, and the screen visibility is poor under outdoor strong light. The trigger mode of traditional AR navigation is passive, often can trigger the explanation only when the user walks to the fixed sign thing of scenic spot, and the visitor is difficult to establish whole cognition to garden overall arrangement and tour line in advance. The route planning function is generally weak, and generally only a fixed recommended route pre-stored in a server is provided, so that dynamic adjustment and personalized re-planning cannot be performed according to the personal physical strength, real-time interest or temporarily changed requirements of tourists. In the aspect of positioning, under complex environments such as forest areas, building groups and the like, a positioning scheme which is simply dependent on GPS or Wi-Fi has serious signal drift, so that AR (augmented reality) guidance is inaccurate to be overlapped, and a user is misled. Furthermore, existing solutions are highly dependent on a continuous, stable mobile network connection. When the network coverage is poor or the network is congested in the peak period of people flow, map data downloading, cloud path planning and voice explanation service are interrupted, so that the navigation experience is completely disabled. For the old people, children and other groups unfamiliar with the intelligent equipment operation, complex questionnaire filling, account login and multilevel menu operation also form a higher use threshold. Therefore, an AR navigation solution that can operate offline, provide active panoramic preview, support natural interaction and dynamic route planning, and have stable high-precision positioning capability is a technical problem to be solved. Disclosure of Invention In order to solve the problems, the invention provides an off-line self-adaptive navigation method and system based on AR glasses, which fundamentally eliminates the dependence on a mobile network by presetting an off-line data packet and ensures the navigation continuity in the case of network dead zone or congestion. And by combining VIO positioning, AR real-time rendering and local beacon correction, stable navigation in a complex environment is realized. In a first aspect, the present invention provides an offline adaptive navigation method based on AR glasses, including the following steps: Presetting an offline map data packet on the local of the AR glasses, wherein the offline map data packet comprises spatial data and communication relations of a plurality of scenic spots; after the AR glasses are started, providing scenic spot overview for the user based on the data packet and receiving scenic spot selection input of the user; Based on the selected scenic spots and the communication relation, planning an optimal tour path in real time on the local area of the AR glasses; Positioning and tracking through a visual-inertial odometer of the AR glasses, and rendering the optimal tour path on a real environment surface in the field of view of a user in real time in an augmented reality guiding mode for navigation; in the navigation process, responding to a path modification instruction of a user, and immediately re-planning a path locally and updating the augmented reality guide; and correcting the positioning result of the vision-inertial odometer through a local beacon signal at a preset correction point. The active interaction mode of firstly overview and then self-selection breaks the passive triggering limitation of the traditional navigation, and supports the instant path re-planning of dynamic instructions, so that the high flexibility and individuation of the route are provided, and the autonomy, fluency and immersion of the navigation are improved. As a preferred aspect of the present invention, the offline map data packet includes: A structured file for describing the coordinates of the scenic spots and the connected graph; An image file for scenic spot preview; and an audio file for speech interpretation. By adopting the optimal design of separately storing the structured file, the image and the audio file for the offline map data packet, the manufacture, the update and the management of the data packet are greatly facilitated. The modularized structure ensures that the adding, deleting, modifying and checking of scenic spot information is more efficient, si