CN-121999188-A - Terminal AR identification system and method

CN121999188ACN 121999188 ACN121999188 ACN 121999188ACN-121999188-A

Abstract

The invention provides a terminal AR identification system and a method, and relates to the technical field of augmented reality, wherein the system comprises a terminal layer, an edge layer and a cloud layer which are sequentially connected; the terminal layer is used for collecting images, carrying out light feature extraction and local matching and rendering augmented reality contents, the edge layer is used for caching feature data, preloading spatial proximity, mining and prefetching association rules and regional matching retrieval, and the cloud layer is used for storing, managing and deeply matching retrieval of global feature data and providing model training and updating services. The method and the device realize high-efficiency real-time identification and high identification accuracy by combining the lightweight feature extraction and local matching of the terminal layer with regional matching retrieval of the edge layer and depth matching retrieval of the cloud layer, and realize multi-scene adaptability by strategies such as spatial proximity preloading and association rule mining prefetching of the edge layer, so that the efficiency, accuracy and scene adaptability of terminal AR identification are improved.

Inventors

WANG WEIJIA
SUN HAORAN
LI SHIJIE

Assignees

中电信数字城市科技有限公司

Dates

Publication Date: 20260508
Application Date: 20260325

Claims (10)

1. The terminal AR identification system is characterized by comprising a terminal layer, an edge layer and a cloud layer which are connected in sequence; the terminal layer is deployed on the mobile terminal equipment and is used for collecting images, carrying out light feature extraction and local matching and rendering augmented reality content; The edge layer is composed of an edge server cluster deployed in a physical space adjacent area and is used for caching feature data, preloading the space adjacent area, mining and prefetching association rules and regional matching retrieval, wherein the preloading of the space adjacent area refers to the prefetching of the feature data of a user space adjacent area from the cloud layer, the mining and prefetching of the association rules refers to the construction of a feature association map based on user history interaction behaviors, and the prefetching of feature combinations with high association degree; the cloud layer is composed of a distributed server cluster, and is used for storing, managing and deep matching retrieval of global feature data and providing model training and updating services.
2. The terminal AR identification system according to claim 1, wherein the terminal layer comprises: The lightweight feature extraction module is used for running an ORB algorithm or a lightweight CNN model and extracting feature vectors from an input image; The local storage module is used for storing high-frequency characteristic data with access frequency reaching a preset first threshold by adopting an SQLite database and combining with an LRU cache strategy; and the sensor auxiliary unit is used for acquiring the motion state and the geographic position information of the data equipment and providing data support for the spatial proximity preloading.
3. The terminal AR identification system according to claim 1, wherein the edge layer comprises: the spatial proximity preloading module is used for pre-pulling characteristic data of a target area from the cloud layer according to the position information and the movement trend prediction information of the user; the association rule mining and prefetching module is used for analyzing the history interaction log of the user, constructing a characteristic association map and prefetching the characteristic combination with high association degree; The delay grabbing module is used for acquiring low-frequency characteristics with the access frequency lower than a preset second threshold value from the cloud layer only when the request of the terminal layer is received; And the cache optimization module is used for executing a cache elimination strategy combining the characteristic access frequency and the spatial correlation and maintaining the consistency with the cloud layer data.
4. The terminal AR identification system according to claim 1, wherein the cloud layer comprises: The distributed feature database adopts a data slicing and multi-copy redundant storage mode and is used for supporting the storage and depth matching retrieval of feature vectors; The message queue and the synchronous service are used for realizing asynchronous transmission of feature update based on the Kafka message queue and broadcasting feature change events of the terminal layer and the edge layer to a global node through the Kafka; and the machine learning service is used for performing incremental training on the feature extraction model based on the newly added feature data at regular intervals and synchronizing the updated lightweight model to the terminal layer.
5. The terminal AR identification system according to claim 1, wherein the feature data stored in the terminal layer, the edge layer and the cloud layer comprise metadata and state identifiers of features, the metadata comprise a locally unique feature identifier, a globally unique cloud identifier and a version number of the terminal, the metadata are used for achieving data synchronization of the terminal layer, the edge layer and the cloud layer, and the state identifiers are used for indicating storage states and/or synchronization states of the corresponding features.
6. The terminal AR identification system according to claim 5, characterized in that said status identification comprises the following three categories: local_only indicates that the corresponding feature is ONLY stored locally at the terminal and is not synchronized to the cloud layer; cloud_ SYNCED, indicating that the corresponding feature has been synchronized to the CLOUD layer; INVALID indicates that the corresponding feature has failed.
7. The terminal AR identification system according to claim 1, wherein when feature data is transmitted among the terminal layer, the edge layer and the cloud layer, data encryption is performed and an encryption transmission mode is adopted; the invisible watermark of the equipment identifier and the timestamp is embedded in the characteristic data; the cloud layer controls the authority of the user to access different feature spaces based on the RBAC model.
8. A terminal AR identification method, characterized in that it is applied to the terminal AR identification system according to any one of claims 1 to 7, and comprises: after preprocessing an input image, the terminal of the terminal layer performs light feature extraction and local matching, and when the local matching fails, the extracted feature vector is uploaded to an edge node of the edge layer; The edge node performs regional matching search on the received feature vector, and uploads the feature vector to the cloud layer when the feature vector is missed; the cloud layer carries out depth matching search on the feature vectors, returns identification results and associated AR content, and synchronously updates the identification results and the associated AR content to the cache of the edge layer; After the terminal acquires the identification result and the associated AR content, the terminal adopts a motion tracking technology to combine the space coordinate information contained in the feature vector to determine and render the space pose of the AR content so as to realize the physical interaction effect.
9. The terminal AR identification method according to claim 8, characterized in that the terminal AR identification method further comprises: when the terminal registers for the first time, the local feature list is associated and synchronized with the cloud layer; When the characteristics change, the terminal pushes the change event to the cloud layer in real time, and the cloud layer broadcasts the change event to other nodes in the terminal layer and the edge layer through a message queue to perform data synchronization; And after the terminal recovers network connection from an offline state, uploading a change event locally recorded during offline to the cloud layer in sequence, and performing conflict detection, data updating and data synchronization by the cloud layer.
10. The terminal AR identification method according to claim 8, characterized in that the terminal AR identification method further comprises: The edge node of the edge layer predicts a target area moved by a user according to the position information and the movement track prediction information reported by the terminal, and pulls feature data corresponding to the target area from the cloud layer to cache; The edge node analyzes a historical interaction log of a user, builds a correlation graph among features, and prefetches relevant feature data for caching according to a high-correlation feature combination mined in the correlation graph; The edge node calculates the priority of the feature data based on the feature access frequency and the spatial correlation with the current position of the user, and performs elimination of the feature data based on the priority; And the edge node regularly compares the version number of the characteristic data with the cloud layer, updates the data marked as out-of-date and triggers incremental synchronization.

Description

Terminal AR identification system and method Technical Field The invention relates to the technical field of augmented reality, in particular to a terminal AR identification system and method. Background Augmented reality (Augmented Reality, abbreviated as AR) is a technology of fusing virtual information (such as 3D model and animation) with a real scene in real time, and allows a user to see an "augmented" real picture through a device such as a mobile phone, for example, a virtual introduction of an overlay exhibit in a museum tour. With the development of mobile internet, computer vision and sensor technology, mobile AR applications are rapidly spreading in various scenes due to portability and universality. However, the existing mobile phone AR identification system has yet to be improved in terms of efficiency, accuracy and scene adaptability. Disclosure of Invention The invention aims to provide a terminal AR identification system and a terminal AR identification method so as to improve the efficiency, accuracy and scene adaptability of terminal AR identification. In a first aspect, the invention provides a terminal AR identification system, which comprises a terminal layer, an edge layer and a cloud layer which are sequentially connected; The terminal layer is deployed on the mobile terminal equipment and is used for collecting images, carrying out light-weight feature extraction and local matching and rendering augmented reality contents; The system comprises an edge layer, an association rule mining pre-fetching layer and an association rule mining pre-fetching layer, wherein the edge layer is formed by an edge server cluster deployed in a physical space adjacent area and is used for caching feature data, carrying out space adjacent pre-loading, carrying out association rule mining pre-fetching and regional matching retrieval, wherein the space adjacent pre-loading refers to pre-fetching the feature data of a user space adjacent area from a cloud layer; The cloud layer is composed of a distributed server cluster, and is used for storing, managing and deep matching retrieval of global feature data and providing model training and updating services. In an alternative embodiment, the terminal layer includes: The lightweight feature extraction module is used for running an ORB algorithm or a lightweight CNN model and extracting feature vectors from an input image; The local storage module is used for storing high-frequency characteristic data with access frequency reaching a preset first threshold by adopting an SQLite database and combining with an LRU cache strategy; And the sensor auxiliary unit is used for acquiring the motion state and the geographic position information of the data equipment and providing data support for the spatial proximity preloading. In an alternative embodiment, the edge layer includes: The spatial proximity preloading module is used for pre-pulling characteristic data of the target area from the cloud layer according to the position information and the movement trend prediction information of the user; the association rule mining and prefetching module is used for analyzing the history interaction log of the user, constructing a characteristic association map and prefetching the characteristic combination with high association degree; the delay grabbing module is used for acquiring low-frequency characteristics with the access frequency lower than a preset second threshold value from the cloud layer only when a request of the terminal layer is received; And the cache optimization module is used for executing a cache elimination strategy combining the characteristic access frequency and the spatial correlation and maintaining the consistency with cloud layer data. In an alternative embodiment, the cloud layer includes: The distributed feature database adopts a data slicing and multi-copy redundant storage mode and is used for supporting the storage and depth matching retrieval of feature vectors; The message queue and the synchronous service are used for realizing asynchronous transmission of feature update based on the Kafka message queue and broadcasting feature change events of the terminal layer and the edge layer to the global node through the Kafka; and the machine learning service is used for performing incremental training on the feature extraction model based on the newly added feature data at regular intervals and synchronizing the updated lightweight model to the terminal layer. In an optional embodiment, the feature data stored in the terminal layer, the edge layer and the cloud layer all comprise metadata and state identifiers of the features, the metadata comprise a locally unique feature identifier of the terminal, a globally unique cloud identifier and a version number, the metadata are used for achieving data synchronization of the terminal layer, the edge layer and the cloud layer, and the state identifiers are used for indicating storage states and/or sy