CN-121482160-B - Visual image-based space positioning method in virtual reality environment

CN121482160BCN 121482160 BCN121482160 BCN 121482160BCN-121482160-B

Abstract

The invention provides a space positioning method based on visual images in a virtual reality environment, and relates to the technical field of mixed reality, wherein the method comprises the steps of identifying image information of at least three two-dimensional codes in the environment, comparing two-dimensional code information in a database with information of the two-dimensional codes to obtain corresponding two-dimensional code ID and three-dimensional world coordinates, and calculating a three-dimensional world plane based on the three-dimensional world coordinates of the corresponding two-dimensional codes; the method comprises the steps of obtaining a three-dimensional world plane normal vector by a three-dimensional world plane, constructing an initial virtual world space A based on a two-dimensional code and based on the three-dimensional world plane and the normal vector, and constructing a virtual world space B describing a large-range space environment based on the environmental point cloud data. The invention realizes high-precision and stable virtual space and real space alignment and reconstruction method.

Inventors

ZHANG YONGZHONG

Assignees

天津不菜科技有限公司

Dates

Publication Date: 20260508
Application Date: 20251120

Claims (10)

1. A method for spatial localization based on visual images in a virtual reality environment, the method comprising: Identifying image information of at least three two-dimensional codes in the environment, comparing the two-dimensional code information in a database with the two-dimensional code information to obtain a corresponding two-dimensional code ID and a three-dimensional world coordinate, and calculating a three-dimensional world plane based on the three-dimensional world coordinate of the corresponding two-dimensional code; Obtaining a normal vector of a three-dimensional world plane through the three-dimensional world plane, constructing an initial virtual world space A taking a two-dimensional code as a reference based on the three-dimensional world plane and the normal vector, and constructing a virtual world space B describing a large-scale space environment based on environment point cloud data; based on the virtual world space A and the virtual world space B, extracting key feature points of the two-dimensional code from the space A to obtain a sampling point matrix A, and extracting corresponding environment feature points from the space B to obtain a sampling point matrix B; Based on the sampling point matrix A and the sampling point matrix B, performing position comparison on the virtual world space A and the virtual world space B to obtain a translation vector A, and performing characteristic point orientation comparison on the sampling point matrix A and the sampling point matrix B to obtain a quaternion rotation A; correcting and reconstructing the virtual world space B through the translation vector A and the quaternion rotation A to obtain a new world matrix fusing the data alignment of the virtual world space A and the virtual world space B; the new world matrix is used as a unified space reference, and virtual objects in real application are mapped and positioned into the new world matrix.
2. The method for spatially locating a virtual reality environment based on visual images according to claim 1, wherein identifying image information of at least three two-dimensional codes in the environment, comparing two-dimensional code information in a database with information of the two-dimensional codes, obtaining corresponding two-dimensional code ID and three-dimensional world coordinates, and calculating a three-dimensional world plane based on the three-dimensional world coordinates of the corresponding two-dimensional codes, and scanning the surrounding environment through an environment perception sensor, and obtaining environment point cloud data, comprising: receiving multi-frame image data of at least three two-dimensional codes, decoding and identifying the multi-frame image data, and extracting two-dimensional vertex coordinates of each two-dimensional code in the image; comparing the extracted two-dimensional code information with a pre-stored two-dimensional code database to obtain a unique ID corresponding to each two-dimensional code and a pre-stored three-dimensional world coordinate in a global coordinate system; Calculating to obtain a fitting three-dimensional world plane containing all the two-dimensional codes through a computer vision algorithm based on pre-stored three-dimensional world coordinates of each two-dimensional code; And receiving the original environment data acquired by the environment sensing sensor, and processing the original environment data to obtain the environment point cloud data describing the surrounding environment geometric structure.
3. The visual image-based spatial localization method of claim 2, wherein the obtaining of the normal vector of the three-dimensional world plane from the three-dimensional world plane, the constructing of the initial virtual world space a based on the two-dimensional code and the normal vector, and the constructing of the virtual world space B describing the wide-range spatial environment based on the environmental point cloud data, comprises: Calculating a normal vector of the three-dimensional world plane based on the defined three-dimensional world plane; determining the orientation of a coordinate system by using the normal vector obtained by calculation through taking the defined three-dimensional world plane as a reference coordinate system plane, and establishing a virtual world space A taking a two-dimensional code as a space origin; Carrying out space structure analysis on the environmental point cloud data, and extracting key geometric features in the environment; based on the extracted key geometric features, a virtual world space B containing large-range environmental geometric information is constructed through a three-dimensional reconstruction algorithm.
4. The visual image-based spatial localization method in a virtual reality environment according to claim 3, wherein extracting key feature points of a two-dimensional code from a space a to obtain a sampling point matrix a and extracting corresponding environmental feature points from a space B to obtain a sampling point matrix B based on a virtual world space a and a virtual world space B comprises: Based on the established virtual world space A, four corner points of the two-dimensional code are selected as key feature points; Calculating three-dimensional coordinate values of each feature point in the virtual world space A according to the selected key feature points, combining the calculated three-dimensional coordinate values according to a preset sequence to obtain a sampling point matrix A, and positioning a space position corresponding to the selected key feature points in the virtual world space B based on the constructed virtual world space B; And extracting three-dimensional coordinate values of corresponding environmental feature points in the virtual world space B according to the positioned space positions, and combining the three-dimensional coordinate values of the extracted environmental feature points according to the same preset sequence to obtain a sampling point matrix B.
5. The visual image-based spatial localization method of claim 4, wherein the performing the position comparison of the virtual world space a and the virtual world space B to obtain the translation vector a and the feature point orientation comparison of the sampling point matrix a and the sampling point matrix B to obtain the quaternion rotation a based on the sampling point matrix a and the sampling point matrix B comprises: Based on the sampling point matrix A and the sampling point matrix B, respectively calculating the barycenter coordinates of all the characteristic points in the sampling point matrix A and the barycenter coordinates of all the characteristic points in the sampling point matrix B; Based on the coordinates of the mass centers, carrying out decentering treatment on the sampling point matrix A and the sampling point matrix B respectively to obtain a corresponding decentering characteristic point matrix A and a corresponding decentering characteristic point matrix B; According to the decentered feature point matrix A and the decentered feature point matrix B, a rotation matrix describing the rotation relationship between the two matrices is obtained through matrix calculation; And converting the rotation matrix into a unit quaternion form to obtain quaternion rotation A.
6. The visual image-based spatial localization method of claim 5, wherein the correcting and reconstructing the virtual world space B by the translation vector a and the quaternion rotation a to obtain a new world matrix fusing the data alignment of the virtual world space a and the virtual world space B comprises: Converting the quaternion rotation A into a corresponding three-dimensional space rotation matrix serving as a rotation transformation reference from space B to space A; performing rotation operation on all point cloud coordinates in the virtual world space B based on the rotation transformation reference, so that the coordinate system orientation of the virtual world space B is aligned with the coordinate system orientation of the virtual world space A; On the basis of completing the rotation transformation of the coordinate system, carrying out translation operation on all point cloud coordinates in the virtual world space B through the translation vector A, and completing the origin position alignment between the virtual world space B and the virtual world space A; and carrying out space data fusion processing on the virtual world space B after rotation and translation transformation, and binding a coordinate system of the virtual world space B and the virtual world space A to obtain a new world matrix of the accurate pose information of the two-dimensional code and the large-range environment geometric information.
7. The visual image based spatial localization method of claim 6, wherein mapping and localizing virtual objects in real applications into the new world matrix using the new world matrix as a unified spatial reference comprises: Receiving initial coordinate data of a virtual object to be positioned, and performing space transformation calculation on the initial coordinate data based on a new world matrix to obtain accurate world coordinates of the virtual object under a unified space reference; determining the final positioning position and the final positioning posture of the virtual object in the real space according to the accurate world coordinates; and returning the coordinate data containing the final positioning position and the gesture to the mixed reality application, and driving the virtual object to stably render and present at the final positioning position.
8. A visual image based spatial localization system in a virtual reality environment, the system implementing the method of any one of claims 1 to 7, comprising: The acquisition module is used for identifying the image information of at least three two-dimensional codes in the environment, comparing the two-dimensional code information in the database with the two-dimensional code information to obtain a corresponding two-dimensional code ID and a three-dimensional world coordinate, and calculating a three-dimensional world plane based on the three-dimensional world coordinate of the corresponding two-dimensional code; The building module is used for obtaining a normal vector of the three-dimensional world plane through the three-dimensional world plane, building an initial virtual world space A taking the two-dimensional code as a reference based on the three-dimensional world plane and the normal vector, and building a virtual world space B describing a large-range space environment based on environment point cloud data; the generating module is used for extracting key feature points of the two-dimensional code from the space A to obtain a sampling point matrix A based on the virtual world space A and the virtual world space B, and extracting corresponding environment feature points from the space B to obtain a sampling point matrix B; the computing module is used for comparing the positions of the virtual world space A and the virtual world space B based on the sampling point matrix A and the sampling point matrix B to obtain a translation vector A, and comparing the characteristic points of the sampling point matrix A and the characteristic points of the sampling point matrix B to obtain a quaternion rotation A; the fusion module is used for correcting and reconstructing the virtual world space B through the translation vector A and the quaternion rotation A to obtain a new world matrix for fusing the data alignment of the virtual world space A and the virtual world space B; and the mapping module is used for mapping and positioning the virtual object in the real application into the new world matrix by taking the new world matrix as a unified space reference.
9. A computing device, comprising: One or more processors; storage means for storing one or more programs which when executed by the one or more processors cause the one or more processors to implement the method of any of claims 1 to 7.
10. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a program which, when executed by a processor, implements the method according to any of claims 1 to 7.

Description

Visual image-based space positioning method in virtual reality environment Technical Field The invention relates to the technical field of mixed reality, in particular to a space positioning method based on visual images in a virtual reality environment. Background The application of the current virtual reality and mixed reality technology in large-space immersive scenes is increasingly wide, such as indoor VR interactive exhibition halls, immersive training and other scenes, the accurate alignment of virtual content and real space is realized by relying on visual positioning technology, a certain indoor VR interactive experience scene is taken as an example, the scene is required to be in an indoor area, a user wears VR equipment to roam and interact with virtual exhibits in the virtual scene, a mainstream Inside-Out visual positioning scheme is adopted, environmental images are acquired through a front RGB camera of the VR equipment, an environmental map is built in real time by combining with SLAM algorithm, and positioning is realized, so that the alignment of the virtual exhibits and a preset interactive triggering area in real space is ensured. However, in the practical application process, the scheme has the following technical defects: As the roaming time length of the user increases, when image feature matching and pose calculation are continuously performed by the SLAM algorithm, tiny positioning errors are continuously accumulated, so that virtual exhibits which are accurately aligned with a real interaction triggering area are gradually offset, when the user walks to a preset triggering area in reality, the virtual exhibits in a VR visual angle are offset to a non-interaction area in real space, the user cannot normally trigger an interaction function, and even if the positioning is restarted through a positioning calibration function carried by the device, after the same roaming time length, offset problems caused by accumulated errors still occur repeatedly, and the immersion feeling and the function usability of a scene are affected. Disclosure of Invention The invention aims to solve the technical problem of providing a space positioning method based on visual images in a virtual reality environment, and a method for realizing high-precision and stable alignment and reconstruction of virtual space and real space. In order to solve the technical problems, the technical scheme of the invention is as follows: In a first aspect, a method for spatial localization based on visual images in a virtual reality environment, the method comprising: Identifying image information of at least three two-dimensional codes in the environment, comparing the two-dimensional code information in a database with the two-dimensional code information to obtain a corresponding two-dimensional code ID and a three-dimensional world coordinate, and calculating a three-dimensional world plane based on the three-dimensional world coordinate of the corresponding two-dimensional code; Obtaining a normal vector of a three-dimensional world plane through the three-dimensional world plane, constructing an initial virtual world space A taking a two-dimensional code as a reference based on the three-dimensional world plane and the normal vector, and constructing a virtual world space B describing a large-scale space environment based on environment point cloud data; based on the virtual world space A and the virtual world space B, extracting key feature points of the two-dimensional code from the space A to obtain a sampling point matrix A, and extracting corresponding environment feature points from the space B to obtain a sampling point matrix B; Based on the sampling point matrix A and the sampling point matrix B, performing position comparison on the virtual world space A and the virtual world space B to obtain a translation vector A, and performing characteristic point orientation comparison on the sampling point matrix A and the sampling point matrix B to obtain a quaternion rotation A; correcting and reconstructing the virtual world space B through the translation vector A and the quaternion rotation A to obtain a new world matrix fusing the data alignment of the virtual world space A and the virtual world space B; the new world matrix is used as a unified space reference, and virtual objects in real application are mapped and positioned into the new world matrix. Further, identifying image information of at least three two-dimensional codes in the environment, comparing two-dimensional code information in a database with information of the two-dimensional codes to obtain corresponding two-dimensional code ID and three-dimensional world coordinates, and calculating a three-dimensional world plane based on the three-dimensional world coordinates of the corresponding two-dimensional codes, and scanning the surrounding environment through an environment perception sensor to obtain environment point cloud data, wherein the me