KR-102961705-B1 - DEVICE, METHOD AND COMPUTER PROGRAM FOR ADJUSTING STRUCTURES OF SPACE

KR102961705B1KR 102961705 B1KR102961705 B1KR 102961705B1KR-102961705-B1

Abstract

A device for adjusting a spatial structure may include a floor plan generating unit that generates a user input floor plan containing a plurality of actual sub-regions, a virtual space generating unit that extracts feature point information for a plurality of walls from a real space image captured of the real space and generates a virtual space for the real space based on the feature point information, and an adjustment unit that adjusts a plurality of virtual sub-regions included in the virtual space based on the user input floor plan.

Inventors

이유
박종경
윤여진
이규철
이주철
이철희

Assignees

주식회사 케이티

Dates

Publication Date: 20260507
Application Date: 20210125

Claims (19)

In a device for adjusting spatial structure, A floor plan generation unit that generates a user input floor plan including multiple actual sub-regions; A virtual space generation unit that extracts feature point information for a plurality of walls from a real space image captured of a real space and generates a virtual space for the real space based on the feature point information; and An adjustment unit that adjusts a plurality of virtual sub-regions included in the virtual space based on the above user input plan; Includes, A spatial structure adjustment device wherein the adjustment unit analyzes the correspondence between a plurality of actual sub-regions defined in the user input plan and a plurality of virtual sub-regions within the virtual space, calculates the positional error of feature point information extracted from the real space image based on the analysis result, and adjusts each virtual sub-region according to the calculated positional error.
In Article 1, A spatial integration unit that establishes a shared wall between the plurality of virtual sub-regions and integrates the plurality of virtual sub-regions to share the shared wall based on the shared wall. A spatial restructuring device that further includes
In Article 2, The above-mentioned space integration unit A space structure adjustment device that sets a wall surface within a preset threshold distance between walls included in the plurality of virtual sub-regions as the shared wall.
In Article 1, The above plan drawing generating unit A space structure adjustment device that selects one space type among multiple space types and selects an actual sub-area to which the selected space type is to be assigned within the entire indoor space area corresponding to the real space.
In Article 4, The above plan drawing generating unit A spatial structure adjustment device that generates the user input floor plan by placing spatial information corresponding to the selected space type in the selected actual sub-area.
In Article 5, The above adjustment unit A calculation unit that calculates a probability distribution for the actual sub-region included in the user input plan and calculates size information for the actual sub-region based on the calculated probability distribution. Includes, A spatial structure adjustment device wherein the size information for the actual sub-region includes the median, width information, and height information of the actual sub-region.
In Article 6, The above adjustment unit is, A spatial estimation unit that estimates a virtual sub-region corresponding to the actual sub-region within the virtual space by overlapping the user input plan view into the virtual space; and A fitting unit that fits a cube corresponding to the actual sub-region to an estimated virtual sub-space within the virtual space. A spatial restructuring device that further includes
In Article 7, A spatial structure adjustment device in which a cube corresponding to the actual sub-region is generated based on size information for the actual sub-region and depth information for the actual sub-region.
In Article 7, The above calculation unit Calculate the distance between the wall of the virtual subspace and the wall of the cube fitted to the virtual subspace, and A space structure adjustment device in which the fitting part adjusts the position of the wall surface of the virtual subspace based on the calculated distance.
In Article 1, The virtual space generation unit estimates location information of feature point information for the plurality of walls constituting the real space from the real space image based on the location value of a sensor installed on the wall of the real space, and A spatial structure adjustment device that generates a virtual space for the real space based on the above feature point information and the location information of the above feature point information.
In a method for adjusting a spatial structure performed by a spatial structure adjustment device, A step of generating a user input plan containing multiple actual sub-regions; A step of extracting feature point information for multiple walls from a real space image captured of the real space; A step of generating a virtual space for the real space based on the above feature point information and A step of adjusting a plurality of virtual sub-regions included in the virtual space based on the above user input plan. Includes, The above adjustment step is, Analyze the correspondence relationship between a plurality of actual sub-regions defined in the above user input plan and a plurality of virtual sub-regions within the above virtual space, and Based on the above analysis results, the positional error of the feature point information extracted from the above real space image is calculated, and A spatial structure adjustment method for adjusting each virtual sub-region according to the position error calculated above.
In Article 11, Step of setting a shared wall between the above plurality of virtual sub-regions and A spatial structure adjustment method further comprising the step of integrating the plurality of virtual sub-regions to share the shared wall based on the shared wall.
In Article 12, The step of setting a shared wall between the plurality of virtual sub-regions above A spatial structure adjustment method comprising the step of setting a wall surface as the shared wall, wherein the distance between wall surfaces included in the plurality of virtual sub-regions is within a preset threshold.
In Article 11, The step of generating the above user input floor plan is A method for adjusting a spatial structure, comprising the step of selecting one spatial type among a plurality of spatial types, and selecting an actual sub-area to which the selected spatial type is to be assigned within the entire indoor spatial area corresponding to the real space.
In Article 14, The step of generating the above user input floor plan is A spatial structure adjustment method comprising the step of generating the user input floor plan by placing spatial information corresponding to the selected space type in the selected actual sub-area.
In Article 11, The step of adjusting a plurality of virtual sub-regions included in the virtual space above A step of overlapping the user input floor plan into the virtual space to estimate a virtual sub-region corresponding to the actual sub-region within the virtual space, and A spatial structure adjustment method comprising the step of fitting a cube corresponding to the actual sub-region to an estimated virtual sub-space within the virtual space.
In Article 16, The step of adjusting a plurality of virtual sub-regions included in the virtual space above A step of calculating the distance between the wall of a virtual subspace and the wall of a cube fitted to the virtual subspace and A spatial structure adjustment method comprising the step of adjusting the position of the wall surface of the virtual subspace based on the above-calculated distance.
In Article 11, The step of creating a virtual space for the aforementioned real space is A step of estimating location information of feature point information for the plurality of walls constituting the real space from the real space image based on the location value of a sensor installed on the wall of the real space and A spatial structure adjustment method comprising the step of generating a virtual space for the real space based on the above feature point information and the location information of the above estimated feature point information.
In a computer program stored on a computer-readable recording medium comprising a sequence of instructions for adjusting a spatial structure, When the above computer program is executed by a computing device, Generate a user input floor plan containing multiple actual sub-areas, and Extract feature point information for multiple wall surfaces from a real-space video captured of the real space, and Based on the above feature point information, a virtual space for the above real space is generated, and Based on the above user input plan, a plurality of virtual sub-regions included in the virtual space are adjusted, and A computer program stored on a computer-readable recording medium, comprising a sequence of instructions for analyzing the correspondence between a plurality of real sub-regions defined in the above user input plan and a plurality of virtual sub-regions within the above virtual space, calculating the positional error of feature point information extracted from the above real space image based on the analysis result, and adjusting each virtual sub-region according to the calculated positional error.

Description

Device, method and computer program for adjusting spatial structures The present invention relates to a device, method, and computer program for adjusting a spatial structure. Methods for identifying and utilizing indoor structures using 3D scene understanding are one of the fields being actively researched along with the growth of the AR/MR industry. Here, research such as semantic segmentation, SLAM, and plane and object detection is used in combination for understanding 3D space. Because the complexity of understanding 3D space is very high, methods to understand sophisticated structures are constantly being researched. Semantic segmentation technology is a technology that generates the structure of a scene by deep learning (e.g., Mask RCNN) RGB images (or RGBD images) to divide the same area between image frames. Multiplane tracking technology is a technique that uses RGB (or RGBD) images to estimate planes corresponding to a scene and integrates them to generate spatial information. However, since indoor spaces are a combination of multiple spaces, it is difficult to accurately determine and understand the spatial structure if only RGB images are used. Simultaneous Localization and Mapping (SLAM) technology is primarily used for autonomous driving in applications such as mobile robots, and is utilized to create maps of the surrounding environment. This SLAM technology defines the camera's position over time by estimating its 3D location through matching feature points in a sequence of images. Once the sequence of camera positions is estimated, the 3D location of an expanded area is calculated using the 3D locations of the feature points estimated along with each camera position. Subsequently, when feature points at the same location as previously acquired 3D feature points are estimated, a point cloud is generated through loop closing. The optimized 3D location of the camera is then estimated using the generated point cloud and the camera. Even if the 3D positions of feature points are calculated using such SLAM technology, they do not possess sufficient accuracy to facilitate planar tracking. Furthermore, when capturing areas with almost no feature points, it is difficult to estimate walls or floors using SLAM technology. Additionally, it is very difficult to define the spatial structure of the entire space from the point cloud acquired through SLAM technology without user interaction (or input information). FIG. 1 is a block diagram of a space structure adjustment device according to one embodiment of the present invention. FIGS. 2a to 2d are drawings for explaining a method for generating a user input plan view according to an embodiment of the present invention. FIGS. 3a and 3b are drawings for explaining a method for generating a cube corresponding to an actual sub-region included in a user input plan according to an embodiment of the present invention. FIGS. 4a to 4h are drawings for explaining a method for adjusting a plurality of virtual sub-regions included in a virtual space according to an embodiment of the present invention. FIGS. 5a to 5c are drawings for explaining a method of placing an object in a virtual space according to an embodiment of the present invention. FIG. 6 is a flowchart illustrating a method for adjusting a spatial structure according to one embodiment of the present invention. Embodiments of the present invention are described below with reference to the attached drawings so that those skilled in the art can easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals. Throughout the specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected," but also cases where they are "electrically connected" with other components interposed between them. Furthermore, when a part is described as "including" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. In this specification, the term "part" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Additionally, one unit may be realized using two or more hardware, and two or more units may be realized by one hardware. Some of the operations or functions described in this specification as being performed by a terminal or device may instead be performed by a server connected to said terminal or device. Likewise, some of the operations or functions described as being performed by a server may also be performed by a terminal or device connected to said server. Hereinafter, specific details for