US-20260127810-A1 - COMPUTER-IMPLEMENTED METHOD FOR UPDATING A REPRESENTATION OF A SPATIAL SCENE

US20260127810A1US 20260127810 A1US20260127810 A1US 20260127810A1US-20260127810-A1

Abstract

A method for updating a representation of a spatial scene includes receiving image points, acquiring acquisition points, and assigning acquisition points to corresponding image points based on associated position information. Respective differences are determined by comparing the position information of each acquisition point with the position information of the assigned image point. Acquisition points that have a difference below a tolerance threshold value are excluded from the update. Image points of the representation are grouped into a sub-representation. It is determined whether an acquisition point lies inside a volume generated by a spatial surrounding area and an image sensor, and the sub-representation image point is excluded from the update if the difference of the acquisition point lying in the volume lies above a movement threshold value. Image points of the representation that are not excluded from the update are updated solely based on acquisition points not excluded from the update.

Inventors

Markus-Hermann Koch

Assignees

Siemens Healthineers Ag

Dates

Publication Date: 20260507
Application Date: 20251107
Priority Date: 20241107

Claims (16)

1 . A method for updating a representation of a spatial scene, the method being computer-implemented and comprising: receiving image points of the representation of the spatial scene; acquiring a plurality of acquisition points of the spatial scene using a camera, wherein each acquisition point of the plurality of acquisition points is assigned position information; assigning acquisition points each to a corresponding image point of the representation based on the associated position information; determining respective differences, the determining of the respective differences comprising comparing the position information of each acquisition point of the plurality of acquisition points with the position information of the assigned image point; excluding the acquisition points of the plurality of acquisition points that have a difference below a tolerance threshold value from the updating, the excluding comprising: grouping a plurality of the image points of the representation into a sub-representation; when the difference lies above the tolerance threshold value for a subset of image points of the plurality of image points of the sub-representation, and below the tolerance threshold value for a further subset of image points of the plurality of image points of the sub-representation, defining a spatial surrounding area for a sub-representation image point of the subset that has a difference above the tolerance threshold value; determining whether at least one acquisition point of the plurality of acquisition points lies inside a volume generated by the defined spatial surrounding area and an image sensor of the camera; and excluding the sub-representation image point from the updating when the difference of the at least one acquisition point lying inside the volume lies above a movement threshold value; updating solely the image points of the representation that are not excluded from the updating solely based on acquisition points not excluded from the updating; and providing the updated representation.
2 . The method of claim 1 , wherein the sub-representation is retained unchanged if no image point of the plurality of image points of the sub-representation is approved for the updating.
3 . The method of claim 1 , wherein the sub-representation is retained unchanged when at most a limited proportion of the plurality of image points of the sub-representation is approved for the updating.
4 . The method of claim 3 , wherein the sub-representation is retained unchanged when a proportion of at most half of the plurality of image points of the sub-representation is approved for the updating.
5 . The method of claim 1 , wherein the surrounding area of the sub-representation image point is defined to be circular or spherical.
6 . The method of claim 1 , wherein the surrounding area is defined such that the surrounding area includes all image points of the plurality of image points of the sub-representation that have a difference lying above the tolerance threshold value.
7 . The method of claim 1 , wherein the comparing of the position information is performed such that a spatial distance is determined as the difference.
8 . The method of claim 1 , wherein the tolerance threshold value equals 5 cm to 50 cm.
9 . The method of claim 8 , wherein the tolerance threshold value equals 15 cm.
10 . The method of claim 1 , wherein the movement threshold value equals 35 cm to 65 cm.
11 . The method of claim 10 , wherein the movement threshold value equals 50 cm.
12 . The method of claim 1 , wherein the movement threshold value is defined as half a distance from the sub-representation image point to the camera.
13 . The method of claim 1 , wherein the representation of the spatial scene has the format of a 3D net model.
14 . The method of claim 1 , wherein in order to obtain the sub-representation, the representation of the spatial scene is segmented, and wherein the segmentation determines related regions of the representation of the spatial scene, with image points that belong to a related region being grouped into the sub-representation.
15 . A provider unit for providing an updated representation of a spatial scene, the provider unit comprising: a computing device configured to: receive image points of the representation of the spatial scene; acquire a plurality of acquisition points of the spatial scene using a camera, wherein each acquisition point of the plurality of acquisition points is assigned position information; assign acquisition points each to a corresponding image point of the representation based on the associated position information; determine respective differences, the determination of the respective differences comprising comparison of the position information of each acquisition point of the plurality of acquisition points with the position information of the assigned image point; exclude the acquisition points of the plurality of acquisition points that have a difference below a tolerance threshold value from the updating, the computing device being configured to exclude the acquisition points that have the different below the tolerance threshold value from the updating comprising the computing device being configured to; group a plurality of image points of the representation into a sub-representation; when the difference lies above the tolerance threshold value for a subset of image points of the plurality of image points of the sub-representation, and below the tolerance threshold value for a further subset of image points of the plurality of image points of the sub-representation, define a spatial surrounding area for a sub-representation image point of the subset that has a difference above the tolerance threshold value; determine whether at least one acquisition point of the plurality of acquisition points lies inside a volume generated by the defined spatial surrounding area and an image sensor of the camera; and exclude the sub-representation image point from the updating when the difference of the at least one acquisition point lying inside the volume lies above a movement threshold value; update solely the image points of the representation that are not excluded from the updating solely based on acquisition points not excluded from the updating; and provide the updated representation.
16 . In a non-transitory computer-readable storage medium that stores instructions executable by one or more processors to update a representation of a spatial scene, the instructions comprising: receiving image points of the representation of the spatial scene; acquiring a plurality of acquisition points of the spatial scene using a camera, wherein each acquisition point of the plurality of acquisition points is assigned position information; assigning acquisition points each to a corresponding image point of the representation based on the associated position information; determining respective differences, the determining of the respective differences comprising comparing the position information of each acquisition point of the plurality of acquisition points with the position information of the assigned image point; excluding the acquisition points of the plurality of acquisition points that have a difference below a tolerance threshold value from the updating, the excluding comprising: grouping a plurality of image points of the representation into a sub-representation; when the difference lies above the tolerance threshold value for a subset of image points of the plurality of image points of the sub-representation, and below the tolerance threshold value for a further subset of image points of the plurality of image points of the sub-representation, defining a spatial surrounding area for a sub-representation image point of the subset that has a difference above the tolerance threshold value; determining whether at least one acquisition point of the plurality of acquisition points lies inside a volume generated by the defined spatial surrounding area and an image sensor of the camera; and excluding the sub-representation image point from the updating when the difference of the at least one acquisition point lying inside the volume lies above a movement threshold value; updating solely the image points of the representation that are not excluded from the updating solely based on acquisition points not excluded from the updating; and providing the updated representation.

Description

This application claims the benefit of German Patent Application No. DE 10 2024 210 705.0, filed on Nov. 7, 2024, which is hereby incorporated by reference in its entirety. BACKGROUND The present embodiments relate to updating a representation of a spatial scene. Computer-implemented methods for updating a representation of a spatial scene are known from the prior art. Known solutions for capturing and updating a spatial scene include acquiring point clouds using a suitable camera (e.g., a depth camera, a time-of-flight camera, or a light imaging detection and ranging (LIDAR) camera. Such cameras allow 3D information to be acquired solely for those elements of a spatial scene that are visible to the camera. As this provides it is not possible to acquire a complete 3D scene, such cameras are also referred to as 2.5D cameras. In the analysis of point clouds of a spatial scene that are acquired using 2.5D cameras or LIDAR cameras, surfaces and edges may be recognized and modeled in the form of a 3D mesh model. An acquired scene may be updated continuously to provide that moving objects and changes are recognized (e.g., as part of a collision avoidance system). Collision avoidance systems are employed in robot-assisted X-ray angiography systems, for example. It is known from the prior art to segment camera-acquired acquisition points of a point cloud. The segmentation leads to the subdivision of the acquisition points into sub-point-clouds. Thus, the acquisition points of a representation are consequently subdivided into subrepresentations. For example, the segmentation may lead to subdivision into sub-point-clouds or subrepresentations that may correspond to elements of the spatial scene, so, for example, to objects or people. It is also known to model such elements, or the associated sub-point-clouds, as a 3D model (e.g., as a 3D mesh model). As a result of such segmentation and modeling, the acquisition points are thus assigned to the segmented sub-point-cloud or the subrepresentation or 3D model (e.g., 3D mesh model) of the associated element of the spatial scene. SUMMARY AND DESCRIPTION The scope of the present invention is defined solely by the appended claims and is not affected to any degree by the statements within this summary. One problem, for example, in the case of real-time applications (e.g., for collision avoidance), is the considerable computational effort involved in transforming point clouds into 3D mesh models. An additional problem is the limited measurement accuracy of the image sensors used. Both impede the recognition of movements and changes in the scene. The recognition should be performed as quickly as possible and without latency. Both are integrated into motion planning for avoiding collisions. The present embodiments are based on the knowledge that static elements of a scene create significant redundancy in the transformation of point clouds into 3D mesh models; for static elements of the scene, which do not move (e.g., walls, tables, equipment), the reprocessing effort for the transformation into a mesh model is not necessary. The present embodiments may obviate one or more of the drawbacks or limitations in the related art. For example, methods for transforming a point cloud into a 3D mesh model for updating a spatial scene are improved and sped up, accuracy is increased, and the redundancy and inefficiency of such methods are reduced by improving the distinguishing between static and dynamic elements of the scene. A computer-implemented method for updating a representation of a spatial scene includes: S1) receiving image points of the representation of the spatial scene; S2) acquiring a plurality of acquisition points of the scene using a camera, with each acquisition point of the plurality of acquisition points being assigned position information; S3) assigning acquisition points each to a corresponding image point of the representation based on the associated position information; S4) determining respective differences by comparing the position information of each acquisition point with the position information of the assigned image point; and S5) excluding from the update the acquisition points that have a difference below a tolerance threshold value. The excluding includes: S6) grouping a plurality of image points of the representation into a sub-representation; S7) if the difference lies above the tolerance threshold value for a subset of the sub-representation image points, and below the tolerance threshold value for a further subset of the sub-representation image points, defining a spatial surrounding area for a sub-representation image point of the subset that has a difference above the tolerance threshold value; S8) determining whether at least one acquisition point lies inside a volume generated by the defined spatial surrounding area and the image sensor of the camera; and S9) excluding the sub-representation image point from the update if the difference of the at le