CN-121986490-A - 6D+ geometric attribute vector matching for motion estimation in dynamic point clouds

Abstract

The encoder may determine a first data set associated with a first Point Cloud Frame (PCF) and a second data set associated with a second PCF. The first and second data sets may include geometry and attribute data. The encoder may construct a first joint vector based on the first data set associated with the first PCF by using the geometry data and the attribute data of the first data set. The encoder may construct a second joint vector based on a second set of data associated with a second PCF. The second joint vector may be constructed using geometry and attribute data of the second data set. The encoder may perform a matching calculation using the first and second joint vectors to determine a match between the point (or set of points) of the first PCF and the point (or set of points) of the second PCF.

Inventors

• M. Kriwokuka
• G. Sandri
• F. SUDO
• B. Chubo

Assignees

• 交互数字CE专利控股有限公司

Dates

Publication Date

20260505

Application Date

20241002

Priority Date

20231009

Claims (20)

1. 1.A method performed by an encoder, the method comprising: Determining a first data set associated with a first point cloud frame and a second data set associated with a second point cloud frame, wherein the first data set includes geometric data and attribute data, and wherein the second data set includes geometric data and attribute data; constructing a first joint vector based on a first data set associated with the first point cloud frame, wherein the first joint vector is constructed using geometry data and attribute data of the first data set; Constructing a second joint vector based on a second data set associated with the second point cloud frame, wherein the second joint vector is constructed using geometry data and attribute data of the second data set, and A matching calculation is performed using the first and second joint vectors to determine a match between points of the first point cloud frame and points of the second point cloud frame.
2. 2. The method of claim 1, further comprising determining one or more motion vectors associated with the first and second point cloud frames based on a match between the points of the first point cloud frame and the points of the second point cloud frame.
3. 3. The method of claim 1, wherein the first joint vector comprises at least six dimensions and the second joint vector comprises at least six dimensions, and wherein the at least six dimensions comprise at least three color components and at least three geometric components.
4. 4. The method of claim 1, wherein constructing the first or second joint vector comprises normalizing geometry data and attribute data, wherein normalizing data comprises dividing geometry and attribute data to determine uniform weights for the geometry and attribute data.
5. 5. The method of claim 1, wherein the first data set is a representation of more than a single point of a first point cloud frame, and wherein the second data set is a representation of more than a single point of a second point cloud frame.
6. 6. The method of claim 1, wherein the first point cloud frame comprises a first set of one or more points and the second point cloud frame comprises a second set of one or more points, wherein performing the matching calculation comprises determining a match between the first set of one or more points and the second set of one or more points by using geometry and attribute data associated with the first set of one or more points and by using geometry and attribute data associated with the second set of one or more points.
7. 7. The method of claim 1, wherein the matching calculation is configured to determine that the point in the first point cloud frame and the point in the second point cloud frame have a lowest euclidean distance between the first joint vector and the second joint vector to determine a match between the point in the first point cloud frame and the point in the second point cloud frame.
8. 8. The method of claim 1, wherein the geometric data comprises one or more geometric components, wherein the attribute data comprises one or more color components, and wherein at least one of the one or more geometric components is assigned a different weight than at least one of the one or more color components.
9. 9. An encoder, comprising: a processor configured to: Determining a first data set associated with a first point cloud frame and a second data set associated with a second point cloud frame, wherein the first data set includes geometric data and attribute data, and wherein the second data set includes geometric data and attribute data; constructing a first joint vector based on a first data set associated with the first point cloud frame, wherein the first joint vector is constructed using geometry data and attribute data of the first data set; Constructing a second joint vector based on a second data set associated with the second point cloud frame, wherein the second joint vector is constructed using geometry data and attribute data of the second data set, and A matching calculation is performed using the first and second joint vectors to determine a match between points of the first point cloud frame and points of the second point cloud frame.
10. 10. The encoder of claim 9, wherein the processor is configured to determine one or more motion vectors associated with the first and second point cloud frames based on a match between the points of the first point cloud frame and the points of the second point cloud frame.
11. 11. The encoder of claim 9, wherein the first joint vector comprises at least six dimensions and the second joint vector comprises at least six dimensions, and wherein the at least six dimensions comprise at least three color components and at least three geometric components.
12. 12. The encoder of claim 9, wherein the processor being configured to construct the first or second joint vector comprises the processor being configured to normalize the geometry data and the attribute data, wherein the processor being configured to normalize the data comprises the processor being configured to divide the geometry and attribute data to determine uniform weights for the geometry and attribute data.
13. 13. The encoder of claim 9, wherein the first data set is a representation of more than a single point of the first point cloud frame, and wherein the second data set is a representation of more than a single point of the second point cloud frame.
14. 14. The encoder of claim 9, wherein the first point cloud frame comprises a first set of one or more points and the second point cloud frame comprises a second set of one or more points, wherein the processor being configured to perform the matching calculation comprises the processor being configured to determine a match between the first set of one or more points and the second set of one or more points by using geometry and attribute data associated with the first set of one or more points and by using geometry and attribute data associated with the second set of one or more points.
15. 15. The encoder of claim 9, wherein the matching calculation is configured to determine that the point in the first point cloud frame and the point in the second point cloud frame have a lowest euclidean distance between the first joint vector and the second joint vector to determine a match between a corresponding point (or set of points) in the first point cloud frame and a corresponding point (or set of points) in the second point cloud frame.
16. 16. The encoder of claim 9, wherein the geometric data comprises one or more geometric components, wherein the attribute data comprises one or more color components, and wherein at least one of the one or more geometric components is assigned a different weight than at least one of the one or more color components.
17. 17. A method performed by an encoder, the method comprising: Determining a first data set associated with a first point cloud frame and a second data set associated with a second point cloud frame, wherein the first data set includes geometric data and attribute data, and wherein the second data set includes geometric data and attribute data, and A matching calculation is performed to determine a match between points of the first point cloud frame and points of the second point cloud frame, wherein the matching calculation is determined based on a comparison between (1) a combination of geometry data and attribute data of the first point cloud frame and (2) a combination of geometry data and attribute data of the second point cloud frame.
18. 18. The method of claim 17, further comprising: Constructing a first data structure based on a first data set associated with a first point cloud frame, wherein the first data structure is constructed using geometry data and attribute data of the first data set, and Constructing a second data structure based on a second data set associated with the second point cloud frame, wherein the second data structure is constructed using geometry data and attribute data of the second data set; wherein the matching calculation is performed by comparing the first data structure and the second data structure to determine a match between i) a point or set of points of the first point cloud frame and ii) a point or set of points of the second point cloud frame.
19. 19. The method of claim 18, wherein the first data structure comprises a first joint vector, and wherein the second data structure comprises a second joint vector.
20. 20. The method of claim 17, further comprising determining one or more motion vectors associated with the first and second point cloud frames based on a match between the points or sets of points of the first point cloud frame and the points or sets of points of the second point cloud frame.

Description

6D+ geometric attribute vector matching for motion estimation in dynamic point clouds Cross Reference to Related Applications The present application claims priority from european patent application number 23306741.2 filed by the european patent office at 10/9 of 2023, the entire contents of which are incorporated herein by reference. Background Advances in 3D capture and rendering technology have enabled new applications and services in the areas of autonomous driving, cultural heritage archiving, immersive telepresence, and virtual/augmented reality, among others. The point cloud has emerged as one of the primary 3D scene representations for such applications. The point cloud consists of a collection of 3D points, each represented by its 3D position (x, y, z) and possibly several properties such as color, transparency, reflectivity, etc. Disclosure of Invention The 6d+ geometric attribute vector, generated by normalizing the geometric and attribute data into a single vector (vector), may be used for vector matching calculations (e.g., for applications such as motion estimation using dynamic point clouds). The 6D+ geometric attribute vector allows for geometric data and attribute data of a given set or block (e.g., points within a point cloud) to be considered jointly (jointly). Using 6D + geometric attribute vectors may allow for more overall (holistically) accurate computations (e.g., finding better point correspondences across different point cloud frames). The encoder may determine a first data set associated with a first point cloud frame and a second data set associated with a second point cloud frame. The first data set may include geometric data and attribute data. The second data set may include geometric data and attribute data. The encoder may construct a first joint vector. For example, the encoder may construct a first joint vector based on a first set of data associated with a first point cloud frame. The first joint vector may be constructed using the geometry data and the attribute data of the first data set. The encoder may construct a second joint vector. For example, the encoder may construct a second joint vector based on a second data set associated with a second point cloud frame. The geometric data and attribute data of the second data set may be used to construct a second joint vector. The encoder may perform a matching calculation using the first and second joint vectors to determine a match between points of the first point cloud frame and points of the second point cloud frame. The encoder may determine one or more motion vectors associated with the first and second point cloud frames, e.g., based on a match between points of the first point cloud frame and corresponding points of the second point cloud frame. The first joint vector may include at least six dimensions. The second joint vector may include at least six dimensions. The at least six dimensions may include at least three color components and at least three geometric components. Constructing the first and/or second joint vector may include normalizing the geometric data and the attribute data. The normalization data (e.g., geometry data and attribute data) may include partitioning the geometry and attribute data to determine uniform (even) weights for the geometry and attribute data. The first data set may be a representation of more than a single point of the first cloud frame. The second data set may be a representation of more than a single point of the second point cloud frame. The first point cloud frame may include a first set of one or more points. The second point cloud frame may include a second set of one or more points. Performing the matching calculation may include determining a match between the first set of one or more points and the second set of one or more points by using geometry and attribute data associated with the first set of one or more points and by using geometry and attribute data associated with the second set of one or more points. The matching calculation may be configured to determine that a point (or set of points) in the first point cloud frame and a point (or set of points) in the second point cloud frame have a lowest euclidean distance between the first joint vector and the second joint vector to determine a match between the point (or set of points) in the first point cloud frame and the point (or set of points) in the second point cloud frame. The geometric data may include one or more geometric components. The attribute data may include one or more color components. At least one of the one or more geometric components may be assigned a different weight than at least one of the one or more color components. The encoder may determine a first data set associated with a first point cloud frame and a second data set associated with a second point cloud frame. The first data set may include geometric data and attribute data. The second data set may include geometric data and attribute