US-20230281870-A1 - STORAGE MEDIUM STORING COMPUTER PROGRAM AND POINT CLOUD PROCESSING APPARATUS

US20230281870A1US 20230281870 A1US20230281870 A1US 20230281870A1US-20230281870-A1

Abstract

A set of program instructions causes a computer to perform: selecting a point pair of a first point and a second point from a three-dimensional point cloud including points indicating a surface of a three-dimensional object; determining a first normal vector at the first point by using points included in a first area; determining a second normal vector at the first point by using points included in a second area; determining a third normal vector at the second point by using points included in a third area; determining a fourth normal vector at the second point by using points included in a fourth area; calculating a feature indicating a geometric feature by using the first normal vector, the second normal vector, the third normal vector, and the fourth normal vector; and performing a particular process by using the feature.

Inventors

TAKASU MASAYA

Assignees

BROTHER IND LTD

Dates

Publication Date: 20230907
Application Date: 20230103
Priority Date: 20220303

Claims (14)

1 . A non-transitory computer-readable storage medium storing a set of program instructions for a computer, the set of program instructions, when executed by a controller of the computer, causing the computer to perform: selecting a point pair of a first point and a second point from a three-dimensional point cloud including a plurality of points indicating a surface of a three-dimensional object, the first point and the second point being separated by a distance within a particular distance range; determining a first normal vector at the first point by using a plurality of points included in a first area of which a distance from the first point is within a first distance range; determining a second normal vector at the first point by using a plurality of points included in a second area of which a distance from the first point is within a second distance range, the second distance range including a distance greater than an upper limit of the first distance range; determining a third normal vector at the second point by using a plurality of points included in a third area of which a distance from the second point is within a third distance range; determining a fourth normal vector at the second point by using a plurality of points included in a fourth area of which a distance from the second point is within a fourth distance range, the fourth distance range including a distance greater than an upper limit of the third distance range; calculating a feature indicating a geometric feature by using the first normal vector, the second normal vector, the third normal vector, and the fourth normal vector; and performing a particular process by using the feature.
2 . The non-transitory computer-readable storage medium according to claim 1 , wherein a lower limit of the second distance range is greater than or equal to the upper limit of the first distance range.
3 . The non-transitory computer-readable storage medium according to claim 1 , wherein a lower limit of the fourth distance range is greater than or equal to the upper limit of the third distance range.
4 . The non-transitory computer-readable storage medium according to claim 1 , wherein one or both of a first condition and a second condition is satisfied, the first condition being that the first distance range is same as the third distance range, the second condition being that the second distance range is same as the fourth distance range.
5 . The non-transitory computer-readable storage medium according to claim 1 , wherein the selecting the point pair includes: selecting a plurality of first-type point pairs from a first three-dimensional point cloud that is generated by measuring a space containing a first three-dimensional object by a three-dimensional sensor, the plurality of first-type point pairs being a plurality of point pairs of the first point and the second point; and selecting a plurality of second-type point pairs from a second three-dimensional point cloud containing a plurality of points indicating a surface of a second three-dimensional object that is a reference three-dimensional object, the plurality of second-type point pairs being a plurality of point pairs of the first point and the second point; wherein the determining the first normal vector includes: determining the first normal vector of each of the plurality of first-type point pairs; and determining the first normal vector of each of the plurality of second-type point pairs; wherein the determining the second normal vector includes: determining the second normal vector of each of the plurality of first-type point pairs; and determining the second normal vector of each of the plurality of second-type point pairs; wherein the determining the third normal vector includes: determining the third normal vector of each of the plurality of first-type point pairs; and determining the third normal vector of each of the plurality of second-type point pairs; wherein the determining the fourth normal vector includes: determining the fourth normal vector of each of the plurality of first-type point pairs; and determining the fourth normal vector of each of the plurality of second-type point pairs; wherein the calculating the feature includes: calculating a first-type feature that is the feature of each of the plurality of first-type point pairs; and calculating a second-type feature that is the feature of each of the plurality of second-type point pairs; and wherein the particular process includes selecting particular combinations, each of the particular combinations being a combination of a first-type point pair and a second-type point pair satisfying a selection condition, the selection condition including that the first-type feature and the second-type feature match or are similar to each other, the first-type point pair being one of the plurality of first-type point pairs, the second-type point pair being one of the plurality of second-type point pairs.
6 . The non-transitory computer-readable storage medium according to claim 5 , wherein the particular process includes: classifying a plurality of first-type features of the plurality of first-type point pairs into a plurality of categories; and classifying a plurality of second-type feature of the plurality of second-type point pairs into the plurality of categories; and wherein the selecting the particular combinations includes selecting the particular combinations from combinations of the first-type point pair and the second-type point pair included in a same category satisfying a low ratio condition, the low ratio condition including that a ratio of a number of the second-type point pair included in a category to a total number of the plurality of second-type point pairs is smaller than or equal to an upper limit.
7 . The non-transitory computer-readable storage medium according to claim 6 , wherein the set of program instructions, when executed by the controller, causes the computer to further perform: calculating an additional feature indicating a geometric feature by using the first point and the second point in addition to the first normal vector, the second normal vector, the third normal vector, and the fourth normal vector; wherein the calculating the additional feature includes: calculating a first-type additional feature that is the additional feature of each of the plurality of first-type point pairs; and calculating a second-type additional feature that is the additional feature of each of the plurality of second-type point pairs; and wherein the selection condition includes the first-type additional feature and the second-type additional feature match or are similar to each other.
8 . The non-transitory computer-readable storage medium according to claim 5 , wherein the particular process includes: determining a local coordinate system by using a particular point pair, the local coordinate system having a first axis, a second axis, and a third axis perpendicular to one another, the particular point pair being the first-type point pair or the second-type point pair that forms the particular combinations; and wherein the determining the local coordinate system includes: calculating a first vector defining the first axis, the first vector having a start point that is the first point and an end point located on a straight line passing through the first point and the second point; calculating a third vector defining the third axis, the third vector being a cross product of the first normal vector and the first vector or a cross product of the second normal vector and the first vector; and calculating a second vector defining the second axis, the second vector being a cross product of the first vector and the third vector.
9 . The non-transitory computer-readable storage medium according to claim 1 , wherein the selecting the point pair includes selecting a plurality of point pairs of the first point and the second point from the three-dimensional point cloud; wherein the determining the first normal vector includes determining the first normal vector of each of the plurality of point pairs; wherein the determining the second normal vector includes determining the second normal vector of each of the plurality of point pairs; wherein the determining the third normal vector includes determining the third normal vector of each of the plurality of point pairs; wherein the determining the fourth normal vector includes determining the fourth normal vector of each of the plurality of point pairs; wherein the calculating the feature includes calculating the feature of each of the plurality of point pairs; wherein the feature includes an angle formed by two normal vectors selected from the first normal vector, the second normal vector, the third normal vector, and the fourth normal vector; and wherein the particular process includes selecting a small angle point pair from the plurality of point pairs, the small angle point pair satisfying a small angle condition indicating that the angle is small.
10 . The non-transitory computer-readable storage medium according to claim 1 , wherein the selecting the point pair includes selecting a plurality of point pairs of the first point and the second point from the three-dimensional point cloud; wherein the determining the first normal vector includes determining the first normal vector of each of the plurality of point pairs; wherein the determining the second normal vector includes determining the second normal vector of each of the plurality of point pairs; wherein the determining the third normal vector includes determining the third normal vector of each of the plurality of point pairs; wherein the determining the fourth normal vector includes determining the fourth normal vector of each of the plurality of point pairs; wherein the calculating the feature includes calculating the feature of each of the plurality of point pairs; wherein the particular process includes: classifying a plurality of features of the plurality of point pairs into a plurality of categories; selecting one or both of the first point and the second point of a point pair included in a category satisfying a low ratio condition, the low ratio condition including that a ratio of a number of the point pair included in the category to a total number of the plurality of point pairs is smaller than or equal to an upper limit; and storing data indicating a selected point in a memory.
11 . The non-transitory computer-readable storage medium according to claim 1 , wherein the selecting the point pair includes selecting a plurality of point pairs of the first point and the second point from the three-dimensional point cloud; wherein the determining the first normal vector includes determining the first normal vector of each of the plurality of point pairs; wherein the determining the second normal vector includes determining the second normal vector of each of the plurality of point pairs; wherein the determining the third normal vector includes determining the third normal vector of each of the plurality of point pairs; wherein the determining the fourth normal vector includes determining the fourth normal vector of each of the plurality of point pairs; wherein the calculating the feature includes calculating the feature of each of the plurality of point pairs; wherein the particular process includes: classifying a plurality of features of the plurality of point pairs into a plurality of categories; selecting a low rate point pair included in a category satisfying a low ratio condition, the low ratio condition including that a ratio of a number of the point pair included in the category to a total number of the plurality of point pairs is smaller than or equal to an upper limit; determining a local coordinate system by using the low rate point pair; and storing data indicating the local coordinate system in a memory; and wherein the determining the local coordinate system includes: calculating a first vector defining a first axis, the first vector having a start point that is the first point and an end point located on a straight line passing through the first point and the second point; calculating a third vector defining a third axis, the third vector being a cross product of the first normal vector and the first vector or a cross product of the second normal vector and the first vector; and calculating a second vector defining a second axis, the second vector being a cross product of the first vector and the third vector.
12 . The non-transitory computer-readable storage medium according to claim 6 , wherein the particular process further includes: calculating a candidate orientation parameter of each of the particular combinations, the candidate orientation parameter being a parameter that projects a coordinate system of the second three-dimensional point cloud onto a coordinate system of the first three-dimensional point cloud such that a local coordinate system of the second-type point pair of the combination satisfying the selection condition is projected onto a local coordinate system of the first-type point pair of the combination satisfying the selection condition; calculating an evaluation value of the candidate orientation parameter of each of the particular combinations to select a best evaluation value; and controlling a robot arm by using the candidate orientation parameter having the best evaluation value.
13 . The non-transitory computer-readable storage medium according to claim 1 , wherein the three-dimensional point cloud includes: a model point cloud including a plurality of points indicating a surface of a reference object; and a scene point cloud including a plurality of points that is generated by measuring a space containing a target object by a three-dimensional sensor; and wherein the particular process includes: performing matching between the reference object and the target object by using the feature calculated for the point pair of the model point cloud and the feature calculated for the point pair of the scene point cloud; determining a position and an orientation of the target object based on the matching; and controlling a robot arm by using the position and the orientation of the target object.
14 . A point cloud processing apparatus comprising: an interface; a memory; and a controller configured to perform: acquiring data indicating a three-dimensional point cloud through the interface or from the memory, the three-dimensional point cloud including a plurality of points indicating a surface of a three-dimensional object; selecting a point pair of a first point and a second point from the three-dimensional point cloud, the first point and the second point being separated by a distance within a particular distance range; determining a first normal vector at the first point by using a plurality of points included in a first area of which a distance from the first point is within a first distance range; determining a second normal vector at the first point by using a plurality of points included in a second area of which a distance from the first point is within a second distance range, the second distance range including a distance greater than an upper limit of the first distance range; determining a third normal vector at the second point by using a plurality of points included in a third area of which a distance from the second point is within a third distance range; determining a fourth normal vector at the second point by using a plurality of points included in a fourth area of which a distance from the second point is within a fourth distance range, the fourth distance range including a distance greater than an upper limit of the third distance range; calculating a feature indicating a geometric feature by using the first normal vector, the second normal vector, the third normal vector, and the fourth normal vector; and performing a particular process by using the feature.

Description

REFERENCE TO RELATED APPLICATIONS This application claims priority from Japanese Patent Application No. 2022-032952 filed on Mar. 3, 2022. The entire content of the priority application is incorporated herein by reference. BACKGROUND ART Techniques have been proposed for estimating a position and an orientation of a three-dimensional object by using three-dimensional point clouds. DESCRIPTION For example, a point pair feature for point pairs selected from a model point cloud is calculated for each point pair. A plurality of key points are selected from the model point cloud. The point pair feature for point pairs in a scene point cloud is calculated. Point pairs with point pair features similar to the point pair features for a plurality of key points are searched for. Coordinate transformation parameters are determined for each of the searched point pairs, and the number of counts is determined for each orientation indicated by the coordinate transformation parameters. A plurality of candidate orientations are selected in descending order of the number of counts. The orientation that maximizes the degree of matching between the model point cloud and the scene point cloud is determined as the final orientation. Three-dimensional point clouds may be used in a variety of processes, such as determination of a position and an orientation. However, the processing of three-dimensional point clouds is not easy, and there is room for improvement. In view of the foregoing, this specification discloses techniques for processing three-dimensional point clouds. According to one aspect, this specification discloses a non-transitory computer-readable storage medium storing a set of program instructions for a computer. The set of program instructions, when executed by a controller of the computer, causes the computer to perform: selecting a point pair of a first point and a second point from a three-dimensional point cloud including a plurality of points indicating a surface of a three-dimensional object, the first point and the second point being separated by a distance within a particular distance range; determining a first normal vector at the first point by using a plurality of points included in a first area of which a distance from the first point is within a first distance range; determining a second normal vector at the first point by using a plurality of points included in a second area of which a distance from the first point is within a second distance range, the second distance range including a distance greater than an upper limit of the first distance range; determining a third normal vector at the second point by using a plurality of points included in a third area of which a distance from the second point is within a third distance range; determining a fourth normal vector at the second point by using a plurality of points included in a fourth area of which a distance from the second point is within a fourth distance range, the fourth distance range including a distance greater than an upper limit of the third distance range; calculating a feature indicating a geometric feature by using the first normal vector, the second normal vector, the third normal vector, and the fourth normal vector; and performing a particular process by using the feature. According to this configuration, an appropriate feature indicating a geometric feature is calculated by using the first point, the first normal vector and the second normal vector at the first point, the second point, the third normal vector and the fourth normal vector at the second point. Further, the particular process is appropriately executed by using the calculated feature. According to another aspect, this specification also discloses a non-transitory computer-readable storage medium storing a set of program instructions for a computer. The set of program instructions, when executed by a controller of the computer, causes the computer to perform: selecting a point pair of a first point and a second point from a three-dimensional point cloud including a plurality of points indicating a surface of a three-dimensional object, the first point and the second point being separated by a distance within a particular distance range; determining a first normal vector at the first point by using a plurality of points included in a first area of which a distance from the first point is within a first distance range; and determining a local coordinate system having a first axis, a second axis, and a third axis perpendicular to one another by using the first point, the second point, and the first normal vector, the determining the local coordinate system including: calculating a first vector defining the first axis, the first vector having a start point that is the first point and an end point located on a straight line passing through the first point and the second point; calculating a third vector defining the third axis, the third vector being a cross product of the f