Search

KR-20260064891-A - System for assuming length of delivery goods from low quality Image without depth information and control method thereof

KR20260064891AKR 20260064891 AKR20260064891 AKR 20260064891AKR-20260064891-A

Abstract

A single-image-based parcel length estimation system without depth information according to the present invention comprises: a Salient Object Detection (SOD) module that detects an object of the parcel from a customer verification image of the parcel collected at a parcel logistics site; and a Corner Detection (CD) module that searches for corners in the object of the parcel detected by the SOD module. The system includes an Object Measurement (OM) module that utilizes neural network-based learning techniques to define the projection relationship between corner pixels and real-world coordinates in the object of the parcel searched by the CD module, and derives the 3D dimensions of the parcel by estimating the real-world length corresponding to the distance between pixels through the output variables of three neural network models. The OM module learns and utilizes a deep learning algorithm that converts the 2D coordinates of the customer verification image of the parcel into 3D real-world coordinates based on the corners of each grid pattern using a grid-patterned flat plate captured in advance. For the XY plane, it learns using a single grid-patterned flat plate with a z-coordinate of 0, and for the XZ plane, it learns using multiple images generated by moving a grid-patterned flat plate standing perpendicular to the ground at regular intervals along the y-axis direction as input data, and derives the 3D dimensions of the parcel according to the learned results. Accordingly, it is possible to measure the 3D dimensions of the parcel from a single black-and-white image that does not contain low-resolution depth information, and by utilizing existing infrastructure, it has the advantage of reducing costs and facilitating the dissemination and expansion of the system.

Inventors

  • 김준
  • 김다윗
  • 김보현

Assignees

  • 한국생산기술연구원

Dates

Publication Date
20260508
Application Date
20241030

Claims (20)

  1. SOD (Salient Object Detection) module for detecting objects of said parcels from customer verification images of said parcels collected at a parcel logistics site; A CD (Corner Detection) module that searches for corners in the object of the parcel detected by the SOD module; and It includes an OM (Object Measurement) module that defines the projection relationship between corner pixels and real-world coordinates in the object of the parcel searched by the CD module using a neural network-based learning technique, and derives the 3D dimensions of the parcel by estimating the real-world length corresponding to the distance between pixels through the output variables of three neural network models, The above OM module utilizes a deep learning algorithm that converts the 2D coordinates of a customer proof image of the parcel into 3D real-world coordinates based on the corners of each grid pattern using a pre-captured grid-patterned flat plate, and learns on a single grid-patterned flat plate with a z-coordinate of 0 for the XY plane, and learns by utilizing multiple images generated by moving a grid-patterned flat plate standing perpendicular to the ground at regular intervals along the y-axis direction as input data for the XZ plane, and derives the 3D dimensions of the parcel based on a single image without depth information according to the learned result.
  2. In paragraph 1, The above OM module is a single image-based delivery cargo length estimation system without depth information, which converts the coordinates (P x , P y ) of the corner pixels of the delivery cargo in the case of the XY plane appearing in the customer proof image of the delivery cargo based on the grid pattern corners extracted from the above grid pattern flat plate into 2D real-world coordinates (W x , W y ) corresponding to the coordinates, and converts the coordinates (P x , P y ) of the corner pixels of the delivery cargo in the case of the XZ plane into 3D real-world coordinates (W x , W y , W z ) corresponding to the coordinates.
  3. In paragraph 2, The above OM module is a single-image-based delivery cargo length estimation system without depth information that estimates the length between corners by utilizing the distance between the origin and the real coordinates of the corner pixel in the XY plane and XZ plane, and the angle that the corner pixel makes with the origin as dependent variables.
  4. In paragraph 3, The above OM module is for the case of the XY plane (Here, origin (W x0 , W y0 ), real coordinates in the XY plane (W x , W y )), for the XZ plane A single image-based delivery cargo length estimation system without depth information that calculates the distance between the real-world coordinates of the corner pixel and the origin by means of (where, origin (W x0 , W y0 , W z0 ), real-world coordinates (W x , W y , W z ))
  5. In paragraph 3, When calculating the angle that the corner pixel makes with the origin, the above OM module calculates the angle between the origin (W x 0 , W y 0 ) and the real-world coordinates (W x , W y ) in a 2D plane in the XY plane, and calculates the angle between the unit vector v z of the Z-axis and the vector v of the corner pixel in a 3D vector space in the XZ plane, and in the XY plane , in the XZ plane A single-image-based parcel length estimation system without depth information calculated by
  6. In paragraph 3, The above OM module constructs individual data sets in each plane, uses the distance between the real-world coordinates of the corner pixel calculated in the XY plane and the origin, and the angle that the corner pixel makes with the origin as output variables, and uses pairs of the corresponding origin (P x 0 , P y 0 ) and the coordinates (P x , P y ) of the corner pixel as input variables, and generates respective data sets Dataset L and Dataset A according to the distance between the real-world coordinates of the corner pixel and the origin, and the angle that the corner pixel makes with the origin, a single image-based parcel length estimation system without depth information.
  7. In paragraph 6, The above OM module is a single image-based parcel length estimation system without depth information, wherein the real distance L es between two adjacent pixel coordinates (P x1 , P y1 ) and (P x2 , P y2 ) among the corner pixels is calculated by the following mathematical formula using the distances L 1 and L 2 between the real coordinates of the corner pixel calculated in the XY plane and the origin, and the angles A 1 and A 2 that the corner pixel makes with the origin. [Mathematical Formula] Here,
  8. In paragraph 3, The above OM module is an index for subdividing the area projected onto the 2D pixel coordinates (P x , P y ) of the corner pixel on the XZ plane. Set, A single-image-based parcel length estimation system without depth information that generates binary variables z and b equal to the number of pairs of planes and uses them as the input structure of a dataset.
  9. In paragraph 8, The above OM module adds a binary variable z b to the pixel coordinate matrix of the corner pixel in the XZ plane, and the z b is a single image-based parcel length estimation system without depth information, which is distinguished as 0 and 1 by the following mathematical formula according to the W z value, which is the real z coordinate on the XZ plane. [Mathematical Formula] (P x , P y , z b ){If W z =0, z b =0, otherwise, z b =1}
  10. In paragraph 6, The angle Au formed by the real-world coordinates ( Wxu , Wyu , Wzu ) among the corner pixels with the origin has a dependent relationship with the distance Lu between the real-world coordinates ( Wxu , Wyu , Wzu ) and the real-world origin coordinates ( Wx0 , Wy0 , Wz0 ), and Lu has a dependent relationship based on trigonometry with the distance Ll between the real-world origin coordinates ( Wx0 , Wy0, Wz0 ) and the real-world coordinates ( Wxl , Wyl , Wzl ) among the corner pixels. The above OM module is a single image-based parcel length estimation system without depth information that constructs Dataset M3 by defining input variables based on the pixel coordinates of the corner pixels and L l , Lu u , and A u as output variables.
  11. In Paragraph 10, A single-image-based delivery cargo length estimation system without depth information, wherein Dataset L and Dataset A are used as inputs and outputs of a deep learning algorithm to estimate the width and height of the bottom surface of the delivery cargo in the XY plane, and Dataset M3 is used to estimate the height of the delivery cargo in the XZ plane.
  12. In Paragraph 11, The above OM module is the length estimated from the pixel coordinates for the corners on the XY plane by a neural network MLP L or MLP A using the above Dataset L or the above Dataset A. or angle A single-image-based parcel length estimation system without depth information that outputs
  13. In Paragraph 11, The above OM module sequentially uses three MLPs according to the output structure of the Dataset M3 based on M3-Net (Neural Network Measuring Three-dimensions) when estimating length and angle for pixel coordinates on the XZ plane, wherein the estimated values obtained by inputting (P x0 , P y0 , 0) and (P x , P y , 0) to the first MLP is the length contained in the ground, which is input to the second MLP along with pixel coordinates and is the estimated length influenced by the Z-axis. Provides information to output, and the above and the above A single-image-based parcel length estimation system without depth information that is added to the input structure of the third MLP to provide information for angle estimation for pixel coordinates (P x , P y , 1).
  14. A step of detecting an object of the said courier shipment from a customer proof image of the courier shipment collected at a courier logistics site; A step of searching for corners in the object of the detected courier shipment; and The method includes the step of defining the projection relationship between corner pixels and real-world coordinates in the object of the parcel discovered using a neural network-based learning technique, and deriving the three-dimensional dimensions of the parcel by estimating the real-world length corresponding to the distance between pixels through the output variables of three neural network models. A control method for a single-image-based parcel length estimation system without depth information, comprising the step of deriving the three-dimensional dimensions of the parcel, wherein the step of deriving the three-dimensional dimensions of the parcel comprises learning and utilizing a deep learning algorithm that converts the two-dimensional coordinates of a customer proof image of the parcel into three-dimensional real-world coordinates based on the corners of each grid pattern using a grid pattern planar plate captured in advance, learning for a single grid pattern planar plate with a z-coordinate of 0 for the XY plane, and learning for the XZ plane by utilizing multiple images generated by moving a grid pattern planar plate set perpendicular to the ground at regular intervals along the y-axis direction as input data, and deriving the three-dimensional dimensions of the parcel according to the learned result.
  15. In Paragraph 14, The step of deriving the three-dimensional dimensions of the above-mentioned courier shipment is, A step of converting the coordinates (P x , P y ) of the corner pixels of the parcel in the XY plane appearing in the customer proof image of the parcel, based on the corners of the grid pattern extracted from the grid pattern flat plate, into 2D real-world coordinates (W x , W y ) corresponding to the coordinates; and A control method for a single-image-based parcel length estimation system without depth information, further comprising the step of converting the coordinates (P x , P y ) of the corner pixel of the parcel into 3D real-world coordinates (W x , W y , W z ) corresponding to the coordinates based on the grid corner extracted from the grid plane plate in the case of the XZ plane.
  16. In paragraph 15, A control method for a single-image-based parcel length estimation system without depth information, comprising the step of deriving the three-dimensional dimensions of the parcel, further including the step of estimating the length between corners by utilizing the distance between the real coordinates of the corner pixel in the XY plane and the XZ plane and the origin, and the angle formed by the corner pixel with the origin as dependent variables.
  17. In Paragraph 16, The step of deriving the three-dimensional dimensions of the above-mentioned courier shipment is, in the case of the XY plane (Here, origin (W x0 , W y0 ), real coordinates in the XY plane (W x , W y )), for the XZ plane A control method for a single image-based parcel length estimation system without depth information, further comprising the step of calculating the distance between the real coordinates of the corner pixel and the origin by means of the origin (wherein, origin ( W x0 , W y0 , W z0 ), real coordinates ( W x , W y , W z )) in the XZ plane.
  18. In Paragraph 16, The step of deriving the 3D dimensions of the above-mentioned parcel involves, when calculating the angle that the corner pixel makes with the origin, calculating the angle between the origin (W x 0 , W y 0 ) and the real-world coordinates (W x , W y ) in the 2D plane in the XY plane, and calculating the angle between the unit vector v z of the Z-axis and the vector v of the corner pixel in the 3D vector space in the XZ plane, and in the XY plane , in the XZ plane A control method for a single image-based parcel length estimation system without depth information, comprising an additional step of calculating by
  19. In Paragraph 16, A control method for a single image-based parcel length estimation system without depth information, comprising the step of deriving the three-dimensional dimensions of the parcel, constructing individual data sets in each plane, using the distance between the real coordinates of the corner pixel calculated in the XY plane and the origin, and the angle formed by the corner pixel with the origin as output variables, and using a pair of the corresponding origin (P x 0 , P y 0 ) and the coordinates of the corner pixel (P x , P y ) as input variables, and generating respective data sets Dataset L and Dataset A according to the distance between the real coordinates of the corner pixel and the origin, and the angle formed by the corner pixel with the origin.
  20. In Paragraph 19, A control method for a single-image-based parcel length estimation system without depth information, which further includes the step of deriving the three-dimensional dimensions of the above parcel, wherein the real distance L es between two adjacent pixel coordinates (P x1 , P y1 ) and (P x2 , P y2 ) among the corner pixels is calculated by the following mathematical formula using the distances L 1 and L 2 between the real coordinates of the corner pixel calculated in the XY plane and the origin, and the angles A 1 and A 2 that the corner pixel makes with the origin. [Mathematical Formula] Here,

Description

System for assuming length of delivery goods from low-quality image without depth information and control method thereof The present invention relates to a single image-based delivery cargo length estimation system without depth information and a control method thereof. More specifically, it relates to a single image-based delivery cargo length estimation system without depth information and a control method thereof that enables three-dimensional dimension measurement of a delivery cargo from a single black-and-white image that does not contain depth information and reduces costs by utilizing existing infrastructure and facilitates the dissemination and expansion of the system. As the size of the online market expands due to the recent growth of e-commerce, the demand for delivery of various consumer products is increasing. Consequently, the importance of the courier industry is growing, and its scale is also expanding. Courier companies are employing strategies such as optimizing delivery routes, introducing electric vehicles, and diversifying delivery methods based on cargo characteristics to maximize profits through the provision of efficient delivery services. In particular, recently, when setting freight rate standards, they are attempting to further subdivide rates by additionally incorporating information on cargo size and volume, in addition to the weight of the parcel that was previously considered. In order to utilize information on the size and volume of parcel shipments as a basis for determining freight rates, it is necessary to automate the measurement of shipment volume. To this end, high-performance sensor-based equipment for measuring the three-dimensional dimensions of shipments is being developed, but commercialization and field application are difficult due to the high cost of introducing the equipment. Figure 1 is an example of an image for customer verification of a parcel collected at a general parcel logistics site. FIG. 2 is a block diagram of a single-image-based parcel length estimation system without depth information according to an embodiment of the present invention. FIG. 3 is a diagram for comparing an original image and an image to which an SSR algorithm has been applied according to an embodiment of the present invention. FIG. 4 is a diagram for comparing an original image and an image to which the SOS algorithm has been applied, according to an embodiment of the present invention. FIG. 5 is an example diagram of the relative position definition of pixels corresponding to major corners in the outline of a courier package according to one embodiment of the present invention. FIG. 6 is a diagram illustrating the curvature estimation scale defined in the LPD-CD algorithm according to one embodiment of the present invention. FIG. 7 is a diagram illustrating the operating principle of the LPD-CD algorithm according to one embodiment of the present invention. FIG. 8 is an example of an implementation of the LPD-CD algorithm according to one embodiment of the present invention. FIG. 9 is a diagram for comparing an original image and a corner image detected after applying a CD module according to an embodiment of the present invention. FIG. 10 is a drawing for explaining corner extraction on the XZ and XY planes using a checkerboard according to an embodiment of the present invention. FIG. 11 is a drawing for explaining the calculation of lengths and angles for each point from the origin in each plane according to an embodiment of the present invention. FIG. 12 is a diagram illustrating a neural network-based model structure according to an embodiment of the present invention. FIG. 13 is a control flow diagram of a single image-based parcel length estimation system without depth information according to an embodiment of the present invention. FIG. 14 is a flowchart showing the specific operation process of an OM module according to one embodiment of the present invention. The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but can be implemented in various other forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the claims. Throughout the specification, the same reference numerals refer to the same components. Throughout the specification, when a part is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, terms such as "...part," "...unit," and "module" as used in the specification refer to a unit that processes at least one function or