Search

KR-20260067664-A - APPARATUS AND METHOD FOR INSPECTION GAP OF VEHICLE DESIGN COMPONENTS USING 3D DEPTH SENSOR AND 2D VISION SENSOR

KR20260067664AKR 20260067664 AKR20260067664 AKR 20260067664AKR-20260067664-A

Abstract

The present invention relates to an apparatus and method for inspecting gaps between vehicle interior parts using a 3D depth sensor and a 2D vision sensor. According to the present invention, the apparatus comprises: a sensor data collection unit that acquires 3D point cloud data and 2D vision data by photographing a vehicle inspection area where vehicle interior part objects are assembled using a 3D depth sensor and a 2D vision sensor; an object ROI extraction unit that detects an object area corresponding to each part as a region of interest within the 3D point cloud data based on edge information of each part object extracted from the 2D vision data; an object rotation translation operation unit that acquires object planar data by planarizing the 3D point cloud data for each object area using a setting algorithm, rotates the body planar coordinate system of the object planar data to be horizontal with the camera planar coordinate system of the 2D vision sensor, and then projects the object planar data onto the camera planar coordinate system; and an object distance derivation unit that performs gap inspection between part objects by calculating the distance between the edges of adjacent objects based on the edge coordinates of the object projected onto the camera planar coordinate system.

Inventors

  • 이중희
  • 오상하
  • 유종호

Assignees

  • 재단법인 경북아이티융합 산업기술원

Dates

Publication Date
20260513
Application Date
20241106

Claims (10)

  1. A sensor data acquisition unit that acquires 3D point cloud data and 2D vision data by photographing a vehicle inspection area where vehicle interior part objects are assembled using a 3D depth sensor and a 2D vision sensor; An object ROI extraction unit that detects an object region corresponding to each part as a region of interest within the 3D point cloud data based on edge information of each part object extracted from the 2D vision data; An object rotation and translation operation unit that obtains object planar data by planarizing 3D point cloud data for each of the above object regions using a setting algorithm, rotates and transforms the body planar coordinate system of the object planar data so that it is horizontal with the camera planar coordinate system of the 2D vision sensor, and then projects the object planar data onto the camera planar coordinate system; and A vehicle interior part gap inspection device comprising an object distance derivation unit that performs gap inspection between part objects by calculating the distance between the edges of adjacent objects based on the edge coordinates of an object projected onto the camera planar coordinate system.
  2. In claim 1, The above object ROI extraction unit is, A vehicle interior part gap inspection device that removes unnecessary areas including background areas from the 3D point cloud data using a Z-axis threshold set for the 3D point cloud data, and detects each object area by applying edge information of each object extracted from the 2D vision data to the remaining area after removal.
  3. In claim 1, The above object rotation and movement operation unit is, A vehicle interior part gap inspection device that obtains plane-fitted object plane data by estimating a plane for 3D point cloud data of each object region using a plane fitting algorithm based on the Least Squares method or RANSAC (RANdom SAmple Consensus).
  4. In claim 1, The above object rotation and movement operation unit is, A vehicle interior part gap inspection device that calculates a rotation matrix by applying the angle between the two normal vectors and the rotation axis vector corresponding to the product of the normal vector of the object plane data and the normal vector of the XY plane of the 2D vision sensor to the Rodriguez rotation formula, and rotates the object plane data using the rotation matrix.
  5. In claim 1, The above 3D depth sensor and the above 2D vision sensor are a vehicle interior part gap inspection device that is pre-calibrated through matching between a 3D observation corresponding point observed by the 3D depth sensor and a 2D projection corresponding point observed by the 2D vision sensor.
  6. In a method for inspecting the clearance of vehicle interior parts performed by a vehicle interior part clearance inspection device, A step of acquiring 3D point cloud data and 2D vision data by photographing a vehicle inspection area where vehicle interior part objects are assembled using a 3D depth sensor and a 2D vision sensor; A step of detecting an object region corresponding to each part within the 3D point cloud data as a region of interest based on edge information of each part object extracted from the 2D vision data; A step of obtaining object planar data by planarizing 3D point cloud data for each of the above object regions using a setting algorithm; A step of projecting the object planar data onto the camera planar coordinate system after rotating the vehicle body planar coordinate system of the object planar data so that it is horizontal with the camera planar coordinate system of the 2D vision sensor; and A method for inspecting gaps between vehicle interior parts, comprising the step of calculating the distance between the edges of adjacent objects based on the edge coordinates of the object projected onto the camera plane coordinate system and performing a gap inspection between part objects.
  7. In claim 6, The step of detecting each of the above object regions is, A method for inspecting gaps in vehicle interior parts, which removes unnecessary areas including background regions from the 3D point cloud data using a Z-axis threshold set for the 3D point cloud data, and detects each object region by applying edge information of each object extracted from the 2D vision data to the remaining area after removal.
  8. In claim 6, The step of acquiring the above object plane data is, A method for inspecting gaps in vehicle interior parts, which obtains plane-fitted object plane data by estimating a plane for 3D point cloud data of each object region using a plane fitting algorithm based on the Least Squares or RANSAC (RANdom SAmple Consensus).
  9. In claim 6, The above rotational transformation step is, A method for inspecting gaps in vehicle interior parts, wherein a rotation axis vector corresponding to the product of the normal vector of the object plane data and the normal vector of the XY plane of the 2D vision sensor and the angle between the two normal vectors are applied to the Rodriguez rotation formula to calculate a rotation matrix, and the object plane data is rotated using the rotation matrix.
  10. In claim 6, A method for inspecting gaps in vehicle interior parts, further comprising the step of pre-calibrating the 3D depth sensor and the 2D vision sensor through matching between a 3D observation corresponding point observed by the 3D depth sensor and a 2D projection corresponding point observed by the 2D vision sensor.

Description

Apparatus and Method for Inspecting Gap of Vehicle Design Components Using 3D Depth Sensor and 2D Vision Sensor The present invention relates to an apparatus and method for inspecting gaps between vehicle parts using a three-dimensional depth sensor and a two-dimensional vision sensor, and more specifically, to an apparatus and method for inspecting gaps between vehicle parts using a three-dimensional depth sensor and a two-dimensional vision sensor that can efficiently perform gap inspection between vehicle parts in an automobile parts assembly process. The vehicle assembly process is the final stage in which units such as interior and exterior trim, electrical components, engines, transmissions, and axles are assembled and installed onto the painted body to complete the vehicle product and verify its quality, thereby finalizing it as a marketable item. The automotive production system must be established to manufacture the maximum quantity within the target timeframe. In the vehicle assembly process, gap inspections are performed between vehicle parts (e.g., chassis, glass, headlamps) to determine the assembly status of the vehicle. However, as conventional gap inspections simply utilize 2D images or are performed directly by personnel, quality problems continue to occur due to inaccuracies in 2D images caused by reflections or curvature of the vehicle body, as well as deviations caused by human error. As such, in conventional cases, errors caused by human error during gap inspection are significant, and gap inspection systems implemented to date require calculations of too many points, resulting in increased computational load and problems of vehicle assembly delays due to system errors. The technology forming the background of the present invention is disclosed in Korean Published Patent No. 10-2024-0106949 (published July 8, 2024). FIG. 1 is a diagram illustrating the configuration of a vehicle interior part gap inspection system using a three-dimensional depth sensor and a two-dimensional vision sensor according to an embodiment of the present invention. FIG. 2 is a diagram showing the configuration of a vehicle interior component clearance inspection device illustrated in FIG. 1. Figure 3 is a drawing illustrating a method for inspecting the clearance of vehicle interior parts using the device of Figure 2. FIG. 4 is a diagram exemplarily showing data for a vehicle inspection area collected from a three-dimensional depth sensor in an embodiment of the present invention. FIG. 5 is a diagram exemplarily showing three-dimensional point cloud data for a remaining part object area after removing background and unnecessary objects in an embodiment of the present invention. FIG. 6 is a diagram showing the optimal normal vector obtained by applying a planar fitting algorithm to three-dimensional point cloud data for a part object in an embodiment of the present invention. FIG. 7 is a diagram illustrating the concept of rotational transformation between the vehicle body plane coordinate system and the camera plane coordinate system in an embodiment of the present invention. FIG. 8 is a diagram illustrating the process of projecting rotationally transformed object plane data onto a camera plane coordinate system in an embodiment of the present invention. Figure 9 is a diagram showing the gap measurement error when projected without performing rotation transformation. Then, with reference to the attached drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification have been given similar reference numerals. Throughout the specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected," but also cases where they are "electrically connected" with other components interposed between them. Furthermore, when a part is described as "including" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. FIG. 1 is a diagram illustrating the configuration of a vehicle interior part gap inspection system using a three-dimensional depth sensor and a two-dimensional vision sensor according to an embodiment of the present invention. As shown in FIG. 1, the vehicle interior part gap inspection system according to an embodiment of the present invention includes a vehicle interior part gap inspection device (100), a three-dimensional depth sensor (200), and a two-dimensional depth sensor (300). The vehicle interior part gap inspection device (100) is connected to the 3D dep