Search

CN-121982100-A - Automatic positioning method for mounting hole site of cabinet type integral hardware based on image recognition

CN121982100ACN 121982100 ACN121982100 ACN 121982100ACN-121982100-A

Abstract

The invention discloses an automatic positioning method for mounting hole sites of cabinet type integral hardware based on image recognition, which comprises the steps of obtaining a multi-view image set of a cabinet type workpiece, wherein the multi-view image set comprises a two-dimensional image and a depth image, and obtaining a standardized image set through noise filtering, distortion correction and region enhancement pretreatment; the method comprises the steps of constructing a three-dimensional semantic model containing cabinet structural features and preset distribution rules of hole sites, dividing candidate areas of the hole sites from the model, extracting geometric, texture and space association features to determine preliminary coordinates of the hole sites, optimizing calibration coordinates by combining the preset rules, and outputting three-dimensional coordinates of a hole site target. The method realizes the full-flow automation of hole site positioning, adapts to multiple types of hole sites and complex counter surfaces, ensures positioning precision and provides support for automatic installation of hardware.

Inventors

  • LIU GUANGQUAN
  • LV NANA

Assignees

  • 呼伦贝尔海拉尔区鑫羽铭科技有限公司

Dates

Publication Date
20260505
Application Date
20260127

Claims (10)

  1. 1. The automatic positioning method for the installation hole site of the cabinet type integral hardware based on image recognition is characterized by comprising the following steps: acquiring a multi-view image set of a cabinet workpiece, wherein the multi-view image set comprises two-dimensional images of different surfaces of the cabinet workpiece and corresponding depth images; Preprocessing the multi-view image set to obtain a standardized image set, wherein the preprocessing comprises image noise filtering, geometric distortion correction and hole site area enhancement; constructing a three-dimensional semantic model of the cabinet workpiece based on the standardized image set, wherein the three-dimensional semantic model comprises structural features of the cabinet workpiece and preset distribution rules of hardware installation holes; Performing feature extraction and identification on a hole site candidate region in the three-dimensional semantic model to determine a preliminary coordinate of a hole site, wherein the feature extraction comprises geometric features, texture features and space associated features of the hole site; And carrying out optimization calibration on the hole site coordinates based on the preliminary coordinates of the hole sites and a preset distribution rule in the three-dimensional semantic model to obtain target three-dimensional coordinates of the hole sites.
  2. 2. The automatic positioning method for installing holes of integral cabinet hardware based on image recognition according to claim 1, wherein the acquiring the multi-view image set of the cabinet workpiece comprises: An image acquisition system consisting of a binocular industrial camera and a monocular close-up camera is adopted, the baseline distance of the binocular industrial camera is set to be a first preset distance range, the resolution is a first preset resolution, and the frame rate is a first preset frame rate range; Fixing the cabinet workpiece on a rotary workbench, setting the stepping angle of the rotary workbench to be a preset stepping angle range, and synchronously triggering a binocular industrial camera and a monocular close-up camera to acquire images when rotating by one stepping angle; after the acquisition is completed, the time sequence alignment is carried out on the binocular image and the monocular close-up image under the same angle, the alignment error is controlled within a preset error threshold value, and a multi-view image set containing preset groups of different view angles is formed.
  3. 3. The automatic positioning method for mounting hole sites of cabinet type integral hardware based on image recognition according to claim 1, wherein the filtering of noise in the image adopts an adaptive bilateral filtering algorithm, and an updating formula of pixel gray values of the adaptive bilateral filtering is as follows: , wherein, For filtered pixels Is used for the gray-scale value of (c), For the gray value of the original pixel, To take the following measures Is a neighborhood of the center of the circle, In order to normalize the weights, the weights are, The spatial standard deviation is a spatial domain standard deviation, the range of the spatial standard deviation is valued, Is the standard deviation of the gray-scale domain, a value gray standard deviation range; Dynamic adjustment according to local texture complexity of image And (3) with Local texture complexity Above the threshold value of the complexity, Taking a first spatial standard deviation range, Taking the first gray standard deviation range Less than or equal to the complexity threshold value, Taking a second spatial standard deviation range, Taking a second gray standard deviation range, and the local texture complexity The contrast characteristic calculation through the gray level co-occurrence matrix is as follows , 、 For the number of gray levels, Is a gray level co-occurrence matrix element.
  4. 4. The automatic positioning method for mounting hole sites of cabinet type integral hardware based on image recognition according to claim 1, wherein the geometric distortion correction of the image through camera calibration parameters comprises: collecting calibration plate images with different angles of a preset calibration image group number by using a checkerboard calibration plate with a preset specification and a checkerboard size as a standard size through a Zhengyou calibration method; calculating camera internal reference matrix through calibration And distortion coefficient Internal reference matrix Wherein 、 Respectively the focal length of the cameras, 、 Respectively the principal point coordinates and distortion coefficients , The first, second and third radial distortion coefficients are respectively, The first tangential distortion coefficient and the second tangential distortion coefficient are respectively; And carrying out distortion correction on the original image based on the internal reference matrix and the distortion coefficient, wherein a correction formula is as follows: The correction formula is: , , wherein, For the image pixel coordinates, In order to correct the post-pixel coordinates, , Is the distance of the distorted image pixel to the center point of the photograph.
  5. 5. The automatic positioning method for mounting hole sites of integral cabinet hardware based on image recognition according to claim 1or 4, wherein the constructing a three-dimensional semantic model of the cabinet workpiece based on the standardized image set comprises: performing three-dimensional matching on binocular image pairs in a standardized image set, and setting a matching window size as a preset matching window size range and a parallax range as a preset parallax range by adopting a semi-global block matching algorithm; calculating three-dimensional point cloud coordinates according to the parallax map obtained by stereo matching and combining the camera internal reference and baseline distance ) The formula is: , , , wherein, For the image pixel coordinates, Is the base line distance of the binocular camera, For the parallax to be a good visual indication, 、 Respectively is Pixel physical size; Registering and fusing the multi-view point cloud, setting a registration error threshold value as a first registration error threshold value range by adopting an iterative nearest point algorithm, and adding semantic tags after fusion, wherein the semantic tags comprise a cabinet door, a side plate, a top plate, a hinge hole area and a handle hole area to form a three-dimensional semantic model.
  6. 6. The automatic positioning method for installing hole sites of integral cabinet hardware based on image recognition according to claim 1, wherein the feature extraction and recognition of the hole site candidate regions in the three-dimensional semantic model comprises the following steps: Dividing a hole site candidate region from the three-dimensional semantic model through an improved region growing algorithm; extracting geometric parameters of a hole site candidate region, including preliminary two-dimensional coordinates of a hole diameter, a hole depth and a hole site center; Calculating depth information of the hole site candidate region by combining the depth image, and generating preliminary three-dimensional coordinates of the hole site; And matching the spatial correlation characteristics of the hole sites with a preset distribution rule, removing pseudo hole sites which do not accord with the distribution rule, and reserving the primary coordinates of the real hole sites.
  7. 7. The automatic positioning method for installing hole sites of cabinet type integral hardware based on image recognition according to claim 6, wherein the hole site candidate areas segmented in the three-dimensional semantic model are segmented by an improved area growing algorithm, specifically: the improved region growing algorithm takes pixels with gray values smaller than a first gray threshold value and included angles between normal vectors and normal vectors on the surface of a workpiece in a three-dimensional semantic model larger than a first angle threshold value as seed points; the growth criterion is Euclidean distance between the point to be grown and the seed point Is smaller than the first distance threshold value and has a gray level difference Is smaller than a first gray level difference threshold value, and the included angle between the normal vector of the point to be grown and the normal vector of the seed point Less than a second angular threshold; Dynamically adjusting a growth threshold in the growth process, and when the number of pixels in the region exceeds a first pixel number threshold, thresholding the Euclidean distance Reducing to a second distance threshold, and gray level difference threshold Reducing to a second gray level difference threshold; and after the growth is finished, performing morphological closing operation on each growth region, and removing holes in the region to obtain hole site candidate regions.
  8. 8. The automatic positioning method for installing hole sites of integral cabinet hardware based on image recognition according to claim 6, wherein the hole site candidate region comprises circular hole sites and rectangular hole sites, and the geometric parameters of the extracted hole site candidate region are as follows: for a circular hole site, calculating the aperture and the center coordinate by adopting a least square circle fitting algorithm, wherein the fitting formula is as follows: , wherein, Is the coordinates of the hole site edge pixels, Is the center of the circle coordinate, In the form of a radius of the pipe, For the total number of circular hole edge pixels, the aperture is obtained by minimizing through iterative solution Pore diameter Calculating the error is less than or equal to the first a geometric error threshold; For rectangular hole sites, calculating the length of the rectangle by adopting a minimum circumscribed rectangle algorithm Width of the steel sheet And the center coordinates The formula is: , , , wherein, Is the coordinates of the hole site edge pixels, 、 Pixels at rightmost side of lateral edges of hole sites respectively The coordinates of the two points of the coordinate system, 、 Pixels at leftmost side of lateral edges of hole sites respectively The coordinates, length and width calculation errors are less than or equal to a second geometric error threshold.
  9. 9. The automatic positioning method for installing hole sites of integral cabinet hardware based on image recognition according to claim 1, wherein the matching of the spatial correlation characteristics of the through hole sites with the preset distribution rules comprises the following steps: The hole site space correlation characteristic is expressed by Euclidean distance matrix among holes, and is expressed by Candidate areas of each hole site, distance matrix , As matrix elements, elements , Is the first The candidate region of each hole site to the first Three-dimensional euclidean distance of the individual hole site candidate regions, 、 Is the first Or (B) or (C) Preliminary three-dimensional coordinates of the individual hole sites; the preset distribution rules comprise equidistant distribution rules of hinge holes and symmetrical distribution rules of handle holes, and the matching degree is calculated by adopting cosine similarity, wherein the formula is as follows: , wherein, Is a hole site distance matrix in a preset distribution rule when And when Sim is smaller than the first similarity threshold, determining a pseudo hole site and eliminating the pseudo hole site.
  10. 10. The automatic positioning method for installing hole sites of integral cabinet hardware based on image recognition according to claim 1 or 9, wherein the optimizing calibration of the hole site coordinates based on preliminary coordinates of the hole sites and preset distribution rules in the three-dimensional semantic model comprises: constructing a coordinate optimization objective function: , wherein, Is a preliminary three-dimensional coordinate of the hole site, For the three-dimensional coordinates of the object, The hole site spacing calculated for the preliminary coordinates, In order to set the distance between the two electrodes to be equal, As the weight coefficient of the light-emitting diode, The number of the true hole sites; Solving an objective function by adopting a gradient descent method, setting a learning rate as a first learning rate range, and setting the iteration times as a first iteration times range until the coordinate variation before and after the iteration is smaller than a first coordinate variation threshold; And calculating the standard deviation of the hole site coordinates after optimization, outputting the target three-dimensional coordinates when the standard deviation is smaller than a first standard deviation threshold value, and otherwise, re-executing the feature extraction.

Description

Automatic positioning method for mounting hole site of cabinet type integral hardware based on image recognition Technical Field The invention relates to the technical field of machine vision positioning, in particular to an automatic positioning method for mounting hole sites of cabinet type integral hardware based on image recognition. Background In the field of mass production of customized furniture and cabinet products, the mounting precision of hardware (such as hinges, handles, sliding rails and the like) directly determines the assembly quality and use experience of cabinet workpieces, the positioning of mounting hole positions of traditional cabinet hardware depends on manual operation, workers need to measure marked hole position coordinates on the surface of a cabinet body by means of tools such as measuring tapes, calipers and the like, the efficiency is low (such as positioning of the hole positions of tens of cabinet bodies can only be completed on a single day), the positioning is easily influenced by manual operation errors (such as reading deviation and marking offset), the hole position precision is difficult to control, the problems of hardware assembly clamping and cabinet door closing are not tight and the like particularly occur when the hole position interval errors exceed 0.5mm, the production cost is greatly increased due to the fact that reworking and correction are needed later, and the traditional manual positioning mode cannot adapt to the core requirements of modern production lines on high precision and high efficiency as the requirements of consumers on customized cabinet products are met from functions to refined experience and upgrade. While hole location techniques based on machine vision have emerged in the industry today, most solutions still have significant limitations. In addition, although three-dimensional modeling is tried to be introduced, a filtering algorithm with fixed parameters is adopted in an image preprocessing stage, so that texture features (such as solid wood textures, veneer textures, baking varnish reflecting surfaces and the like) of the surface of a cabinet workpiece are difficult to adapt, noise filtering is not thorough or the details of hole positions are blurred, the accuracy of the segmentation and feature extraction of a candidate region of a subsequent hole position is greatly reduced, and the false hole position false recognition rate is extremely high. In the prior art, in the hole position identification process, the utilization of the distribution rules is often ignored, the effectiveness of the hole positions is judged only through the geometric features (such as aperture and shape) of a single hole position, the pseudo hole positions formed by scratches, scars and processing flaws on the surface of a cabinet body cannot be removed, in addition, part of the technology introduces space correlation features, but in the distance matrix calculation and matching degree evaluation stage, a three-dimensional space characteristic optimization algorithm of the cabinet hole positions is not adopted, the matching degree calculation error is larger, and when the number of the hole positions exceeds a plurality of hole positions, the identification accuracy of the real hole positions is obviously reduced, and the positioning requirements of complex cabinet workpieces (such as multi-door cabinets and combined cabinets) are difficult to meet. Chinese patent (CN 120543632A) discloses a positioning method and a positioning system for mounting holes, wherein the searching range is reduced by means of a geometric model and a physical size preset by a measured part, invalid edge interference caused by full-image detection is avoided by minimum surrounding rectangular frames, positioning efficiency and robustness are improved, angular point coordinates are calculated firstly based on a conversion relation between the physical size and pixel size through a two-step strategy of coarse positioning and partial ROI fine positioning, then the angular point coordinates are accurately positioned in a partial area by using a HoughCircle algorithm, adaptability to illumination change and partial shielding is high, deep learning training is not needed, hardware deployment cost is low and implementation is easy, but obvious defects are caused that if the part is excessively dependent on the geometric model and the physical size of the measured part, if machining errors exist or the geometric model is not matched with an actual workpiece, positioning errors are directly caused, meanwhile, only for single type mounting hole designs, space correlation characteristics among holes are not considered, porous collaborative positioning scenes are difficult to cope, scratches, scars and other pseudo holes cannot be effectively removed, and applicability in complex hole scenes is limited in porous position and multi-type hole scenes. Chinese patent (CN 119313729A) discloses a scr