CN-121981937-A - Method for monitoring fastening sequence of automobile tire nuts

CN121981937ACN 121981937 ACN121981937 ACN 121981937ACN-121981937-A

Abstract

The invention discloses a method for monitoring the fastening sequence of automobile tire nuts, which relates to the technical field of industrial automatic assembly monitoring, and comprises the following steps of camera calibration in the pretreatment stage; the method comprises the steps of placing a tightening gun bound with a marking plate on a calibration tool, collecting multi-frame images, solving to obtain relative position parameters of a gun head and the marking plate, and sequentially recording coordinates of each target nut under a camera coordinate system according to nut fastening sequences. The real-time monitoring stage comprises the steps of image acquisition, solving of the gesture of a marking plate under a camera coordinate system, calculation of real-time coordinates of a gun head under the camera coordinate system by combining of the gesture of the marking plate under the camera coordinate system and relative position parameters, comparison of the gun head real-time coordinates under the camera coordinate system and target nut coordinates, enabling screwing of the gun to execute fastening operation if the position deviation is within a preset threshold value, and otherwise enabling no screwing of the gun. The invention realizes high-precision monitoring and error-proofing control on the nut fastening process, and improves the assembly quality and efficiency.

Inventors

LIN CHUANWEN
WANG DINGMIN
LIU RUI
PENG CHAO
CHEN FANGFANG
CUI HAIYING
LU SHENGGAN
XING JING

Assignees

合肥中安数据科技有限公司

Dates

Publication Date: 20260505
Application Date: 20251205

Claims (10)

1. The method for monitoring the fastening sequence of the automobile tire nuts is characterized by comprising the following steps of: s1, a pretreatment stage: S11, camera calibration, namely acquiring internal parameters, external parameters and distortion parameters of a camera by adopting a checkerboard calibration plate through multi-pose image acquisition, angular point extraction, parameter estimation and optimization; S12, calibrating the relative position of the gun head and the marking plate, namely placing a tightening gun bound with the marking plate on a calibration tool, collecting multi-frame images by a camera, and solving to obtain relative position parameters of the gun head and the marking plate, wherein the relative position parameters comprise a rotation matrix R ref and a translation vector t ref ; s13, teaching storage, namely sequentially recording the coordinates of each target nut under a camera coordinate system according to the nut fastening sequence; S2, a real-time monitoring stage: s21, acquiring images in the process of fastening the automobile tire nuts in real time by a camera, and detecting and identifying tires and a marking plate positioned on a tightening gun; s22, estimating the gesture of the marking plate, namely solving the gesture of the marking plate under a camera coordinate system based on a 6D gesture estimation method; S23, calculating the position of the gun head, namely combining the gesture of the marking plate under the camera coordinate system and the relative position parameter of the gun head and the marking plate, and calculating to obtain the real-time coordinate of the gun head under the camera coordinate system; s24, error proofing judgment, namely comparing real-time coordinates of the gun head under a camera coordinate system with coordinates of the target nut under the camera coordinate system, if the position deviation is within a preset threshold value, enabling the gun to be screwed down to execute fastening operation, and otherwise, enabling the gun not to be screwed down to execute fastening operation.
2. The method for monitoring the tightening sequence of automobile tire nuts according to claim 1, wherein the camera calibration of step S11 specifically comprises: s111, a checkerboard calibration plate is adopted as a calibration object; S112, adjusting the spatial position of a calibration object, and shooting a plurality of groups of images with different postures; s113, extracting angular point coordinates of the checkerboard from the image by adopting an angular point detection algorithm; S114, based on a pinhole camera model, primarily estimating internal parameters and external parameters under ideal imaging conditions; S115, taking radial distortion in an actual imaging process into consideration, and estimating distortion parameters by adopting a least square method; s116, optimizing the internal parameters, the external parameters and the distortion parameters by adopting a maximum likelihood estimation method.
3. The method for monitoring the fastening sequence of nuts of automobile tires according to claim 1, wherein the calibrating of the relative position of the gun head and the marking plate in the step S12 specifically comprises: s121, placing the calibration tool under a camera, firstly, not placing the tightening gun, identifying the circle center of a middle cylinder of the calibration tool to obtain the position of the circle center under a pixel coordinate system, and converting the circle center under the camera coordinate system according to internal parameters of the camera to obtain the position of the circle center under the camera coordinate system; s122, binding a marking plate on the tightening gun, selecting one point from marking points of the marking plate as a reference point, obtaining the position of the reference point under a pixel coordinate system, and converting the reference point under a camera coordinate system according to an internal reference of the camera, so as to obtain the position of the reference point under the camera coordinate system; S123, marking the position of a reference point under a camera coordinate system as x1, marking the position of a gun head as x2, meeting x2=R ref ×x1+t ref between the two, and acquiring for multiple times to obtain the position relation between a plurality of pairs of gun heads and the reference point, thereby solving a rotation matrix R ref and a translation vector t ref .
4. The method for monitoring the tightening sequence of automobile tire nuts as claimed in claim 1, wherein the estimation of the posture of the marking plate in step S22 is specifically as follows: s221, identifying mark points on the mark plate, and constructing an initial coordinate system according to the relative positions of the mark points; s222, establishing an object coordinate system with the center of the marking plate as an origin, and acquiring fixed coordinates of the marking point under the object coordinate system; s223, establishing a camera coordinate system with a camera optical center as an origin; s224, constructing a central coordinate system, wherein the origin of the central coordinate system coincides with the center of the marking plate, and the base vector is consistent with the camera coordinate system; S225, deducing a rotation matrix R through the coordinate relation of the mark points under the object coordinate system and the central coordinate system; S226, obtaining a translation vector t according to the original point displacement of the center coordinate system and the camera coordinate system; s227, the 6D pose of the marker panel in the camera coordinate system is described together using the rotation matrix R and the translation vector t.
5. The method for monitoring the fastening sequence of the automobile tire nuts according to claim 1 is characterized in that in the real-time monitoring stage, the identification and track tracking of the tightening tool are further carried out, the identification result of the tightening tool is verified at the same time when error prevention judgment is carried out, if the position deviation is within a preset threshold value and the tightening tool is in compliance, the tightening gun is enabled to execute the fastening operation, and otherwise, the tightening gun is not enabled to execute the fastening operation.
6. The method of claim 5, wherein the improved YOLOv network identifies the tightening tool in the image and the improved YOLOv network uses MobileNetV as the backbone network.
7. The method for monitoring the fastening sequence of the automobile tire nuts according to claim 5, wherein the track of the movement of the tightening tool is tracked through Deepsort algorithm, the track is tracked, the position of the tightening tool is predicted through Kalman filtering algorithm, and the object association is achieved by combining the apparent feature similarity with the movement feature similarity.
8. An electronic device comprising a processor, an input device, an output device and a memory, the processor, the input device, the output device and the memory being connected in sequence, the memory being for storing a computer program, the computer program comprising program instructions, the processor being configured to invoke the program instructions to perform a method of monitoring a nut tightening sequence of an automotive tyre as claimed in any one of claims 1 to 7.
9. A readable storage medium, characterized in that the storage medium stores a computer program comprising program instructions which, when executed by a processor, cause the processor to perform a method of monitoring the tightening sequence of automobile tyre nuts as claimed in any one of claims 1 to 7.
10. A computer program, characterized in that the computer program comprises program instructions, the processor being configured to invoke the program instructions for performing a method for monitoring the tightening sequence of nuts of a vehicle tyre according to any one of claims 1-7.

Description

Method for monitoring fastening sequence of automobile tire nuts Technical Field The invention relates to the technical field of industrial automatic assembly monitoring, in particular to a method for monitoring the fastening sequence of nuts of automobile tires. Background In the field of high-precision assembly such as automobile manufacturing, the fastening sequence and the position accuracy of nuts directly influence the assembly quality and the service life of products. The traditional nut fastening monitoring mode is mostly dependent on manual visual observation or simple mechanical limiting, and has the problems of low monitoring precision, easy error, incapability of realizing full-flow traceability and the like. With the development of industrial automation technology, a monitoring scheme based on machine vision is gradually applied to the field of assembly, but the existing scheme has the defects that firstly, camera calibration accuracy is insufficient, so that position calculation errors are large, secondly, a tool posture estimation method is complex, instantaneity is poor, thirdly, the stability of tool identification and track tracking is insufficient and is easily interfered by industrial environment, and thirdly, an integrated error prevention mechanism is lacked, and the control of a fastening tool cannot be effectively linked. Therefore, a device and a method for monitoring the nut tightening sequence with high accuracy and high stability are needed to solve the problems of the prior art. Disclosure of Invention In order to overcome the defects in the prior art, the invention provides a method for monitoring the fastening sequence of the automobile tire nuts, which realizes high-precision monitoring and error-proofing control on the nut fastening process and improves the assembly quality and efficiency. In order to achieve the above purpose, the present invention adopts the following technical scheme, including: a method for monitoring the fastening sequence of automobile tire nuts comprises the following steps: s1, a pretreatment stage: S11, camera calibration, namely acquiring internal parameters, external parameters and distortion parameters of a camera by adopting a checkerboard calibration plate through multi-pose image acquisition, angular point extraction, parameter estimation and optimization; S12, calibrating the relative position of the gun head and the marking plate, namely placing a tightening gun bound with the marking plate on a calibration tool, collecting multi-frame images by a camera, and solving to obtain relative position parameters of the gun head and the marking plate, wherein the relative position parameters comprise a rotation matrix R ref and a translation vector t ref; s13, teaching storage, namely sequentially recording the coordinates of each target nut under a camera coordinate system according to the nut fastening sequence; S2, a real-time monitoring stage: s21, acquiring images in the process of fastening the automobile tire nuts in real time by a camera, and detecting and identifying tires and a marking plate positioned on a tightening gun; s22, estimating the gesture of the marking plate, namely solving the gesture of the marking plate under a camera coordinate system based on a 6D gesture estimation method; S23, calculating the position of the gun head, namely combining the gesture of the marking plate under the camera coordinate system and the relative position parameter of the gun head and the marking plate, and calculating to obtain the real-time coordinate of the gun head under the camera coordinate system; s24, error proofing judgment, namely comparing real-time coordinates of the gun head under a camera coordinate system with coordinates of the target nut under the camera coordinate system, if the position deviation is within a preset threshold value, enabling the gun to be screwed down to execute fastening operation, and otherwise, enabling the gun not to be screwed down to execute fastening operation. Preferably, the camera calibration in step S11 specifically includes: s111, a checkerboard calibration plate is adopted as a calibration object; S112, adjusting the spatial position of a calibration object, and shooting a plurality of groups of images with different postures; s113, extracting angular point coordinates of the checkerboard from the image by adopting an angular point detection algorithm; S114, based on a pinhole camera model, primarily estimating internal parameters and external parameters under ideal imaging conditions; S115, taking radial distortion in an actual imaging process into consideration, and estimating distortion parameters by adopting a least square method; s116, optimizing the internal parameters, the external parameters and the distortion parameters by adopting a maximum likelihood estimation method. Preferably, the calibrating of the relative position of the gun head and the marking plate in the step S12 specifically inc