CN-121981355-A - Symmetrical workpiece flexible tightening system and method based on vision and self-adaptive mirror image

CN121981355ACN 121981355 ACN121981355 ACN 121981355ACN-121981355-A

Abstract

The invention relates to the technical field of industrial automation and discloses a symmetrical workpiece flexible tightening system and method based on vision and self-adaptive mirror image, wherein the system comprises a positioning and fixing module, a vision acquisition module, a flexible tightening control module and a tightening operation module; the flexible tightening control module is used for identifying single-side tightening point coordinates and synchronously carrying out dynamic calibration on an actual symmetrical plane of the workpiece, planning an optimal tightening path considering efficiency and safety, generating a tightening operation instruction on the other side through self-adaptive mirror image transformation based on the actual symmetrical plane, and controlling the tightening operation module to output so as to realize double-side flexible tightening operation of the workpiece with the symmetrical structure. The invention can realize simultaneous screwing of two sides of a machined workpiece, gets rid of the traditional method of calculating screwing points and presetting paths in advance, plans different optimal paths when facing different symmetrical structure workpieces, realizes flexible screwing, and has the advantages of high production efficiency, strong universality, good flexibility adaptability of production and the like.

Inventors

LIU MINGZHOU
CAO BINGKUN
HU JING
GE MAOGEN
LING LIN
ZHANG XI

Assignees

合肥工业大学

Dates

Publication Date: 20260505
Application Date: 20260123

Claims (10)

1. A vision and adaptive mirror image based symmetric workpiece flexible tightening system, comprising: the positioning and fixing module is used for fixing the symmetrical workpiece to be processed after the flexible tightening control module receives the tray in-place signal; The visual acquisition module is used for acquiring visual image data of the symmetrical workpiece to be processed and transmitting the visual image data to the flexible tightening control module; The flexible tightening control module is used for coordinating the cooperative work of the modules, receiving external IO signals and visual image data, identifying single-side tightening point coordinates, synchronously carrying out dynamic calibration on the actual symmetrical plane of the workpiece, planning an optimal tightening path considering efficiency and safety, generating a tightening operation instruction on the other side through self-adaptive mirror image transformation based on the actual symmetrical plane, and controlling the output of the tightening operation module so as to realize double-side flexible tightening operation of the workpiece with the symmetrical structure; the tightening operation module is used for receiving a tightening control instruction of the flexible tightening control module and simultaneously implementing tightening operation on two sides of the symmetrical workpiece to be processed according to the tightening control instruction; The flexible tightening control module is in signal connection with the positioning fixing module and the tightening operation module, is in communication connection with the vision acquisition module, and can accurately realize flexible tightening operation on various symmetrical products through cooperation of the positioning fixing module, the vision acquisition module, the flexible tightening control module and the tightening operation module.
2. The symmetrical workpiece flexible tightening system based on visual and adaptive mirroring according to claim 1, wherein the flexible tightening control module comprises a visual processing unit, a path planning unit and a motion control unit; the visual processing unit is used for receiving visual image data and preprocessing, identifying single-side tightening point coordinates and synchronously carrying out dynamic calibration on the actual symmetry plane of the workpiece so as to realize online intelligent compensation on machining and installation errors; The path planning unit is used for planning an optimal tightening path considering efficiency and safety by utilizing a multi-target collaborative optimization algorithm and combining a single-side tightening point coordinate, and generating an optimal tightening path on the other side by self-adaptive mirror image transformation based on an actual symmetry plane to obtain an optimal tightening path for double-side operation; the motion control unit is used for generating a tightening operation instruction of double-side synchronous operation based on an optimal tightening path of double-side operation and combining a master-slave synchronous control strategy, and controlling the tightening operation module to output according to the tightening operation instruction of the double-side synchronous operation so as to realize double-side flexible tightening operation of a symmetrical structure workpiece.
3. The symmetrical workpiece flexible tightening system based on vision and self-adaptive mirror image according to claim 2, wherein the vision processing unit is used for receiving the vision image data and preprocessing, identifying the coordinates of the single-side tightening points and synchronously carrying out dynamic calibration on the actual symmetry plane of the workpiece so as to realize on-line intelligent compensation of machining and installation errors, and comprises the following steps: Receiving visual image data, performing convolution denoising processing on the image to improve the quality of the visual image, and performing normalization processing on pixel values in the visual image to improve the data stability; Uniformly scaling the normalized visual image to a preset fixed size to complete size standardization so as to match the input requirement of a target detection model based on deep learning; Identifying and positioning pixel coordinates of tightening points of all bolts in the standardized visual image by using a target detection model based on deep learning; based on a coordinate transformation relation, mapping the pixel coordinates of the tightening points from a pixel coordinate system to a world coordinate system to generate a unilateral tightening point coordinate set under a three-dimensional world coordinate system, wherein the expression of the coordinate transformation relation is as follows: ; ; ; Wherein p world is world coordinates, H is homography matrix obtained based on hand-eye calibration, p pixel is pixel coordinates, For proportional symbols, T is a transposed symbol, (X, Y, Z) is three-dimensional world coordinates, To tighten the dot pixel coordinates; determining a three-dimensional coordinate point set required by fitting a symmetrical plane by using a monocular vision pose estimation technology, and carrying out anti-interference treatment on the three-dimensional coordinate point set by combining an anti-interference fitting algorithm to determine an actual symmetrical plane of a symmetrical workpiece to be processed; and taking the actual symmetry plane of the symmetrical workpiece to be processed as a reference of mirror image transformation, and realizing online intelligent compensation of processing and installation errors.
4. The symmetrical workpiece flexible tightening system based on vision and self-adaptive mirror image as claimed in claim 3, wherein the object detection model based on deep learning automatically learns and extracts visual characteristics of bolts in a layered manner through a main network, adopts a multi-scale characteristic fusion mechanism to fuse semantic information of deep characteristics with detail information of shallow characteristics for processing bolts with different sizes, simultaneously introduces an attention enhancement mechanism, takes a feature map in forward propagation as input, generates an attention weight map through coordinate information embedding and feature coding, combines weighted calculation to obtain a new feature map enhanced by the attention mechanism so as to enhance the feature response of key areas of the bolts and inhibit background interference, and outputs surrounding frame parameters of each bolt through a boundary frame regression algorithm at the head of the object detection model based on the deep learning to obtain pixel coordinates, width and height of tightening points and complete positioning; The weighted calculation of the attention weight graph and the original feature graph is expressed as follows: ; Wherein F enhanced is a new feature map enhanced by an attention mechanism, F is a feature map in forward propagation, For the multiplication on an element-by-element basis, The function is activated for Sigmoid, and a is the attention weight graph.
5. A vision and adaptive mirror image based symmetric workpiece flexible tightening system as claimed in claim 3, wherein the determining the three-dimensional coordinate point set required for fitting the symmetric plane by monocular vision pose estimation technology, and performing anti-interference processing on the three-dimensional coordinate point set in combination with an anti-interference fitting algorithm, determining the actual symmetric plane of the symmetric workpiece to be processed comprises: establishing a camera projection model by using pre-calibrated camera internal parameters and distortion parameters, and establishing a perspective projection mapping relation among a pixel coordinate system, an image physical coordinate system and a camera coordinate system; acquiring pixel coordinates of the feature points in the visual image, taking the known physical size of the feature points as constraint based on a perspective mapping relation, constructing a 3D-2D projection equation by utilizing a PnP algorithm, minimizing Euclidean distance errors of the re-projection of the feature points to an image plane by an iterative algorithm, and solving the initial three-dimensional pose of each feature point relative to a camera; Constructing a reprojection cost function taking the workpiece pose and the space coordinates of the characteristic points as variables by using an initial three-dimensional pose as an initial value and utilizing a beam method adjustment, and carrying out joint optimization on the space observation values of a plurality of characteristic points by minimizing the square sum of global reprojection errors; Three non-collinear points are randomly extracted from a three-dimensional coordinate point set to serve as a minimum sample set to generate a candidate plane equation, normal line distances from all points in the three-dimensional coordinate point set to a candidate plane are calculated, the number of internal points with the statistical distance smaller than a preset deviation threshold value is counted by using an evaluation function, a candidate plane with the largest number of internal points and the smallest sum of squares of the distances from the internal points to the plane is searched through cyclic iteration, and the candidate plane is determined to be an actual symmetrical plane of a symmetrical workpiece to be processed.
6. The vision-and-adaptive mirror-image-based symmetrical workpiece flexible tightening system according to claim 2, wherein the path planning unit, when using a multi-objective collaborative optimization algorithm to plan an optimal tightening path considering both efficiency and safety in combination with a single-side tightening point coordinate, and generating an optimal tightening path of the other side by adaptive mirror image transformation based on an actual symmetry plane, obtains the optimal tightening path of the double-side operation, comprises: the method comprises the steps of constructing a multi-target collaborative optimization model by synchronously realizing optimization targets of time optimization, stable motion, load balancing and geometric behavioristic; solving the multi-target collaborative optimization model by utilizing a mixed optimization strategy integrating global exploration and local fine search capability and combining single-side tightening point coordinates to obtain a globally optimal tightening sequence; The planned optimal tightening sequence is automatically mapped to the other side of the symmetrical workpiece to be processed through self-adaptive coordinate mirror transformation by combining an actual symmetry plane obtained by dynamic calibration, an optimal tightening path of the other side is generated, and the optimal tightening path of double-side operation is obtained, wherein the expression of the coordinate mirror transformation is as follows: ; ; In the formula, For the tightening point after the mirror image transformation, Is a homogeneous matrix, which is a matrix of homogeneous size, In order to tighten the point of the screw, The matrix is a unit matrix, n is a unit normal vector of an actual symmetry plane, T is a transposed symbol, and D is a constant term; Based on the optimal tightening path of double-side operation, full-path simulation is performed in the constructed virtual three-dimensional space, the space interference risk of the double-side tightening operation module in the complex motion process is dynamically detected, and the path points or the running postures are finely adjusted according to the detection result, so that active safety protection in the planning stage is realized.
7. The vision-and-adaptive mirror-image-based symmetric workpiece flexible tightening system of claim 6, wherein the expression of the objective function of the multi-objective collaborative optimization model is: ; ; ; ; In the formula, For the optimal path, arg min is a variable corresponding to the minimum value, R is a tightening path to be optimized, Ω is a set of feasible paths satisfying geometric and kinematic constraints, As a result of the total time cost of the path, In order for the path to be smooth, For the degree of load variance of the double-sided mechanism, W 1 、w 2 、w 3 、w 4 is the weight coefficient of the total time cost of the path, the smoothness of the path, the load difference of the double-side mechanism and the singular potential avoidance index, t f is the end time of the path, t 0 is the end time of the path, d is the differential operator, r (t) is the position vector function, The loads of the left side mechanism and the right side mechanism respectively, In order to be a proportional sign of the sign, Is the condition number of the jacobian matrix.
8. The vision and adaptive mirror image based symmetric workpiece flexible tightening system according to claim 6, wherein the hybrid optimization strategy integrating global exploration and local fine search capability effectively avoids sinking into a local optimal solution by cooperating with a plurality of search mechanisms, and is characterized in that the search step length and direction are dynamically adjusted: ; ; In the formula, For the state vector or position coordinates of the system at the t +1 iteration, For the search step increment in the t-th iteration, 、 Two coefficients for balancing exploration dynamics and development, which are adaptively adjusted with time or iteration state respectively, In order to explore the terms in a global manner, Is a local development project.
9. The vision-and-adaptive mirror-image-based symmetric workpiece flexible tightening system of claim 6, wherein the detection of the risk of spatial interference is based on a geometric model of a mechanism link and a spatial distance calculation: ; where d is the minimum distance between the links of the mechanism, For each joint configuration of the mechanism at the moment t, dist is a distance calculation function, link i is the ith connecting rod in The lower spatial position, link j (x), is the j-th Link in A lower spatial position; When (when) When the potential collision risk is judged, the path points or the motion postures are finely adjusted according to the adjustment quantity, so that active safety protection in the planning stage is realized; Wherein the adjustment amount is solved by the following expression: ; In the formula, In order to adjust the amount of the liquid, To adjust the two-norm square of the quantity, Is a preset safety threshold.
10. A vision-and-adaptive mirror-image-based symmetrical workpiece flexible tightening method, implemented on the basis of the vision-and-adaptive mirror-image-based symmetrical workpiece flexible tightening system as claimed in any one of claims 1 to 9, characterized in that the method comprises: s1, connecting a flexible tightening control module with a positioning and fixing module and a tightening operation module in a signal manner, and connecting the flexible tightening control module with a vision acquisition module in a communication manner; s2, after the flexible tightening control module receives the tray in-place signal, the positioning and fixing module is controlled to fix the symmetrical workpiece to be processed; S3, after the symmetrical workpieces to be processed are fixed, the flexible tightening control module sends out a hardware starting signal, and the vision acquisition module receives the signal and acquires vision image data of the symmetrical workpieces to be processed and uploads the vision image data to the flexible tightening control module; S4, the flexible tightening control module receives visual image data, recognizes single-side tightening point coordinates, synchronously performs dynamic calibration on an actual symmetry plane of the workpiece, and plans an optimal tightening path considering efficiency and safety; s5, the flexible tightening control module generates a tightening operation instruction of the other side through self-adaptive mirror image transformation based on an actual symmetry plane, and controls the output of the tightening operation module so as to realize double-side flexible tightening operation of the symmetrical structure workpiece; S6, the tightening operation module receives a tightening control instruction of the flexible tightening control module, and simultaneously performs tightening operation on two sides of the symmetrical workpiece to be machined according to the tightening control instruction; And S7, after all the bolts are screwed up to the standard, resetting the screwing operation module to a safe position, sequentially releasing the fixing of the positioning and fixing module, and recovering to an initial standby state to wait for the next operation cycle.

Description

Symmetrical workpiece flexible tightening system and method based on vision and self-adaptive mirror image Technical Field The invention relates to the technical field of industrial automation, in particular to a symmetrical workpiece flexible tightening system and method based on vision and self-adaptive mirror image. Background The bolt tightening of the symmetrical structure workpiece is a key process in the field of industrial automatic assembly, and directly relates to the realization of modular assembly of products and the aim of efficient batch production. However, the traditional symmetrical structure workpiece is mainly additionally provided with a lifting and rotating mechanism, and single-side serial operation is adopted. The current mainstream tightening method mainly relies on the fact that tightening points are calculated in advance and tightening paths are preset, and a system executes tightening operation according to the preset paths, and the working principle is as follows: 1) The tightening point is calculated in advance, wherein the determination of the tightening point needs to be calculated in advance, then the tightening point is input into a control system, and when the tightening gun moves to the tightening point, the control system sends out a signal to carry out tightening; 2) The fixed path execution, namely setting a moving path in a control system in advance according to each workpiece, and controlling the tightening gun to move according to the path by the system; 3) Single-side serial operation, for symmetrical workpieces, the system is executed in sequence. After all the screwing tasks on one side are completed, the workpiece is turned over by one hundred and eighty degrees by the lifting and rotating mechanism, and then the other side is screwed. However, this method faces a great challenge in terms of productivity and versatility in the face of the current trend of flexible manufacturing of "multiple varieties, small batches". In a flexible assembly workshop, the sizes and the layout and the number of tightening points of symmetrical workpieces are different due to different product models, and a coordinate path preset in a system aiming at a single workpiece model cannot be flexibly adjusted, so that the flexible assembly workshop is difficult to adapt to the production requirements of multiple varieties. In addition, after one side is screwed up, the lifting and rotating mechanism is started to turn the workpiece over 180 degrees, and then the other side is operated, so that the production beat is greatly prolonged. For the problems in the related art, no effective solution has been proposed at present. Disclosure of Invention Aiming at the problems in the related art, the invention provides a symmetrical workpiece flexible tightening system and a symmetrical workpiece flexible tightening method based on vision and self-adaptive mirror image, so as to overcome the technical problems in the prior art. For this purpose, the invention adopts the following specific technical scheme: according to one aspect of the present invention, there is provided a vision and adaptive mirror image based symmetric workpiece flexible tightening system comprising: the positioning and fixing module is used for fixing the symmetrical workpiece to be processed after the flexible tightening control module receives the tray in-place signal; The visual acquisition module is used for acquiring visual image data of the symmetrical workpiece to be processed and transmitting the visual image data to the flexible tightening control module; The flexible tightening control module is used for coordinating the cooperative work of the modules, receiving external IO signals and visual image data, identifying single-side tightening point coordinates, synchronously carrying out dynamic calibration on the actual symmetrical plane of the workpiece, planning an optimal tightening path considering efficiency and safety, generating a tightening operation instruction on the other side through self-adaptive mirror image transformation based on the actual symmetrical plane, and controlling the output of the tightening operation module so as to realize double-side flexible tightening operation of the workpiece with the symmetrical structure; the tightening operation module is used for receiving a tightening control instruction of the flexible tightening control module and simultaneously implementing tightening operation on two sides of the symmetrical workpiece to be processed according to the tightening control instruction; The flexible tightening control module is in signal connection with the positioning fixing module and the tightening operation module, is in communication connection with the vision acquisition module, and can accurately realize flexible tightening operation on various symmetrical products through cooperation of the positioning fixing module, the vision acquisition module, the flexible tightening c