CN-121805153-B - Method, system, medium and product for detecting aircraft panel rivet quality

CN121805153BCN 121805153 BCN121805153 BCN 121805153BCN-121805153-B

Abstract

The invention discloses a method, a system, a medium and a product for detecting the quality of an aircraft panel rivet, wherein the method comprises the following steps: the monocular moving camera collects a plurality of aircraft panel images of the aircraft section panel to be detected at a plurality of equidistant positions, and performs distortion correction processing on each aircraft panel image. Acquiring first coordinate information and corresponding identification parameters of each rivet point and each rivet hole in an image coordinate system after adjacent correction, constructing a first point cloud set based on an objective function constructed by a beam adjustment method and used for realizing the minimization of a reprojection error, acquiring second coordinate information of each rivet point and each rivet hole, performing matching analysis on the first point cloud set and a second point cloud set generated by a pre-constructed digital model of an aircraft panel, obtaining second abnormal coordinate information of an abnormal rivet point and a rivet hole, converting the second abnormal coordinate information into third coordinate information in a coordinate system of a three-dimensional projector, and projecting and labeling through the three-dimensional projector. According to the scheme, the detection efficiency and accuracy of abnormal rivet points and rivet holes on the aircraft panel are improved.

Inventors

Cai Shuaipeng
ZHANG XIANFA
DENG JINGYU
BU XIAOYAN
Cao jialing
Feng Jingxuan

Assignees

上海飞机制造有限公司

Dates

Publication Date: 20260512
Application Date: 20260309

Claims (10)

1. A method of detecting aircraft panel rivet quality performed by a system for detecting aircraft panel rivet quality, the system comprising a base, a linear slide rail disposed on the base, a monocular motion camera disposed on the linear slide rail, and a three-dimensional projector disposed on the base, the method comprising: the method comprises the steps of acquiring a plurality of aircraft panel images of a to-be-detected aircraft section panel placed below a detection system at a plurality of equidistant positions by controlling a monocular motion camera to translate on a linear sliding guide rail; Carrying out distortion correction processing on each aircraft panel image according to a pre-constructed camera image radial distortion correction model and a camera image tangential distortion correction model to obtain each corrected image; Respectively identifying rivet points and rivet holes in each corrected image, and acquiring first coordinate information of each rivet point and each rivet hole in a current image coordinate system and corresponding identification parameters according to identification positions of each rivet point and each rivet hole in adjacent corrected images; Acquiring second coordinate information of each rivet point and each rivet hole in a sliding rail base coordinate system based on an objective function constructed by a beam adjustment method and used for realizing the minimization of a reprojection error according to the first coordinate information; Constructing a first point cloud set according to the second coordinate information, and carrying out matching analysis on the first point cloud set and a second point cloud set generated by distributing rivet points and rivet holes on a pre-constructed digital model of the aircraft panel to obtain second abnormal coordinate information of abnormal rivet points and abnormal rivet holes in a sliding rail base coordinate system; And converting the second abnormal coordinate information into third coordinate information in a three-dimensional projector coordinate system, and projecting and marking abnormal rivet points and abnormal rivet holes matched with the third coordinate information on the current aircraft section panel through the three-dimensional projector.
2. The method of claim 1, wherein prior to capturing a plurality of aircraft panel images of the aircraft section panel under test placed under the inspection system at a plurality of equidistant positions by controlling translation of the monocular motion camera on the linear slide rail, comprising: Calibrating the positions of a slide rail base coordinate system, a camera coordinate system and a three-dimensional projector coordinate system on a detection system of the rivet quality of the aircraft panel; And determining the conversion relation of coordinate values of each point in the rivet points and the rivet holes of the panel of the airplane in the coordinate system of the slide rail base, the coordinate system of the camera and the coordinate system of the three-dimensional projector according to the camera external parameters of the monocular movement camera.
3. The method of claim 1, wherein performing a distortion correction process on each aircraft panel image based on a pre-constructed camera image radial distortion correction model and a camera image tangential distortion correction model to obtain each corrected image comprises: the related formulas for confirming the pre-constructed camera image radial distortion correction model, the camera image tangential distortion correction model and the image distortion coordinate information are as follows: ; ; ; ; Wherein k1 is a first order radial distortion coefficient, k2 is a second order radial distortion coefficient, k3 is a third order radial distortion coefficient, p1 is a first tangential distortion component, p2 is a second tangential distortion component, Is the abscissa after radial distortion of the camera image, Is the ordinate of the camera image after radial distortion, As the abscissa after tangential distortion of the camera image, Is the ordinate of the camera image after tangential distortion, As the integrated abscissa after the distortion of the camera image, The coordinate system is characterized in that x is the comprehensive ordinate after the camera image is distorted, x is the abscissa after the camera image is distorted and corrected, y is the ordinate after the camera image is distorted and corrected, and r is the distance between the coordinate of the target pixel point and the origin of the pixel coordinate system after the tangential distortion and correction of the image; Sequentially acquiring an aircraft panel image as a current camera distortion image; Taking the horizontal coordinate value of each pixel point in the current camera distortion image as And After x and y corresponding to each pixel point are calculated through the formulas, a corrected image corresponding to the current camera distortion image is obtained; And returning to execute the operation of sequentially acquiring one aircraft panel image as the current camera distortion image until the processing of all aircraft panel images is completed.
4. The method of claim 2, wherein the identifying of the rivet points and the rivet holes in each corrected image is performed separately, and the first coordinate information of each rivet point and each rivet hole in the current image coordinate system and the corresponding identification parameters are obtained according to the identified positions of each rivet point and each rivet hole in the adjacent corrected image, including: Respectively identifying rivet points and rivet holes in each corrected image to obtain identification positions of the rivet points and the rivet holes in each corrected image; Acquiring first coordinate information of each rivet point and each rivet hole in a current image coordinate system according to the identification positions of the rivet points and the rivet holes in each corrected image and the acquisition positions of the monocular motion camera when acquiring each aircraft panel image corresponding to each corrected image respectively; Sequentially acquiring a corrected image serving as a current processing image according to the sequence of acquisition positions from front to back, and acquiring an adjacent processing image matched with the next acquisition position of the current processing image; Matching the rivet points and rivet holes respectively identified in the current processing image and the adjacent processing image, and setting identification parameters of the successfully matched rivet points and rivet holes in the current processing image and the adjacent processing image as target values; And returning to execute the operation of sequentially acquiring one corrected image as the current processed image according to the sequence from front to back of the acquisition position until the processing of all corrected images is completed.
5. The method of claim 4, wherein obtaining second coordinate information of each rivet point and each rivet hole in the slide base coordinate system based on an objective function constructed by a beam adjustment method for achieving the minimization of the re-projection error, comprises: moving interval distance for translating and collecting aircraft panel images on slide rail according to monocular movement camera Obtaining an objective function constructed by a beam adjustment method for realizing the minimization of the re-projection error: ; wherein m is the total number of rivets and rivet holes, n is the number of image acquisition positions of the monocular moving camera on the slide rail, Is the sign of whether the j-th rivet point or rivet hole is in the i-th corrected image, if so, then 1, If not, then For 0, f x is the x-axis focal length in the image coordinate system, f y is the y-axis focal length in the image coordinate system, u 0 and v 0 are the coordinates of the center point of the image in the image coordinate system, u ij and v ij are the two-dimensional image coordinates of the jth rivet point or rivet hole in the ith corrected image, x wj ,y wj ,z wj is the second coordinate information of the jth rivet or rivet hole in the slide base coordinate system, and R 1 and T 1 respectively represent the rotation matrix and the translation matrix of the image coordinates in the slide base coordinate system into the camera coordinate system in the 1 st camera acquisition image; And solving the constructed objective function for realizing the minimization of the reprojection error, and calculating to obtain second coordinate information of each rivet point and each rivet hole on the aircraft panel image in the slide rail base coordinate system.
6. The method of any one of claims 1-5, wherein constructing a first point cloud set according to each second coordinate information, and performing matching analysis on the first point cloud set and a second point cloud set generated by distribution of rivet points and rivet holes on a pre-constructed digital model of the aircraft panel to obtain second abnormal coordinate information of abnormal rivet points and abnormal rivet holes in a slide rail base coordinate system, wherein the method comprises: sequentially acquiring a rivet point or a rivet hole as a current identification object in a first point cloud set constructed under a slide rail base coordinate system, and acquiring current second coordinate information corresponding to the current identification object; in the second point cloud set, after standard second coordinate information corresponding to the current identification object is obtained, matching analysis is carried out on the current second coordinate information and the standard second coordinate information; if the matching result between the current second coordinate information and the standard second coordinate information accords with the preset abnormal screening rule, determining the current second coordinate information as second abnormal coordinate information; The abnormal screening rules comprise a missing hole screening rule, a multiple hole screening rule, a missing nail screening rule and a hole nail position deviation excessive screening rule; and returning to execute the processing of sequentially acquiring one rivet point or rivet hole as the current identification object in the first point cloud set constructed under the coordinate system of the slide rail base until the processing of all rivet points and rivet holes is completed.
7. The method of claim 2, wherein converting the second abnormal coordinate information to third coordinate information in a three-dimensional projector coordinate system and projecting, by the three-dimensional projector, abnormal rivet points and abnormal rivet holes on the current aircraft section panel that match the third coordinate information, comprises: The second abnormal coordinate information of the rivet point or the rivet hole in the coordinate system of the slide rail base is converted into third coordinate information in the coordinate system of the three-dimensional projector according to the coordinate system conversion relation; And determining the positions of the abnormal rivet points and the abnormal rivet holes according to the third coordinate information in a three-dimensional projector coordinate system, and projecting and marking the corresponding positions on the current aircraft section panel through the three-dimensional projector.
8. A system for detecting aircraft panel rivet quality, the system comprising: The device comprises a base, a linear sliding guide rail arranged on the base, a monocular moving camera arranged on the linear sliding guide rail, a three-dimensional projector arranged on the base and a controller; The controller is configured to perform a method of implementing the aircraft panel rivet quality detection method of any one of claims 1-7.
9. A computer readable storage medium storing computer instructions for causing the aircraft panel rivet quality detection system to perform the aircraft panel rivet quality detection method of any one of claims 1-7.
10. A computer program product, characterized in that the computer program product comprises a computer program which, when executed by a detection system for aircraft panel rivet quality, implements a method for detecting aircraft panel rivet quality according to any one of claims 1-7.

Description

Method, system, medium and product for detecting aircraft panel rivet quality Technical Field The invention relates to the technical field of machine vision, in particular to a method, a system, a medium and a product for detecting the quality of rivets on an aircraft panel. Background The riveting is a core connection mode of the aircraft wall plate, the stress concentration is easily generated in the riveting structure by the pressurization and depressurization cycle in the aircraft flight, the quality problems of rivet loosening, rivet hole edge cracks and the like directly affect the structural integrity and flight safety of the aircraft, and the riveting device is an important detection object for aviation manufacture and maintenance. The quality detection of the rivet on the surface of the airplane section is required to identify the problems of rivet leakage, multiple rivets, error positions of the rivets and the like, and accurate three-dimensional reconstruction modeling of the rivet is a key for realizing the detection, and the prior art mainly comprises three types of realization modes: and scanning the surface of the aircraft by adopting a three-dimensional laser scanner, dividing the positions of the rivets to complete three-dimensional modeling, acquiring images by using a two-dimensional ultra-high definition camera to identify the rivets, acquiring a detection result by combining the corresponding relation between the two-dimensional images and the rivets of the three-dimensional digital model, detecting, positioning and splicing the rivets in different frame images by using a triangular camera, and constructing a rivet distribution model. The technical scheme has obvious defects that the three-dimensional laser scanning method is long in detection time and complex in operation flow, the rivet corresponding relation between the two can not be determined independently by a method of combining a two-dimensional camera with a three-dimensional digital model, the corresponding relation needs to be preset, practical application is limited, and the triangular camera splicing modeling scheme is difficult to quickly and effectively match the corresponding relation of rivet points obtained by detection of different cameras, so that modeling efficiency and accuracy are affected. Disclosure of Invention The embodiment of the invention provides a method, a system, a medium and a product for detecting the quality of an aircraft panel rivet, which are used for acquiring an aircraft panel image through a constructed aircraft panel rivet quality detection system, generating a coordinate three-dimensional structure model of an aircraft panel rivet point and a rivet hole, and matching the model with a corresponding digital model to quickly locate an abnormal rivet point and the rivet hole. According to an aspect of an embodiment of the present invention, there is provided a method of detecting aircraft panel rivet quality, the method being performed by a system for detecting aircraft panel rivet quality, the system comprising a base, a linear slide rail disposed on the base, a monocular motion camera disposed on the linear slide rail, and a three-dimensional projector disposed on the base, the method comprising: the method comprises the steps of acquiring a plurality of aircraft panel images of a to-be-detected aircraft section panel placed below a detection system at a plurality of equidistant positions by controlling a monocular motion camera to translate on a linear sliding guide rail; Carrying out distortion correction processing on each aircraft panel image according to a pre-constructed camera image radial distortion correction model and a camera image tangential distortion correction model to obtain each corrected image; Respectively identifying rivet points and rivet holes in each corrected image, and acquiring first coordinate information of each rivet point and each rivet hole in a current image coordinate system and corresponding identification parameters according to identification positions of each rivet point and each rivet hole in adjacent corrected images; Acquiring second coordinate information of each rivet point and each rivet hole in a sliding rail base coordinate system based on an objective function constructed by a beam adjustment method and used for realizing the minimization of a reprojection error according to the first coordinate information; Constructing a first point cloud set according to the second coordinate information, and carrying out matching analysis on the first point cloud set and a second point cloud set generated by distributing rivet points and rivet holes on a pre-constructed digital model of the aircraft panel to obtain second abnormal coordinate information of abnormal rivet points and abnormal rivet holes in a sliding rail base coordinate system; And converting the second abnormal coordinate information into third coordinate information in a three-dimensional proj