CN-116448008-B - Stripe analysis method based on physical information deep learning

CN116448008BCN 116448008 BCN116448008 BCN 116448008BCN-116448008-B

Abstract

The invention discloses a stripe analysis method based on physical information deep learning, and aims to provide a high-efficiency and accurate three-dimensional contour analysis method. The method comprises the steps of projecting a single high-frequency fringe pattern to a scene to be detected by using a projector, and acquiring a single-frame fringe image by using a camera. And obtaining an initial low-precision wrapping phase by adopting a Fourier transform contour measurement method based on deep learning enhancement, and obtaining an initial phase diagram of a scene to be detected. A lightweight DNN is used for phase analysis and fourier and back propagation of the phase loss function is calculated to output reliable high quality phase results for different types of samples. And reconstructing a three-dimensional contour of the scene to be detected according to the relation between the image coordinate system and the camera coordinate system by combining a phase unfolding algorithm and a three-dimensional reconstruction algorithm. The invention improves the accuracy and efficiency of stripe analysis, has wide application prospect, and is especially suitable for the fields of fine industrial detection and three-dimensional reconstruction.

Inventors

FENG SHIJIE
CHE YUXUAN
YIN WEI
ZUO CHAO
CHEN QIAN
HU YAN

Assignees

南京理工大学

Dates

Publication Date: 20260505
Application Date: 20230329

Claims (6)

1. The stripe analysis method for deep learning based on physical information is characterized by comprising the following steps: Step 1, projecting a single high-frequency fringe pattern to a scene to be measured, collecting the single high-frequency fringe pattern by a camera, obtaining an initial wrapping phase by adopting a Fourier transform contour measurement method based on deep learning enhancement, and obtaining an initial phase map of the scene to be measured; step 2, inputting an initial phase diagram and a high-frequency fringe diagram into a lightweight network, and predicting package phase numerator and denominator to obtain a real phase diagram; Step 3, calculating Fourier loss function values of the initial parcel phase map obtained in the step 1 and the real phase map obtained in the step 2, and calculating a phase loss value by using the phase loss function; Step 4, updating the initial phase diagram into the real phase diagram obtained in the step 2, repeating the steps 2 and 3, and repeatedly iterating the network until the Fourier loss function value and the phase loss function value of the network are converged, so as to obtain the numerator and the denominator of the light-weight DNN neural network output; step 5, obtaining a high-precision parcel phase diagram according to the numerator and denominator output by the lightweight DNN neural network; and 6, reconstructing a three-dimensional contour of the scene to be detected according to the relation between the image coordinate system and the camera coordinate system by using a phase unfolding algorithm and a three-dimensional reconstruction algorithm.
2. The method for analyzing stripes of deep learning based on physical information according to claim 1, wherein the specific method for obtaining the initial wrapping phase by using the fourier transform contour measurement method based on deep learning enhancement is as follows: acquiring high-frequency fringe image Fourier spectrum of (a) ; After Fourier transform and spectrum centering of the input tensor, the method adopts Size-learnable filter To attenuate input spectra Center of the machine The specific formula is: Wherein, the Representing the spectrum obtained by weakening the zero order by fourier transform profilometry, Representing a high-frequency streak image Fourier spectrum of (a) Representing a Hadamard product; Using filters Extracting first-order spectrum information with a size of K2×H2xw 2: Wherein, the Is set to be from Estimated by a step-phase shift method 。
3. The stripe analysis method based on deep learning of physical information according to claim 1, wherein the lightweight network comprises a context path, a spatial path and a feature fusion module, the context path collects edge and phase features with larger acceptance fields by rapidly downsampling and encoding global context information, guides refined advanced features to learn, the spatial path captures encoded spatial information, outputs low-level features, and features from the context path and the spatial path are connected by the feature fusion module and output package phase numerator and denominator by upsampling prediction.
4. The method for analyzing stripes of deep learning based on physical information according to claim 1, wherein the specific formula for calculating the fourier loss value by using the fourier loss function in step 3 is: the specific formula of the phase loss value calculated by using the phase loss function is as follows: Wherein, the Is a fringe phase diagram of the input, Is a true phase diagram obtained by a 12-step phase shift method, Is a phase diagram obtained by fourier transform profilometry, , And Is that , And 2D discrete Fourier transform output by the Fourier transform contour measurement module based on deep learning enhancement.
5. The method for stripe analysis based on deep learning of physical information according to claim 1, wherein in step 3, molecules outputted by using lightweight DNN neural network are used And denominator Obtaining a high-precision parcel phase diagram : 。
6. The method for stripe analysis based on deep learning of physical information according to claim 1, wherein the three-dimensional phase expansion is performed on the wrapping phase outputted by the network, and the three-dimensional contour of the scene to be detected is reconstructed according to the relation between the image coordinate system and the camera coordinate system, so as to obtain the three-dimensional coordinate value of the scene under the camera coordinate system : Wherein, the The operation of taking the modulus is shown, Representing a translation vector between the left and right cameras, And Is the principal point coordinate parameter of the camera, And Is the focal length parameter of the camera in the x and y directions, Is taken as a point Sub-pixel disparity values at the pixel locations.

Description

Stripe analysis method based on physical information deep learning Technical Field The invention belongs to the technical field of optical measurement, and particularly relates to a stripe analysis method for deep learning based on physical information. Background Optical metrology is a versatile metrology technology, which uses light as an information carrier for non-contact measurement, and is the basis for manufacturing, basic research and engineering applications. With the invention of lasers and optoelectronic devices (CCDs), many optical metrology methods and instruments are being used in the most advanced manufacturing processes, precision positioning and quality assessment due to their ability to measure accuracy, sensitivity, repeatability and speed. In optical metrology, the desired physical properties of an object (profile, distance, deformation, etc.) are encoded into the observed measurements (e.g., deformed fringe/speckle images). The success of conventional optical metrology methods depends largely on the forward model of image formation and its inverse, so they are referred to as model-driven or physical-driven methods. However, these image formation models are only approximate estimates of the actual phenomena governed by the laws of physics and are subject to interference from experimental environments (such as motion, vibration, and nonlinearity) and object surface characteristics (such as sparkle and translucency), resulting in problems with weak performance of simple models or complex models that cannot be solved negatively and accurately under poor measurement conditions. With the explosive growth of available data and computing resources, deep learning, a "data-driven" machine learning technique, has achieved impressive success in many areas, such as computer vision and computational imaging. Deep learning penetrates almost all aspects of optical metrology and provides solutions to many challenging problems, such as fringe denoising, fringe analysis, digital holographic reconstruction. Phase retrieval from fringe images is a fundamental task and is also a typical case in many applications of deep learning in optical metrology. In recent years, high-speed three-dimensional shape measurement is widely used in various fields such as industrial quality detection and three-dimensional face recognition. Among the many excellent methods, fringe Projection Profilometry (FPP) based on structured light and triangulation principles has proven to be one of the high performance techniques for measuring high speed motion and rapid deformation due to its inherent non-contact, full-field, accurate and efficient nature. The main stream FPP can be simply divided into two stripe analysis algorithms, fourier transform profilometry FTP and phase shift profilometry PSP. Fourier transform profilometry is very suitable for dynamic three-dimensional acquisition, it can obtain a phase map by one fringe pattern, but this method has the problem of spectral overlap, limiting its measurement quality. Phase shift profilometry is more robust than fourier transform profilometry, can obtain pixel-level phase measurements with higher resolution and accuracy, but it typically requires multiple fringe patterns to reconstruct the three-dimensional topography of the object. The stripe analysis method based on deep learning effectively solves the defects of the phase shift contour measurement method, and obtains the high-precision wrapping phase through single-frame stripe analysis. However, the stripe analysis network based on deep learning has the defects of poor generalization performance and unstable output quality, and is easy to cause uncertainty errors in the three-dimensional measurement process, so for the stripe analysis method based on deep learning, in order to realize high-precision three-dimensional measurement, a more robust deep learning stripe analysis method is not yet available. Disclosure of Invention In order to solve the technical defects in the prior art, the invention provides a stripe analysis method based on deep learning of physical information, which overcomes the defect of poor generalization performance by integrating a lightweight DNN and a learning-enhanced Fourier transform contour measurement method module, thereby reconstructing a high-precision three-dimensional contour of a scene to be detected. The technical scheme for realizing the purpose of the invention is that the stripe analysis method for deep learning based on physical information comprises the following steps: Step 1, projecting a single high-frequency fringe pattern to a scene to be measured, collecting the single high-frequency fringe pattern by a camera, obtaining an initial wrapping phase by adopting a Fourier transform contour measurement method based on deep learning enhancement, and obtaining an initial phase map of the scene to be measured; step 2, inputting an initial phase diagram and a high-frequency fringe diagram i