CN-121997728-A - Multi-channel non-lambertian surface FPP-PS fusion depth measurement method guided by failure parting module

CN121997728ACN 121997728 ACN121997728 ACN 121997728ACN-121997728-A

Abstract

A multi-channel non-lambertian surface FPP-PS fusion depth measurement method guided by a failure typing module is characterized in that the prior method generally judges a failure area under a single failure type based on a fixed threshold or an empirical rule, and a real mixed failure form is difficult to accurately establish, so that the effectiveness of a subsequent repair or fusion algorithm is limited. The method comprises the steps of constructing a multi-source combined feature model by using the acquired multi-source physical features, processing the multi-source combined feature model through a failure feature encoder to respectively form prototype vectors, carrying multi-source combined feature data of an actual object to be detected and each prototype vector into a failure weight calculation model to respectively obtain weights of corresponding restoration channels, and finally outputting calculation results of restoration phases by each restoration channel in a weighted fusion mode.

Inventors

FAN KUO
TAN JUN
DING YONG
SHEN YINGJIE
LV DAGANG
WANG JUANJUAN
LI CHEN
ZHANG CHAO
WANG JIANFEI

Assignees

哈尔滨工业大学
中电投工程研究检测评定中心有限公司

Dates

Publication Date: 20260508
Application Date: 20260115

Claims (6)

1. A method for measuring depth by fusion of a multi-channel non-lambertian surface FPP-PS guided by a failure parting module is characterized in that after an acquired multi-source physical feature is constructed into a multi-source combined feature model, the multi-source combined feature model is processed by a failure feature encoder to form prototype vectors respectively, multi-source combined feature data of an actual object to be detected and each prototype vector are brought into a failure weight calculation model to obtain weights of corresponding repair channels respectively, and finally each repair channel outputs a calculation result of a repair phase in a weighted fusion mode.
2. The failure parting module-guided multichannel non-lambertian surface FPP-PS fusion depth measurement method of claim 1, wherein the multisource physical characteristics are multisource physical information fused by FPP and PS, and the multisource physical characteristics are obtained by the following steps: Firstly, obtaining an original measurement result aiming at stripe projection profilometry and photometric stereo method, constructing a multi-source physical feature for representing a phase failure state through the original measurement result, and constructing a combined physical feature vector for each pixel position (x, y): In the above-mentioned method, the step of, A multi-element physical feature vector at the corresponding pixel; a corrupted phase value obtained by fringe projection profilometry; the stripe modulation degree is used for representing the stripe quality and contrast level; Is a local gradient characteristic of the damaged phase in space; is a surface normal vector estimated by photometric stereo method; Is the image brightness information for the corresponding pixel location.
3. The failure parting module-guided multichannel non-lambertian surface FPP-PS fusion depth measurement method of claim 1 or 2, wherein after constructing a multi-source joint feature model, the process of processing the multi-source joint feature model by a failure feature encoder to form prototype vector models respectively is as follows: After the multi-source physical characteristics are obtained, the multi-source physical characteristics are input into a failure characteristic coding module, the failure characteristic coding module carries out nonlinear mapping and characteristic compression processing on the original physical quantities in the multi-source physical characteristics, so that low-dimensional embedded characteristics capable of representing phase failure states are obtained, and the failure characteristic coding process is expressed as follows: In the above-mentioned method, the step of, Embedding a vector for the failure feature at pixel (x, y); Encoding a function for the failure feature; is a multi-source physical feature vector; then, typical distribution position processing of different phase failure mechanisms is carried out, a plurality of failure prototype vectors are introduced, and a calculation formula is as follows: In the above-mentioned method, the step of, A failure prototype vector corresponding to the i-th phase failure mechanism; d is the dimension of the failure feature embedding space; n is the number of preset failure mechanisms; By calculating the distance relation between the embedded features and each failure prototype, the network establishes the relative membership of the current pixel to different failure mechanisms in the feature space, thereby completing the continuous failure perception modeling process.
4. The method for measuring depth by fusion of FPP-PS of a multi-channel non-Lambert surface guided by a failure parting module according to claim 3, wherein the process of obtaining corresponding repair channels by each prototype vector model through failure weight calculation is to build a weight relation of a current pixel under different failure mechanism assumptions by calculating similarity between embedded features and each failure prototype after obtaining failure feature embedded vectors and corresponding failure prototypes, and to characterize relative degrees of multiple failure mechanisms under the coexistence state of the same pixel position, and finally to respectively construct corresponding phase repair channels according to each preset failure mechanism, and to carry out targeted correction processing on damaged phases, the calculation process is as follows: the failure weight is obtained by carrying out normalization calculation on the distance between the embedded feature and each failure prototype, and the corresponding calculation formula is as follows: Weights calculated by the above The relative credibility of the pixel (x, y) under the assumption of the ith class failure mechanism; In the phase repairing stage, the network respectively constructs corresponding phase repairing channels aiming at each type of failure mechanism, each repairing channel independently calculates phase compensation quantity under the same input condition, and a phase repairing function corresponding to the ith type of failure mechanism is expressed as In the above-mentioned method, the step of, A network is repaired for the phase corresponding to the class i failed subspace.
5. The method for measuring depth by fusion of FPP-PS of a multi-channel non-lambertian surface guided by a failure parting module according to claim 1, wherein the network obtains a final repair phase by carrying out weighted fusion on phase compensation amounts output by each repair channel, and a calculation formula is as follows: The network structure performs end-to-end training under the constraint of the uniform phase repairing objective function, ensures that the failure feature codes, the failure prototype positions and the parameters of each repairing channel are cooperatively optimized in the training process, and accordingly realizes the self-adaptive modeling and repairing process of various phase failure mechanisms under the condition of complex reflecting surfaces.
6. The failure parting module-guided multichannel non-lambertian surface FPP-PS fusion depth measurement method of claim 5, wherein the calculation process of outputting the repair phase of each repair channel in a weighted fusion mode is as follows: weight distribution constraint loss is introduced, and the corresponding calculation formula is as follows: In the above-mentioned method, the step of, The method is characterized in that a smoothness loss term used for restraining failure weight distribution is used for preventing the weight collapse phenomenon of a network in the initial training stage; As auxiliary constraint, the method and the system act on a network training process together with the phase restoration main loss, and under the condition of comprehensively considering the phase restoration precision and the training stability, the calculation formula of the network overall loss function is as follows: In the above formula, lambda is a weight coefficient for balancing the influence degree of different loss terms; The network overall loss function takes the accuracy of the final repair phase as a core optimization target, and the failure typing, the failure subspace modeling and the phase repair process are integrated into the same optimization framework, so that the repair calculation process taking the convergence into consideration in the overall training process is completed.

Description

Multi-channel non-lambertian surface FPP-PS fusion depth measurement method guided by failure parting module Technical Field The invention particularly relates to a multi-channel non-lambertian surface FPP-PS fusion depth measurement method guided by a failure parting module. Background Fringe projection profilometry (Fringe Projection Profilometry, FPP) and photometric stereo (Photometric Stereo, PS) are two types of optical measurement methods commonly used in current complex curved surface three-dimensional measurements. The FPP realizes high-precision depth recovery through phase calculation, the technology projects periodic stripe coding patterns to the surface of a measured object through a projector, stripes are deformed due to modulation of the height of the surface of the object, then a camera captures the deformed stripe images, phase information in the images is decoded and unfolded through a computer, and a system calibration parameter and a triangulation principle are combined, and finally real three-dimensional coordinates of the surface of the object are mapped, so that high-precision reconstruction of the three-dimensional shape of the object is realized. The photometric stereo method is a non-contact measurement technology for reconstructing normal vectors and three-dimensional morphology of an object surface by shooting a plurality of object images illuminated by light sources in different directions under a fixed visual angle through a single camera based on illumination change and object surface reflection characteristics, and PS estimates the surface normal through brightness information under a multi-illumination condition and is used for supplementing geometric details and local morphology features. In practical engineering applications, the object to be measured often has complex reflective properties, such as a metal surface, a coated surface, or a non-lambertian reflective surface. Under the above conditions, the phase measurement result of the FPP is easily affected by factors such as saturation, modulation attenuation, non-lambertian reflection and the like, so that local or structural phase distortion is generated, and the problem of unstable normal estimation in a high-reflection area is also likely to occur in the PS method. The existing method generally judges the failure area based on a fixed threshold or an empirical rule, and different failure mechanisms are difficult to distinguish accurately, so that the effectiveness of a subsequent repair or fusion algorithm is limited. The following detail difficulties exist in the process of integrating fringe projection profilometry and luminosity three-dimensional, and the three core layers of information incompatibility, complex system calibration coupling and insufficient data registration accuracy caused by measurement principle difference are concentrated, and the specific analysis is as follows: The FPP calculates absolute three-dimensional coordinates through stripe phase change, is sensitive to macroscopic geometrical contours of an object, but is limited to resolution of microscopic surface textures by stripe period, and the photometric stereo method calculates surface normal vectors through illumination gray level differences, is good at capturing microscopic morphologies, has no absolute depth reference, and is easy to accumulate errors through integration. The measurement scale and the information emphasis point of the two are naturally misplaced, and the situation of macroscopic contour and microscopic texture fault can occur when the two are directly fused. Secondly, FPP requires strict external parameter calibration of combining a projector and a camera, ensures the mapping relation between stripe phases and three-dimensional coordinates, and photometric stereo requires accurate calibration of directions and intensities of multiple light sources and requires a fixed camera view angle. The fusion system is required to complete joint calibration among the projector, the camera and the multiple light sources, the error of any link can affect the measurement results of the two methods at the same time, and the complexity of the calibration model rises exponentially. Then, in terms of spatial registration, the point cloud data of the FPP and the normal vector data of the photometric stereo method are based on the same field of view, but the accuracy matching difficulty at the pixel level is high, and small camera shake or object displacement can cause spatial dislocation of the two types of data. In the aspect of time synchronization, the FPP needs to project multi-frame fringe images, the photometric stereo method needs to switch multiple light sources to shoot multi-frame gray images, the image acquisition time sequences of the two images need to be strictly synchronized, and otherwise, data mismatch can occur in a dynamic scene. Finally, FPP requires stable diffuse reflection characteristic of the object surface, avoids