CN-122023732-A - Polarization spectrum three-dimensional imaging method and device using intelligent algorithm

CN122023732ACN 122023732 ACN122023732 ACN 122023732ACN-122023732-A

Abstract

The invention provides a polarized spectrum three-dimensional imaging method and device utilizing an intelligent algorithm, which relate to the field of three-dimensional imaging, and are characterized in that a three-dimensional model and polarized spectrum images are acquired, the three-dimensional model and the polarized spectrum images are processed to obtain real normals, combined polarized information, normal priori information and spectrum information, a polarized spectrum three-dimensional normal reconstruction model based on physical constraint is constructed by utilizing the combined polarized information, the normal priori information and the spectrum information, an optimal polarized spectrum three-dimensional normal reconstruction model is obtained by utilizing the real normals to train the polarized spectrum three-dimensional normal reconstruction model, a loss function and an evaluation index are set for evaluating the model, polarized spectrum information of a target scene is input into the polarized spectrum three-dimensional normal reconstruction model to generate a predicted normal of the target scene, and the three-dimensional shape of the target is reconstructed according to the predicted normal. The polarization spectrum three-dimensional imaging method and device utilizing the intelligent algorithm effectively improve the reconstruction accuracy of the three-dimensional normal.

Inventors

WANG XIAOXU
ZHANG ZHILIN
LU QIANBO
HAN ZIYU
LI WUKAI
LIU BOXING

Assignees

西北工业大学

Dates

Publication Date: 20260512
Application Date: 20260204

Claims (10)

1. The polarization spectrum three-dimensional imaging method utilizing the intelligent algorithm is characterized by comprising the following steps of: collecting a three-dimensional model and a polarized spectrum image, and processing the three-dimensional model and the polarized spectrum image to obtain a real normal, combined polarized information, normal priori information and spectrum information; constructing a polarization spectrum three-dimensional normal reconstruction model based on physical constraint by utilizing combined polarization information, normal priori information and spectrum information, obtaining an optimal polarization spectrum three-dimensional normal reconstruction model by utilizing a real normal training polarization spectrum three-dimensional normal reconstruction model, and evaluating the model by setting a loss function and an evaluation index; And inputting the polarization spectrum information of the target scene into a polarization spectrum three-dimensional normal reconstruction model to generate a predicted normal of the target scene, and reconstructing the three-dimensional morphology of the target according to the predicted normal.
2. The polarization spectrum three-dimensional imaging method using intelligent algorithm according to claim 1, wherein constructing the polarization spectrum three-dimensional normal reconstruction model based on physical constraint by using the real normal, combined polarization information, normal priori information and spectrum information comprises Respectively encoding the combined polarization information, the normal priori information and the spectrum information, and extracting normal priori features, spectrum features and polarization information features; Blurring the prior characteristic, the spectrum characteristic and the polarization information characteristic of the normal for multiple times; And fusing the characteristic, the normal priori characteristic, the spectrum characteristic and the polarization information characteristic obtained by each blurring, then carrying out the sharpening process to obtain a predicted normal characteristic, and decoding the predicted normal characteristic to obtain a predicted normal.
3. The method of three-dimensional imaging of polarized light spectrum using intelligent algorithm according to claim 2, wherein the method comprises fusing the characteristic, multi-scale characteristic, normal priori characteristic, spectral characteristic and polarization information characteristic obtained by each blurring, and performing a sharpening process to obtain predicted normal characteristic, comprising Fusing the prior characteristic, the spectrum characteristic and the characteristic obtained by blurring each time to obtain a first normal characteristic, fusing the characteristic obtained by blurring of the last layer through a maximum fusion layer to obtain a second normal characteristic, and performing a clearing treatment on the second normal characteristic and the first normal characteristic obtained by the last layer to obtain a third normal characteristic; And sequentially fusing the third normal line feature with the first normal line feature obtained in the upper layer to obtain a final predicted normal line feature.
4. The polarization spectrum three-dimensional imaging method using intelligent algorithm according to claim 1, wherein the expression of the loss function is: ; Where W and H represent the width and height, respectively, of the image, n i,j represents the true normal vector of the image at pixel (i, j), and For the reconstruction normal vector of the pixel, the cosine loss function quantifies the accuracy of the prediction result by measuring the cosine value of the included angle between the true normal vector and the reconstruction normal vector.
5. The polarization spectrum three-dimensional imaging method using intelligent algorithm according to claim 1, wherein the evaluation index comprises average angle error, median error angle and root mean square error; And the ratio of the number of samples with the error angle less than or equal to 11.25 degrees to the total number of samples evaluated, the ratio of the number of samples with the error angle less than or equal to 22.5 degrees to the total number of samples evaluated, and the ratio of the number of samples with the error angle less than or equal to 30 degrees to the total number of samples evaluated.
6. A polarization spectrum three-dimensional imaging method using intelligent algorithm according to claim 3, wherein said blurring process comprises And compressing the normal priori features, the spectrum features and the polarization information by adopting a maximum pooling layer, and then setting a plurality of convolution layers to extract the features.
7. A polarization spectrum three-dimensional imaging method using intelligent algorithm according to claim 3, wherein said sharpening process comprises And processing the features of the previous layer of the sharpness processing through bilinear interpolation to enable the resolution of the features to be matched with the output of the current fusion layer, and then integrating the features with the fusion features obtained by the current layer again.
8. A polarization spectrum three-dimensional imaging method using intelligent algorithm according to claim 3, wherein the feature of the last layer of blurring processing is fused by a maximum fusion layer to obtain a second normal feature, comprising And after normalization processing is carried out on the normal priori feature, the spectrum feature and the polarization information feature of the final layer of blurring processing, independent calculation and different subspace calculation are respectively carried out, the independent calculation result, the different subspace calculation result and the normalization result are spliced and fused, and then the global information of each branch is effectively fused through two convolution layers, so that a second normal feature is obtained.
9. The method for three-dimensional imaging of polarized light spectrum using intelligent algorithm according to claim 1, wherein the processing of the three-dimensional model and the polarized light spectrum image to obtain the true normal, the combined polarization information, the normal priori information and the spectrum information comprises Aligning the three-dimensional model with the two-dimensional image to render a real normal; and analyzing the polarized spectrum image to obtain combined polarization information, normal priori information and spectrum information.
10. A polarization spectrum three-dimensional imaging device utilizing intelligent algorithm is characterized by comprising The data acquisition module is used for acquiring the three-dimensional model and the polarized spectrum image; The data processing module is used for processing the three-dimensional model and the polarized spectrum image to obtain a real normal, and combining the polarized information, the normal priori information and the spectrum information; the model construction module is used for constructing a polarization spectrum three-dimensional normal reconstruction model based on physical constraint by utilizing the combined polarization information, the normal priori information and the spectrum information; The model training module is used for training the polarization spectrum three-dimensional normal reconstruction model by utilizing the real normal to obtain an optimal polarization spectrum three-dimensional normal reconstruction model; The evaluation module is used for setting a loss function and an evaluation index to evaluate the model; the prediction module is used for inputting the polarization spectrum information of the target scene into the polarization spectrum three-dimensional normal reconstruction model to generate a prediction normal of the target scene, and reconstructing the three-dimensional morphology of the target according to the prediction normal.

Description

Polarization spectrum three-dimensional imaging method and device using intelligent algorithm Technical Field The invention relates to the field of three-dimensional imaging of monocular cameras, in particular to a polarized spectrum three-dimensional imaging method and device utilizing an intelligent algorithm. Background The traditional polarization three-dimensional imaging technology is limited by azimuth ambiguity and zenith angle multivalue problems, so that the three-dimensional normal reconstruction precision is low. Although techniques such as multi-view, photometric stereo, binocular vision and depth camera are combined with polarized three-dimensional imaging to effectively improve three-dimensional normal reconstruction accuracy, these methods involve complex imaging processes and face challenges such as image matching difficulties and detection distance limitations. With the continuous development of deep learning in the field of polarization imaging, a data-driven polarization three-dimensional imaging method gradually draws a great deal of attention. There are studies to propose a polarized three-dimensional face reconstruction based on deep learning. The method combines a pre-trained three-dimensional shape model (3 DMM), directly estimates a rough depth map of a human face from a polarized image, then corrects a fuzzy human face normal calculated by a physical method, is mainly suitable for three-dimensional reconstruction of human face features, is difficult to adapt to other complex objects, is researched to introduce the problem of non-orthogonal projection in the process of watching coding polarization three-dimensional imaging into a normal prediction network model, is complicated in processing flow by combining a polarization camera and a dual-equipment imaging system of a depth camera to acquire normal data, and is researched to provide a design based on physical confidence priori aiming at noise interference difference of transmission and reflection components in a transparent object and to reduce the influence of transmission components by combining a polarization angle loss function. According to the method, four fuzzy normal line prior, polarization degree and polarization angle images under specular reflection are used as inputs of a neural network model, although the design achieves remarkable effect in a transparent object scene, the normal line reconstruction performance may be degraded to a certain extent when a target scene mainly based on diffuse reflection is processed, a polarization three-dimensional normal line reconstruction method based on a U-shaped generation countermeasure network is researched and designed, however, the four fuzzy normal lines under the specular reflection model and the fuzzy normal lines of two fuzzy normal lines under the diffuse reflection model are input into the neural network for training, the dependence of the network on specific characteristics is excessively strong due to excessive redundant normal line prior information, a polarization three-dimensional imaging system combining a polaroid and an event camera is provided, and then three-dimensional normal line reconstruction is performed by utilizing deep learning, but a mode of relying on a rotating polaroid still has a certain limit. In high speed motion or dynamic scenarios, the rotating polarizer may not be able to capture all polarization information in real time, resulting in data loss or distortion. In recent years, polarization three-dimensional imaging based on deep learning remarkably improves the accuracy of three-dimensional normal reconstruction by introducing a neural network to deeply mine polarization characteristics under a single view angle. However, due to the lack of sufficient physical prior information, neural networks still face difficulties in handling complex scenes such as high light reflection and multiple materials. Therefore, effective balance between physical priori information of the target reflection light field and strong modeling capability of the neural network is explored and established, and a new thought can be provided for realizing a single-view high-dimensional imaging scheme with higher precision and stronger adaptability. Disclosure of Invention The invention aims to provide a polarized spectrum three-dimensional imaging method and device by utilizing an intelligent algorithm, which utilize the physical characteristics of polarized spectrum to correct fuzzy normal priori information of specular reflection and diffuse reflection, fully integrate the physical characteristics of light intensity, polarization, spectrum and the like of a reflected light field, improve the reconstruction accuracy of a three-dimensional normal, and are more beneficial to processing complex scenes such as high reflection, mixed materials and the like by combining the input of the polarized information, normal priori and spectrum information. In order to achieve the abov