Search

CN-121982221-A - Three-dimensional repairing method based on stripe projection profilometry

CN121982221ACN 121982221 ACN121982221 ACN 121982221ACN-121982221-A

Abstract

The invention discloses a three-dimensional stereo restoration method and system based on stripe projection contour operation, which mainly solve the problems of large calculated amount, texture loss and lack of training data in the traditional three-dimensional stereo restoration. The system comprises an MEMS stripe projection unit, an intelligent repair calculation unit and a 3D printing execution unit. According to the method, firstly, an FPGA is utilized to control an MEMS galvanometer to carry out high-precision three-dimensional scanning, then three-dimensional data is reduced to a depth map, intelligent prediction and texture repair of damaged areas are completed through improving U-Net generation countermeasure network, a repair point cloud model is obtained through reverse mapping, a 3D printer is imported, and accurate re-engraving of an entity is completed through automatic parameter matching. The invention realizes the full-flow closed loop from damaged entity scanning to complete entity printing, has the characteristics of high-precision reconstruction, vivid texture restoration and automatic manufacturing, and can adapt to the three-dimensional restoration requirements of most of the objects in the real world, and the digital protection of cultural relics is the key application scene thereof.

Inventors

  • HAN MIN
  • SUN TIANYU
  • YAN LIJIE
  • Shen Ziyan
  • DONG RUI

Assignees

  • 北京化工大学

Dates

Publication Date
20260505
Application Date
20260224

Claims (9)

  1. 1. The three-dimensional repairing method based on the fringe projection contour operation is characterized by sequentially executing the steps of depth image acquisition, repairing model training, depth image repairing optimization and three-dimensional reconstruction post-processing, and realizing three-dimensional repairing of most of entities in the real world, and specifically comprises the following steps: (1) The method comprises the steps of obtaining a depth image, namely controlling MEMS galvanometer to modulate line laser by using an FPGA, projecting a standard sine stripe pattern to a target object at a high speed, compensating the galvanometer angle in real time by using a pulse feedback subunit, synchronously collecting a stripe image sequence on the surface of the object, obtaining high-precision original three-dimensional data through phase resolution, converting the high-precision original three-dimensional data into the standard depth image, performing mask processing on the standard depth image by using a random irregular polygon algorithm generated by random seeds, and constructing training data corresponding to the damage and the integrity; (2) Constructing a generating countermeasure network (GAN) comprising a generator network and a discriminator network, wherein the generator network adopts a U-Net structure integrating attention gating, receives a damaged depth map, performs edge fidelity optimization through bilateral filtering to output a predicted repair map, and the discriminator network adopts a PatchGAN structure to distinguish the predicted repair map from a real standard depth map and completes network training through a mixed loss function; (3) The depth map repairing optimization, namely inputting an actual damaged depth map of an object to be repaired into a trained generator to obtain an original repairing depth map, denoising and smoothing the original repairing map by adopting bilateral filtering, and outputting the optimized repairing depth map while retaining edge information; (4) And (3) three-dimensional reconstruction post-processing, namely reversely mapping the filtered two-dimensional restoration depth image into three-dimensional space coordinates according to internal parameters of a camera in the three-dimensional reconstruction process of the structured light, generating a restored three-dimensional point cloud model, and obtaining a final three-dimensional restoration model through optimization processing. (5) And (3) entity printing and re-engraving, namely importing the repaired three-dimensional digital model into a 3D printing execution unit, automatically printing 3D according to the geometric characteristics of the model through a built-in parameter matching module, and directly outputting the repaired entity object.
  2. 2. The method of claim 1, wherein the hardware unit employed in the three-dimensional data acquisition step comprises: The system comprises an FPGA control subunit, an MEMS vibrating mirror subunit, a line laser emission subunit, an image acquisition subunit and an angle detection feedback subunit; the FPGA control subunit drives the MEMS galvanometer to swing at a fixed frequency, the angle detection subunit acquires the swinging angle of the galvanometer in real time, and the pulse feedback subunit transmits an angle deviation signal back to the FPGA to realize dynamic compensation, so that the stability of stripe projection is ensured.
  3. 3. The method of claim 1, wherein in the physical print reproduction step: and an automatic adjusting module arranged in the system automatically prints in a 3D mode according to the appearance characteristics of the model and the physical properties of the materials, and conversion from the digital model to the physical entity can be completed without manual intervention.
  4. 4. The method according to claim 1, wherein the generator network adopts a dual modified U-Net structure that fuses attention gating and smoothing residuals, specifically comprising: (1) The encoder part consists of 8 downsampling convolution modules, wherein each module comprises a convolution layer, a batch normalization layer and an activation function and is used for gradually extracting the characteristic information of the depth image; (2) The smooth residual bottleneck layer is arranged between the tail end of the encoder and the head end of the decoder and consists of 6 to 9 smooth residual modules connected in series, wherein each smooth residual module comprises two convolution layers and batch normalization layers which are sequentially connected, and the input and the output of the modules are added in element level through jump connection, so that smooth texture features in a deep feature space are realized and gradient disappearance is prevented; (3) The decoder part consists of 7 up-sampling transpose convolution modules and is used for gradually restoring the characteristic dimension of the depth image; (4) Attention-gating skip connection, namely setting an attention-gating module between corresponding feature levels of the encoder and the decoder, suppressing background feature response of the non-repair area, and splicing the weighted feature map with the decoder features.
  5. 5. The method of claim 1, wherein when masking the standard depth image, randomly selecting a center point in a non-background area of the depth image, determining a damage radius range according to a preset damage degree threshold, randomly generating polygons with 6 to 15 vertexes in the radius range, and filling pixel values in the polygon area to zero to simulate irregular collapse and defect of an entity.
  6. 6. The method according to claim 1, wherein the structured light three-dimensional reconstruction step based on fringe projection profilometry comprises the steps of projecting a coded fringe pattern on the surface of a target object, collecting a deformed fringe image modulated on the surface of the object through a camera, completing reconstruction of the three-dimensional shape of the object by using a phase unwrapping and phase-height mapping algorithm, obtaining a three-dimensional grid model of the target object, and converting the three-dimensional grid model into a standard depth image meeting the training requirement of a repair model.
  7. 7. The method of claim 1, wherein the hybrid loss function combines the contrast loss and the pixel consistency loss, the weight parameters of the generator and the arbiter are alternately updated by a back propagation algorithm, the loss function is minimized by an Adam optimizer, and the training is performed alternately until the model converges.
  8. 8. The method of claim 1, wherein the bilateral filtering processing can preserve edge geometric details and depth jump features in the physical depth image while denoising is achieved, and guarantee the accuracy of subsequent three-dimensional reconstruction.
  9. 9. The method according to claim 1, wherein in the three-dimensional reconstruction post-processing, the inverse mapping of the three-dimensional space coordinates is completed by traversing the filtered repair depth image pixels and the gray values thereof, and meanwhile, the surface flatness of the point cloud model is optimized through normal estimation, and the final three-dimensional repair model is generated after outlier rejection and unified coloring rendering.

Description

Three-dimensional repairing method based on stripe projection profilometry Technical Field The invention relates to the technical field of computer vision and three-dimensional digital restoration, in particular to a cultural relic three-dimensional point cloud restoration method and system based on a generated countermeasure network. Background The application requirements of the three-dimensional stereo repair technology in various fields such as cultural relic protection, industrial manufacturing, digital cultural relic creation and the like are increasing, however, when aiming at various entity repairs, the problems of large calculated amount, detail loss and lack of training data generally exist in the traditional three-dimensional digital repair technology, and the high-precision and high-efficiency repair requirements are difficult to meet: The method comprises the following steps of directly processing point cloud data by using a 3D convolutional neural network, wherein the three-dimensional point cloud data has inherent characteristics of disorder, unstructured and sparsity, so that the calculated amount of the network grows exponentially, the consumption of a display memory is huge, a high-resolution repair model is difficult to train, fine high-frequency texture details on the surface of an object cannot be effectively captured, the problems of geometric structure distortion and texture blurring are easy to occur after repair, and the problems are particularly prominent in entity repair with high detail requirements such as cultural relics and precision industrial parts; The deep learning repair which depends on real damage-complete pairing data, namely training a three-dimensional repaired deep learning model requires a large amount of paired damaged-complete entity data as training samples, but in practical application, registration data of the same object in a perfect state and a damaged state are difficult to obtain no matter cultural relics, industrial parts or other entities, so that the model lacks an effective training sample and cannot adapt to object repair requirements of different types and different damage degrees; The three-dimensional reconstruction and repair dislocation is that the traditional repair technology does not effectively fuse the three-dimensional reconstruction of the structured light and the later repair, the three-dimensional model obtained by reconstruction is difficult to directly adapt to the repair flow, a large amount of manual pretreatment is needed, the geometric details of the object are easy to lose in the treatment process, and the final repair precision is influenced. The problems are solved, so that the existing three-dimensional digital repair technology has obvious defects in repair precision, calculation efficiency and practical applicability, a new three-dimensional repair technology is urgently needed, a structured light three-dimensional reconstruction and intelligent repair algorithm is fused, the bottleneck of the prior art is broken through, and high-precision and high-efficiency repair of various entities is realized. Disclosure of Invention In order to overcome the defects of large calculated amount, texture detail loss, lack of training data and difficult topology structure reconstruction in the existing three-dimensional digital repair technology, the invention provides a three-dimensional stereo repair method based on stripe projection profilometry, which fuses structured light three-dimensional reconstruction and generation of an anti-network repair algorithm, realizes high-precision and high-efficiency repair of three-dimensional point clouds of most of objects in the real world, solves the problem of insufficient training samples of a repair model, improves the practical application and generalization capability of the repair technology, and repairs cultural relics into key application scenes of the method. The method comprises the steps of firstly completing three-dimensional morphology scanning and standard depth image acquisition of a target object through a structured light three-dimensional reconstruction technology, reducing the dimension of a complex three-dimensional repair problem to a two-dimensional depth image domain for processing, constructing training data through artificial damage simulation, completing repair learning from a damaged depth image to a complete depth image through generating an antagonism network, finally optimizing the repaired depth image, completing reverse projection by combining camera parameters in the three-dimensional reconstruction process, realizing accurate reconstruction of three-dimensional point cloud, and guaranteeing geometric structure and edge detail fidelity of a repair model through a post-processing mechanism. Compared with the prior art, the invention has the beneficial effects that: (1) The high-precision high-speed restoration of the two-dimensional depth map comprises the steps of r