Search

CN-121999130-A - Three-dimensional reconstruction method and system based on multi-view image fusion

CN121999130ACN 121999130 ACN121999130 ACN 121999130ACN-121999130-A

Abstract

The invention discloses a three-dimensional reconstruction method and a system based on multi-view image fusion, wherein the method is used for generating a segmentation mask by adaptively segmenting an obtained long graph line by line; performing cylindrical surface unfolding mapping on the target surface according to the segmentation mask result to obtain a corrected plane texture, performing seamless fusion processing on the plane texture under a plurality of view angles to generate a 360-degree panoramic texture, performing scale conversion and displacement processing on texture U coordinates based on the pixel width W of the panoramic texture, mapping the panoramic texture subjected to the scale conversion and displacement processing on the texture U coordinates to a three-dimensional cylindrical model, and generating a high-fidelity three-dimensional reconstruction model of the target object. The invention can effectively solve the problems of insufficient image segmentation precision, incomplete correction of perspective distortion of the cylinder, visual flaws of multi-view texture splicing and joints in three-dimensional model rendering in the three-dimensional reconstruction process of an elongated cylindrical object (such as a steel wire rope).

Inventors

  • JIA FUYIN
  • JIA HAN

Assignees

  • 徐州市三森威尔矿山科技有限公司
  • 三森威尔(上海)机器人有限公司

Dates

Publication Date
20260508
Application Date
20251230

Claims (10)

  1. 1. A three-dimensional reconstruction method based on multi-view image fusion, characterized in that the method comprises the following steps: acquiring image sequences of the surface of a target object through at least two imaging devices distributed along the circumference, and splicing the image sequences of each imaging device in the vertical direction to acquire a corresponding long graph; performing self-adaptive segmentation on the obtained long graph line by line to generate a segmentation mask; performing cylindrical surface unfolding mapping on the target surface according to the segmentation mask result to obtain corrected plane textures; performing seamless fusion processing on the plane textures under a plurality of view angles to generate 360-degree panoramic textures; Performing linear or affine transformation on the texture U coordinates based on the pixel width W of the panoramic texture; and mapping the panoramic texture subjected to scale transformation and displacement processing on the texture U coordinate to a three-dimensional cylindrical model, and generating a high-fidelity three-dimensional reconstruction model of the target object.
  2. 2. The multi-view image fusion-based three-dimensional reconstruction method according to claim 1, wherein the adaptively segmenting the obtained long images row by row to generate segmented masks comprises coarse positioning based on a global gray threshold and local refinement based on pixel-level gradients in the coarse positioning neighborhood.
  3. 3. The multi-view image fusion-based three-dimensional reconstruction method according to claim 2, wherein the performing coarse localization based on a global gray threshold and performing local refinement based on a pixel level gradient in a coarse localization neighborhood comprises: graying the input color image to obtain a gray scale map; Scanning the gray level graph line by line from the left end and the right end to the center, and determining a coarse positioning point of each line by applying an adjustable global gray level threshold; Calculating the horizontal center differential gradient of the pixel in the rough positioning adjacent domain, and determining the point with the maximum gradient value as the accurate edge position of the row; for rows where edges cannot be located, using located edges of adjacent rows to maintain edge continuity; based on the exact edges of all the rows, a binary split mask is generated and output.
  4. 4. A three-dimensional reconstruction method based on multi-view image fusion according to claim 1 or 3, wherein the cylindrical surface expansion mapping is based on an arcsine function, a nonlinear mapping function as a mathematical mapping model or an empirical lookup table obtained by experimental calibration for interpolation mapping.
  5. 5. The multi-view image fusion-based three-dimensional reconstruction method according to claim 4, wherein the seamless fusion process comprises: Based on an energy map, solving an optimal suture line through a dynamic programming algorithm meeting a state transition equation M (i, j) =E (i, j) +min {. The above, and combining a Laplacian pyramid to fuse plane textures of a plurality of view angles, wherein the energy map is calculated based on a gray level difference absolute value, a gradient domain difference or a structural similarity difference.
  6. 6. The three-dimensional reconstruction method based on multi-view image fusion according to claim 1, wherein linear or affine transformation is performed on texture U-coordinates, said transformation comprising a scaling factor scal and a translation amount transform that satisfies scale= (W- α)/W and transform = β/W, wherein α and β are preset positive real numbers, and α and β preferably do not exceed 5% of the pixel width.
  7. 7. The three-dimensional reconstruction method based on multi-view image fusion according to claim 1 or 6, wherein the method further comprises preprocessing original image data acquired by a camera before the corresponding long image is acquired.
  8. 8. The three-dimensional reconstruction method based on multi-view image fusion according to claim 7, wherein the preprocessing of the raw image data acquired by the camera comprises: Geometrically calibrating each imaging device, obtaining an internal reference matrix and a distortion coefficient of each imaging device, and carrying out radial and tangential distortion correction on an original image frame by utilizing the obtained internal reference matrix and the distortion coefficient; Carrying out radiation calibration on photoelectric response of a camera by adopting an ash card or a uniform light source; and the cameras are synchronized or aligned in a mode of hardware trigger signals or software time stamps, so that the time consistency of the image frames with different visual angles during acquisition is ensured.
  9. 9. A multi-view image fusion-based three-dimensional reconstruction system, the system comprising: the image acquisition unit comprises at least two imaging devices distributed along the circumferential direction and is used for acquiring an image sequence of the surface of the target object; The long-chart splicing unit is used for splicing the image sequence of each preprocessed imaging device in the vertical direction to generate a corresponding long chart; an adaptive segmentation unit for performing adaptive segmentation on the long graph line by line to generate a segmentation mask; a cylindrical expansion unit for performing cylindrical expansion mapping on the target surface according to the split mask to obtain a corrected planar texture; The texture fusion unit is used for performing seamless fusion processing on the plane textures under a plurality of view angles so as to generate 360-degree panoramic textures; the coordinate correction unit is used for performing scale transformation and displacement processing on the texture U coordinate based on the pixel width W of the panoramic texture before mapping the panoramic texture to the three-dimensional cylindrical model; And the three-dimensional modeling unit is used for mapping the panoramic texture subjected to coordinate correction to the surface of the three-dimensional cylindrical model to generate a high-fidelity three-dimensional reconstruction model of the target object.
  10. 10. The three-dimensional reconstruction system based on multi-view image fusion according to claim 9, further comprising a preprocessing unit connected with the image acquisition unit and used for preprocessing the original image data acquired by the image acquisition unit, wherein the preprocessing comprises the steps of geometrically calibrating each imaging device to acquire an internal reference matrix and a distortion coefficient, performing radial and tangential distortion correction on original image frames by utilizing the internal reference matrix and the distortion coefficient, performing radiation calibration on photoelectric response of a camera by adopting an gray card or a uniform light source, and synchronizing or aligning each imaging device by means of a hardware trigger signal or a software time stamp to ensure the time consistency of acquisition of image frames of different view angles.

Description

Three-dimensional reconstruction method and system based on multi-view image fusion Technical Field The application belongs to the technical field of three-dimensional imaging, and particularly relates to a three-dimensional reconstruction method and system based on multi-view image fusion. Background In the field of industrial nondestructive testing, accurate assessment of surface states of key bearing parts such as steel wire ropes is a core link for guaranteeing safe production and equipment maintenance. The traditional detection method, such as manual visual inspection or two-dimensional image analysis, has the obvious technical defects that the manual visual inspection is high in subjectivity and low in efficiency, damage cannot be quantitatively evaluated, and the two-dimensional image analysis can record surface information, but cannot accurately evaluate the actual severity of defects of three-dimensional structures such as dents, abrasion, wire breakage and the like due to the lack of three-dimensional space dimensions, so that an evaluation result is inaccurate. To solve the above problems, the industry has begun to study three-dimensional reconstruction techniques based on multi-view images. However, in the process of realizing automated high-fidelity three-dimensional reconstruction, the following four technical problems to be solved are generally faced with the prior art: 1. the accuracy and the robustness of image segmentation are insufficient, namely, under the conditions of uneven illumination and background noise which are common in the industrial field, the traditional segmentation algorithm (such as a single global threshold method) is difficult to separate the steel wire rope main body from the complex background stably and accurately. The segmented edges often have burrs, breaks or pixel-level deviations, which directly lead to planar texture distortion on which the final reconstructed three-dimensional model is based, and are the primary technical bottleneck affecting reconstruction accuracy. 2. The correction of the perspective distortion of the cylinder is not thorough, namely, the camera with a fixed machine position is used for shooting the cylindrical object, and the image of the cylindrical object inevitably generates serious perspective distortion, namely, the details of the central area of the image are stretched, and the areas close to the two side edges are seriously compressed. Without effective mathematical model correction for such nonlinear distortions, the resulting planar texture would not truly reflect the actual dimensions and details of portions of the object surface, and any subsequent defect measurement or quantitative analysis based on that texture would be unreliable. 3. The multi-view texture stitching has obvious visual defects that images from different cameras and under different illumination environments are directly stitched, and stepped color difference bands and macroscopic seams are easily generated at the stitching positions. Some existing fusion algorithms can alleviate this problem, but often introduce blurring or ghosting at the expense of image sharpness. How to achieve a truly "seamless" and "color consistent" high definition fusion is a major challenge in the prior art. 4. When the finally generated plane texture image is wrapped on the surface of the three-dimensional cylindrical model, an unnatural bright line, dark line or pixel dislocation often occurs at a 360-degree splicing seam of the model due to the inherent precision problem when floating point number sampling is carried out on texture coordinates at the boundary of u=0 and u=1 in computer graphics, so that 360-degree continuous and smooth observation experience of the model is seriously affected, and a great obstacle for improving three-dimensional visual quality is formed. In view of the above, there is a lack of a complete solution in the prior art that can systematically, automatically, and with high quality solve all the above problems, and ultimately generate a high-fidelity, flawless three-dimensional digital twin model for precision detection. Disclosure of Invention The invention aims to provide a three-dimensional reconstruction method and system based on multi-view image fusion. The method aims to systematically solve a series of core technical problems of low reconstruction precision of a three-dimensional model, obvious splicing seams and color differences of textures, inherent rendering seams of a rendering model and the like in the prior art, so as to realize automatic and high-fidelity digital twin construction of the surfaces of slender cylindrical objects such as steel ropes and the like. The invention is realized according to the following technical scheme: the invention provides a three-dimensional reconstruction method based on multi-view image fusion, which comprises the following steps: acquiring image sequences of the surface of a target object through at least t