CN-121544812-B - Multiscale scene reconstruction method based on multisource data fusion

CN121544812BCN 121544812 BCN121544812 BCN 121544812BCN-121544812-B

Abstract

The invention discloses a multiscale scene reconstruction method based on multisource data fusion, which relates to the technical field of three-dimensional scene reconstruction, and comprises the steps of calculating modulation angle marks of images by utilizing row direction statistical signals, dividing an image frame sequence into a plurality of modulation angle channels with consistent illumination states, independently constructing coarse-level geometric fragments representing scene macroscopic topology and fine-level geometric fragments containing specific illumination strip characteristics aiming at each channel, generating fine-level credibility weights for quantitatively distinguishing real details and strip artifacts by calculating cross-channel geometric divergence among the fine-level geometric fragments, selectively fusing the multi-channel fine-level geometric based on a coarse-level basic model and utilizing the weights, and outputting a multiscale three-dimensional scene model. According to the invention, the space-time invariance of the real texture and the phase dependence difference of the artifacts are utilized, and the high-frequency texture details are well reserved while the geometric wave noise of the rolling shutter is accurately removed.

Inventors

ZHANG JIANHAI
WANG QIAN
ZHOU XIN
LI JIANYI
WU DUN
FU JIAWEI
XU YANGJIE
ZHANG YUZHOU
WANG RUISHENG
GE JIANFEI
ZHOU TING

Assignees

宝略科技(浙江)有限公司

Dates

Publication Date: 20260508
Application Date: 20260116

Claims (10)

1. The multi-scale scene reconstruction method based on multi-source data fusion is characterized by comprising the following steps of: Carrying out row direction statistical signal extraction on each frame of image in an image frame sequence, and solving a modulation angle mark representing the relative phase position of the image acquisition moment in an ambient illumination change period; dividing the image frame sequence into a plurality of modulation angle channels based on the modulation angle markers; Unifying image frames in each modulation angle channel to a global coordinate system by using pose data, and constructing a coarse-level geometric segment representing scene macro topology and a fine-level geometric segment containing illumination strip characteristics aiming at each modulation angle channel; For the corresponding position in the reconstruction space, calculating the cross-channel divergence degree after obtaining the geometric feature quantity of the fine-level geometric segment of each modulation angle channel at the position, and generating a fine-level credibility weight for quantitatively distinguishing the real detail from the illumination stripe artifact according to the calculated cross-channel divergence degree; generating a coarse-level base model based on the coarse-level geometric segments of the modulation angle channel; and fusing the fine-level geometric fragments of all the modulation angle channels by utilizing the fine-level credibility weight, and outputting a multi-scale three-dimensional scene model by combining the coarse-level basic model.
2. The multi-scale scene reconstruction method based on multi-source data fusion according to claim 1, wherein for each frame image in the sequence of image frames, an arithmetic average of gray values of all pixels of each line of the frame image is calculated, and a one-dimensional line average signal is generated.
3. The multi-scale scene reconstruction method based on multi-source data fusion according to claim 2, comprising: carrying out frequency domain analysis on the one-dimensional line average signal, determining the corresponding spatial angular frequency at the maximum position of the spectrum response, and constructing a linear regression model containing cosine terms and sine terms; Fitting the one-dimensional line average value signal by using a least square method, and solving a first coefficient of a cosine term and a second coefficient of a sine term; A phase angle determined by the first coefficient and the second coefficient is calculated using a two-parameter arctangent function and is used as a modulation angle index.
4. The multi-scale scene reconstruction method based on multi-source data fusion according to claim 1, comprising: initializing a preset number of channel angle centers, and executing iterative clustering steps, wherein the method comprises the following steps: Calculating the circumferential distance between each modulation angle mark and each channel angle center, distributing each modulation angle mark to a channel corresponding to the channel angle center with the minimum circumferential distance, calculating the average angle of all modulation angle marks in each channel, and taking the average angle as the updated channel angle center; And repeating the iterative clustering step until a preset convergence condition is met, and obtaining a plurality of divided modulation angle channels.
5. The multi-scale scene reconstruction method based on multi-source data fusion according to claim 1, comprising: Setting coarse-level voxel resolution parameters, and establishing a global three-dimensional grid space according to the coarse-level voxel resolution parameters; For each modulation angle channel, traversing all image frames in the channel, and calculating global geometric observation data of each frame by using pose data, wherein the global geometric observation data is a three-dimensional point cloud set which is uniformly aligned to a world coordinate system after rigid body transformation; And mapping all global geometric observation data belonging to the channel into a three-dimensional grid space by adopting a voxelized grid fusion method, counting the number of points and barycenter coordinates falling into each grid unit, reserving effective voxels containing enough points, and connecting the barycenters of adjacent effective voxels to generate the coarse-level geometric fragment.
6. The multi-scale scene reconstruction method based on multi-source data fusion according to claim 5, comprising: setting a fine-level voxel resolution parameter, wherein the numerical value of the fine-level voxel resolution parameter is smaller than the width of the ambient lighting strip; constructing a high-precision grid based on fine-level voxel resolution parameters for each modulation angle channel; Calculating a directional distance value from the center of each micro voxel to a nearest observation point in the high-precision grid, and fusing multi-frame observation data by using a weighted average strategy; And extracting the zero level set isosurface by using a moving cube algorithm, and generating a fine-level geometric segment containing the characteristics of the illumination strip.
7. The multi-scale scene reconstruction method based on multi-source data fusion according to claim 6, comprising: For each position in the reconstruction space, acquiring a truncated symbol distance value of a fine-level geometric segment of each modulation angle channel at the position, wherein the truncated symbol distance value is a directed distance from the position to the nearest surface of the fine-level geometric segment; and taking the truncated symbol distance value as the geometric feature quantity, calculating the statistical variance of the geometric feature quantity of each modulation angle channel at the position, and taking the statistical variance as the cross-channel divergence at the position.
8. The multi-scale scene reconstruction method based on multi-source data fusion according to claim 7, comprising: For each position in the reconstruction space, calculating an exponent power of a natural constant, wherein the exponent part is a negative value of the cross-channel bifurcation degree at the position divided by twice the square of a preset sensitivity parameter, and obtaining a fine-scale credibility weight at the position, and the value range of the fine-scale credibility weight is 0 to 1.
9. The multi-scale scene reconstruction method based on multi-source data fusion according to claim 8, comprising: for each position in the reconstruction space, acquiring a coarse-level truncated symbol distance value of a coarse-level geometric segment of each modulation angle channel at the position; calculating the arithmetic average value of the coarse level cut-off symbol distance values of all modulation angle channels at the position by adopting a full-channel arithmetic average strategy, and taking the arithmetic average value as a coarse level basic distance value at the position; And forming a global unified coarse-level basic distance field by the coarse-level basic distance values at all positions in the reconstruction space, and taking the global unified coarse-level basic distance field as a coarse-level basic model.
10. The multi-scale scene reconstruction method based on multi-source data fusion according to claim 9, comprising: For each position in the reconstruction space, calculating the sum of fine-level credibility weights of all modulation angle channels at the position; judging whether the sum is lower than a preset credibility threshold value or not; If the sum is lower than the credibility threshold, directly adopting a coarse-level basic distance value of the coarse-level basic model at the position as a final symbol distance value of the position; If the sum is greater than or equal to the credibility threshold, carrying out weighted average on the truncated symbol distance value of each modulation angle channel at the position by utilizing the fine-level credibility weight, and taking the obtained weighted average as the final symbol distance value of the position; And extracting a zero level set surface from a final distance field formed by final symbol distance values of all positions in the reconstruction space by using an isosurface extraction algorithm, and generating a multi-scale three-dimensional scene model.

Description

Multiscale scene reconstruction method based on multisource data fusion Technical Field The invention relates to the technical field of three-dimensional scene reconstruction, in particular to a multi-scale scene reconstruction method based on multi-source data fusion. Background In the field of industrial digital twin and facility operation and maintenance inspection, a main stream operation mode is realized by utilizing a multi-source acquisition device mounted on a mobile robot or a handheld terminal to reconstruct a high-precision three-dimensional scene. To meet millimeter-scale measurement requirements, such operations are typically performed at an industrial site equipped with a high intensity ac lighting system, where algorithms need to map and construct a three-dimensional spatial geometric model based on the acquired multi-source image sequence. However, the ubiquitous nature of the stroboscopic light source in industrial sites is highly susceptible to asynchronous coupling with the rolling shutter mechanism of the imaging device, resulting in horizontal bands of alternating brightness in the original image. In the existing three-dimensional reconstruction process, the algorithm generally has difficulty in distinguishing illumination variation of a radiation domain from texture of a geometric domain, and the two-dimensional illumination stripes are easily and incorrectly interpreted as normal fluctuation of a three-dimensional surface, so that wavy artifacts which are not originally existed on the surface of the model are generated. The false geometric noise can mask the real defects such as micro cracks and the like, and seriously influences the reliability of automatic measurement, and if the conventional smooth filtering means is adopted for denoising, key high-frequency details such as nameplate characters, screw hole edges and the like can be erased while the artifacts are restrained, so that the high-fidelity of the model is difficult to maintain while the illumination interference is removed. Disclosure of Invention Aiming at the defects of the prior art, the invention provides a multiscale scene reconstruction method based on multisource data fusion, which solves the problems that geometric stripe artifacts are generated in a reconstruction model due to the coupling of a rolling shutter effect and ambient illumination modulation, and real high-frequency details are difficult to retain while the artifacts are removed. In order to achieve the above purpose, the invention is realized by the following technical scheme: Carrying out row direction statistical signal extraction on each frame of image in an image frame sequence, and solving a modulation angle mark representing the relative phase position of the image acquisition moment in an ambient illumination change period; dividing the image frame sequence into a plurality of modulation angle channels based on the modulation angle markers; Unifying image frames in each modulation angle channel to a global coordinate system by using pose data, and constructing a coarse-level geometric segment representing scene macro topology and a fine-level geometric segment containing illumination strip characteristics aiming at each modulation angle channel; For the corresponding position in the reconstruction space, calculating the cross-channel divergence degree after obtaining the geometric feature quantity of the fine-level geometric segment of each modulation angle channel at the position, and generating a fine-level credibility weight for quantitatively distinguishing the real detail from the illumination stripe artifact according to the calculated cross-channel divergence degree; generating a coarse-level base model based on the coarse-level geometric segments of the modulation angle channel; and fusing the fine-level geometric fragments of all the modulation angle channels by utilizing the fine-level credibility weight, and outputting a multi-scale three-dimensional scene model by combining the coarse-level basic model. Compared with the prior art, the method has the following beneficial effects: According to the multiscale scene reconstruction method based on multiscale data fusion, continuous time-varying dynamic light interference is discretized into a plurality of channel subsets with stable internal states by utilizing modulation angle marks, multiscale expression of coarse-level frameworks and fine-level details is constructed, an unsupervised true-false identification mechanism based on cross-channel geometric divergence degree is established by utilizing the fact that a real structure has space-time invariance and strip artifacts have phase-dependent intrinsic difference, and the mechanism can accurately lock and remove false geometric wave noise, meanwhile real high-frequency textures such as nameplate characters and micro cracks are well reserved, so that the technical bottleneck that key details are inevitably erased when the strip artifa