CN-115272573-B - Real-time reconstruction method of dynamic human body

CN115272573BCN 115272573 BCN115272573 BCN 115272573BCN-115272573-B

Abstract

The invention discloses a real-time reconstruction method of a dynamic human body, which comprises the following steps of connecting a human body measuring device, solving the relative gesture between the internal parameters of a camera and two phases, acquiring a reference state image, dividing a speckle region, reconstructing a reference state three-dimensional image, acquiring an image on a time sequence, reconstructing the three-dimensional image of the human body speckle on the time sequence, and carrying out rapid triangularization on a dynamic three-dimensional point cloud. On the premise of ensuring the reconstruction precision, the point cloud density of the reconstructed human body data can be greatly improved, so that the reconstructed point cloud data can more fully express the local details of the human body, the real-time calculation of the dynamic human body can be completed, and the on-line measurement and reconstruction of the human body are realized.

Inventors

TANG ZHENGZONG
LIU JIALE
LI LEIGANG
ZHANG PEI
LU JINJI

Assignees

渭南领智三维科技有限公司
渭南领智三维科技有限公司

Dates

Publication Date: 20260421
Application Date: 20220726
Priority Date: 20220726

Claims (11)

1. A method for real-time reconstruction of a dynamic human body, the method comprising the steps of: S1, connecting a anthropometric device, wherein the anthropometric device comprises a first camera and a second camera; s2, calibrating the first camera and the second camera, and solving the relative attitude between the parameters in the cameras and the two cameras ; Wherein the method comprises the steps of And A rotation matrix and a translation vector for a camera coordinate system of the second camera to a camera coordinate system of the first camera; S3, acquiring a reference state image, namely, taking a first image acquired by a camera and used for dividing a calculation area as the reference state image, opening a speckle projector, projecting speckle on the surface of a human body, and acquiring a human body speckle image; S4, dividing speckle areas, namely dividing an area to be calculated according to the speckle distribution area and the human body distribution area on the left image, and determining the matching of the left image and the right image and reconstructing the range of the human body; S5, three-dimensional reconstruction of a reference state, namely firstly selecting seed points on a left image, then searching matching points of the seed points in a range of a strip set up and down the epipolar line, then completing matching of all data points in subareas divided in S4, and finally utilizing the calibration in S2 Reconstructing a human body three-dimensional point cloud in a reference state; S6, collecting images on a time sequence, namely collecting image streams of human body speckle at the same time; S7, three-dimensional reconstruction of human body speckle on a time sequence, namely, aiming at the image flow acquired in the step S6, according to the continuity of human body dynamic motion, directly taking deformation parameters of speckle data points at the previous moment relative to a reference state as corresponding initial values of corresponding data points at the current moment, finishing a fine matching process of each data point on a GPU, and finishing verification and repair of the GPU matching result on the CPU to finish points failing in fine matching; S8, outputting the three-dimensional point cloud reconstructed by the human body in time sequence in real time; in step S7, three-dimensional reconstruction on time series is performed as follows: s71, using a sequence of t0, t1, t2, t 3..tn to represent a time sequence of camera acquired images; S72, finishing data point matching of the images from the time t0 to the time t1 and solving deformation parameters; S73, acquiring initial value data of data points of the image at the time ti+1, taking the positions of all calculation nodes on the image at the time ti+1 as the fine matching initial values of the corresponding nodes on the image at the time ti+1, and taking the deformation parameters of all data nodes on the image at the time ti as the iteration initial values of the deformation parameters of the corresponding nodes on the image at the time ti+1; s74, copying data, namely copying the initial value data in the step S72 into a global memory of the GPU, copying left and right image data into a texture memory of the GPU, and copying the relative gestures of the first camera and the second camera And the distortion model coefficients are copied to the constant memory Constant menory of the GPU; S75, executing a kernel function, completing a fine matching process of data points in the GPU kernel function, reconstructing the same-name points after fine matching, and outputting correct data points after fine matching and reconstructed three-dimensional data points by verifying correct data points matched with distinguishing marks and incorrect data points matched with distinguishing marks after the GPU is matched; S76, copying data, namely copying the data point pairs solved in the step S74 and deformation parameters of the point pairs, three-dimensional point coordinates into a CPU memory; S77, aiming at the data point pairs copied back to the CPU in the step S75, judging matching failure points and repairing part of node data which are failed in fine matching according to the matching failure flag bits copied back by the GPU in the CPU, and reconstructing coordinates of the repairing points on the CPU; And S78, assigning the point data repaired in the step S77 to the initial value of the data point in the image at the time ti+2, and circularly executing the steps S73-S77 until the user stops the program.
2. The method of real-time reconstruction of a dynamic human body according to claim 1, wherein in the step S1, the human body measuring device further comprises a workstation, a speckle projector, a camera and a speckle projector control box, wherein the LED light source in the speckle projector is controlled to work through command pulses, the LED light source generates short-wavelength blue light, and the speckle projector enables the light source to project on the surface of the human body through a speckle sheet according to the design requirement of an optical path to generate a speckle pattern.
3. The method for real-time reconstruction of a dynamic human body according to claim 1, wherein in the step S2, the steps of calibrating the first camera and the second camera are as follows: S21, aiming at a plurality of calibration plates with different directions which are used for calibration and are sequentially placed in the depth of field range of the camera, the first camera and the second camera collect the calibration plates at the plurality of positions simultaneously and sequentially to generate a plurality of images; S22, performing image processing on the acquired images, calculating pixel coordinates of coding points and non-coding points on the calibration plate, and solving internal parameters of the cameras and relative postures among the cameras according to a beam adjustment algorithm by utilizing the coordinates on each image and the three-dimensional coordinates on the calibration plate.
4. The method for real-time reconstruction of a dynamic human body according to claim 1, wherein in the step S4, the step of dividing the speckle region is detailed as follows: And selecting a speckle region to be calculated with a complex shape on the left camera image in a reference state by adopting three basic shape tools of a rectangle, an ellipse and a polygon, and intersection and union Boolean operation between the three shapes, wherein the speckle region to be calculated with the complex shape is used for a calculation range of subsequent image matching.
5. The method for real-time reconstruction of a dynamic human body according to claim 1, wherein in the step S5, the step of three-dimensionally reconstructing the reference state is: S51, seed point selection, namely automatically selecting a seed point array on the first camera acquired image according to interval step parameters of seed point distribution, wherein the seed point array is expressed as Wherein ; S52, rough matching of seed points, wherein the relative gestures of the first camera and the second camera calibrated by the cameras in S2 are used for Solving the seed point by using epipolar constraint in double-view geometry Estimated position on the second camera acquisition image (right image) The position corresponds to a line on the right image; adopting correlation strategies of left and right image subregions, such as Normalized Cross Correlation (NCC), performing integral pixel search in the strip range of polar lines, selecting the point with the maximum correlation coefficient as a matching point to obtain integral pixel coordinates matched with seed points ; S53, fine matching of seed points, on the basis of the step S52, to obtain Performing fine matching for an initial value, and performing gray interpolation on a search domain by utilizing a subpixel interpolation algorithm to obtain coordinates of a subpixel level and a corresponding gray value; After sub-pixel interpolation is completed, sub-pixel matching is performed by adopting a fine matching correlation coefficient calculation function, and the center of a search area with the largest correlation coefficient is selected on a right image as a matching point through searching of sub-pixels, so that sub-pixel matching coordinates are obtained ; S54, diffusing seed points, and solving the coordinate increment of the corresponding point of the seed point array on the right image according to the coordinates of the corresponding point of the seed point array on the right image in the step S53 relative to the coordinates of the seed point on the left image ; Taking the seed point as the center, diffusing a certain step length to the four directions of the seed point, namely the up, down, left and right directions to obtain four image points, and taking the seed point as the center Assigning coarse matching coordinate increment to the four image points, then directly carrying out the fine matching process in the step S53 to obtain the fine matching coordinates and coordinate increment of the four points on the right image, and then sequentially taking the four points as centers to complete the matching of all the data points in the speckle area in the step S54 by adopting the same thought; S55, three-dimensional reconstruction of data points in the speckle region, and completing three-dimensional reconstruction of all the matching point pairs obtained in the step S54 on the GPU.
6. The method for real-time reconstruction of a dynamic human body according to claim 1, wherein in step S75, the verifying that the distinguishing mark matches the correct and the incorrect data points specifically includes verifying the verification item, assigning 1 to the correct data point matching and assigning-1 to the failed data point matching, and the verification item includes: a. The equation in iteration has no solution, and the iteration is not converged due to excessive iteration times; b. the gray residual error of the corresponding sub-region of the image is too large; c. the re-projection error of the reconstructed three-dimensional point is larger than a threshold value; d. Reconstructing three-dimensional point coordinates the depth is greater than the depth of field.
7. The method for real-time reconstruction of a dynamic human body according to claim 1, wherein in step S8, three-dimensional point cloud data of the human body is output in real time.
8. The method for reconstructing a dynamic human body in real time according to claim 1, wherein the internal parameters in step S2 include a focal length, a principal point deviation and a distortion model coefficient, and the first camera and the second camera are calibrated by a multi-step method and a beam adjustment method, the multi-step method is specifically that a calibration plate is respectively placed at a plurality of different positions in a common field of view of the first camera and the second camera, and the two cameras acquire a plurality of pairs of images simultaneously.
9. The method of real-time reconstruction of a dynamic human body according to claim 1, wherein in step S5, the set stripe range is a range of 3 pixels above and below a epipolar line.
10. A method for real-time reconstruction of a dynamic human body as set forth in claim 3, wherein, The calibration plate in step S21 is a plane plate in which annular coding mark points and circular non-coding mark points are regularly distributed, and the size of the calibration plate used in calibration is consistent with the measurement format of the camera, that is, the imaging of the calibration plate in the camera occupies approximately 80% of the area of the image within the depth of field of the camera.
11. The method of real-time reconstruction of dynamic human body according to claim 5, wherein in step S53, sub-region sub-pixel search and matching are performed by using zero-mean normalized minimum distance sum-of-squares coefficient ZNSSD, in step S55, matching point data is copied to GPU global memory, and the relative postures of the first camera and the second camera are copied And copying the distortion model coefficients into a constant memory of the GPU, completing three-dimensional reconstruction of the data points on the GPU by utilizing a kernel function for removing distortion and reconstructing calculation of the GPU, and finally copying the three-dimensional point data back into the CPU.

Description

Real-time reconstruction method of dynamic human body Technical Field The invention relates to the fields of computer vision, image processing technology and three-dimensional measurement, in particular to a real-time reconstruction method of a dynamic human body. Background With the continuous progress of technology, three-dimensional measurement technology is continuously developed. The prior art method comprises the technologies of stereoscopic vision, close-range photogrammetry, structured light projection, digital image correlation method and the like. The method relies on the characteristics of the surface structure, texture and the like of the object, the reconstructed point cloud is sparse, the operation amount is large, and the quality of the point cloud is often low. The close-range photogrammetry technology uses one or more cameras to shoot a plurality of images with different directions on an object, calculates the image coordinates of corresponding points in images with different visual angles by using a digital image processing technology, and solves the three-dimensional coordinates of the corresponding points by using an optimization algorithm. The method is different from a stereoscopic vision method in that firstly, the stereoscopic vision method needs to calibrate a camera in advance, but the photogrammetry technology does not need to calibrate in advance, and secondly, the three-dimensional data information is directly solved by utilizing an optimization algorithm after a two-dimensional image is obtained. The method has the advantages of simple hardware structure, flexible shooting mode and wide application in reconstruction and measurement of large scenes. The structured light projection technology projects a spot pattern (such as random speckle and dot spots) with a specific structure to the surface of an object through a spot projector on the basis of stereoscopic vision, and the speckle pattern of the surface of the object is acquired through a camera, so that three-dimensional information of the surface of the object is calculated. Compared with a stereoscopic vision method and a photogrammetry technology, the method has the advantages of higher efficiency for acquiring three-dimensional data and higher measurement precision, and is widely applied to the fields of industrial measurement, human body scanning and the like. The structured light projection technology is combined with the digital image correlation method to complete the three-dimensional reconstruction of the dynamic human body, so that the high-density point cloud is reconstructed, the correlation of image subareas needs to be calculated in a large amount, a large amount of calculation resources are needed, and the operation is difficult. Disclosure of Invention The invention aims to solve the problems that a large amount of calculation resources are needed for completing three-dimensional reconstruction of a dynamic human body and operation is difficult, and provides a real-time reconstruction method of the dynamic human body. The technical problems of the invention are solved by the following technical scheme: The invention discloses a real-time reconstruction method of a dynamic human body, which comprises the following steps: S1, connecting a anthropometric device, wherein the anthropometric device comprises a first camera and a second camera; s2, calibrating the first camera and the second camera, and solving relative postures (R, T) between parameters in the cameras and the two cameras; Wherein R and T are rotation matrices and translation vectors of the camera coordinate system of the second camera to the camera coordinate system of the first camera; S3, acquiring a reference state image, namely, taking a first image acquired by a camera and used for dividing a calculation area as the reference state image, opening a speckle projector, projecting speckle on the surface of a human body, and acquiring a human body speckle image; S4, dividing speckle areas, namely dividing an area to be calculated according to the speckle distribution area and the human body distribution area on the left image, and determining the matching of the left image and the right image and reconstructing the range of the human body; S5, three-dimensional reconstruction of a reference state, namely firstly selecting seed points on a left image, then searching matching points of the seed points in a range of a strip set up and down from an epipolar line, then completing matching of all data points in the subareas divided in S4, and finally reconstructing three-dimensional point cloud of a human body under the reference state by using (R, T) calibrated in S2; S6, acquiring images on a time sequence, namely acquiring image streams of human body speckles simultaneously in a hardware triggering mode; s7, three-dimensional reconstruction of human body speckle on a time sequence, namely, aiming at the image flow acquired in the step S6, according t