CN-115375745-B - Absolute depth measurement method based on polarized microlens light field image parallax angle

CN115375745BCN 115375745 BCN115375745 BCN 115375745BCN-115375745-B

Abstract

The invention relates to the technical field of light field imaging measurement, in particular to an absolute depth measurement method based on a polarized microlens light field image parallax angle. The method comprises the steps of 1) classifying image pixel levels based on multi-view sub-aperture depth maps to provide a reference for optimizing a parallax map, 2) constructing a matching cost equation based on pixel classification results to achieve global optimization of a relative depth map from thick to thin, 3) constructing a linear mapping model from the relative depth map to an absolute depth map based on a hardware structure of a micro-lens light field camera to achieve depth calibration, and 4) combining the depth calibration with internal parameters of the light field camera to obtain absolute depth physical parameters of an observation target through calculation. Therefore, physical measurement and characterization of the three-dimensional absolute size of the real observation object are realized, absolute depth information of a target to be detected in the light field image can be obtained, and especially, absolute depth information acquisition of weak textures and shielding targets can be realized.

Inventors

YAN LEI
SHANG KE
YAN XU
ZHAO HONGYING
SUN SHANLIN
LIN YI
Qin Jianqi
ZHAO HAIMENG
ZUO ZHENGKANG
ZOU AICHENG
ZHANG ZIHAN

Assignees

桂林航天工业学院
北京大学

Dates

Publication Date: 20260512
Application Date: 20220602

Claims (7)

1. The absolute depth measurement method based on the polarized microlens light field image parallax angle is characterized by comprising the following steps: step one, providing a reference for optimizing a parallax map based on image pixel-level classification of a multi-view sub-aperture depth map; step two, constructing a matching cost equation based on pixel classification results, and realizing global optimization relative depth map from thick to thin; thirdly, based on a hardware structure of the micro-lens light field camera, establishing a linear mapping model from a relative depth map to an absolute depth map, so as to realize depth calibration; Combining depth calibration with internal parameters of the light field camera, and calculating to obtain absolute depth physical parameters of an observation target; the matching cost equation in the second step is that all pixels are divided into three types of different pixel types by adopting a pixel classification method, different cost functions are constructed by utilizing multi-view correlation, and the matching cost equation is determined by a formula (3): (3) Where V represents the set of all view angles of the light field image, V represents the image angle, p represents the image space coordinates (x, y), mean represents the median function, min represents the minimum function, Representing the constructed initial matching cost equation, wherein I (p, c) represents the gray value of the pixel p in the central sub-aperture c, I (p, v) represents the gray value of the pixel p in the sub-aperture v, N (v) is the number of sub-apertures contained in the neighborhood sub-aperture set, and the number is an integer larger than 0.
2. The method for absolute depth measurement based on polarized microlens light field image parallax angle according to claim 1, wherein the step one is characterized in that the optimization reference for establishing a parallax image is obtained by utilizing an image pixel classification means to obtain an image depth image, dividing pixels into shielded pixels, weak texture pixels and stable pixels according to a multi-view sub-aperture depth image, enabling the extracted depth image to be consistent on a local object, dividing the image into a patch structure, the shielded pixels in the image are obtained by a formula (1), (1) Where depth c (p c ) represents the depth value of pixel p c in the central sub-aperture c, depth i (p c +(n c −n i ) represents the homonymous point depth value in the ith sub-aperture corresponding to the central sub-aperture pixel p c , where n c is the index value of the central sub-aperture, ni is the index value of the ith non-central sub-aperture, i is the sequence number of the non-central sub-aperture, where i=1, 2, the..k, k is the total number of non-central sub-apertures, (n c −n i ) represents the index difference of the central sub-aperture and the ith sub-aperture for determining the offset position of the homonymous point, and the weak texture pixel is obtained by formula (2): Representing the minimum of a point matching cost function, Is the next smallest value of the matching cost function, Represents a threshold value and, (2)。
3. The absolute depth measurement method based on polarized microlens light field image parallax angle according to claim 2, wherein the global optimization relative depth map in the second step is characterized in that a cost function of different types is constructed by utilizing multi-view correlation, the cost function is optimized by utilizing an adaptive window so as to establish a relative depth map, the relative depth map is segmented into a patch structure after noise removal by utilizing weighted median filtering, the depth information of the whole patch is fitted according to accurate depth information of stable pixels by utilizing RANSAC plane fitting for each patch, and the matching cost equation is corrected by utilizing the depth information after fitting, so as to obtain the global optimal relative depth map through multiple iterations, wherein the iterative formula is (4) Represents the initial matching cost equation for the ith iteration Representing the matching cost equation obtained by WTA (Winner takes all) method: (4)。
4. the absolute depth measurement method based on the polarized microlens light field image parallax angle according to claim 3, wherein the absolute depth scaling in the third step obtains relative depth information by a stereo matching method, builds a linear mapping model based on the hardware structure of the microlens light field camera, and obtains the following formula (5) according to a polar coordinate parallax angle calculation formula: (5) Wherein the method comprises the steps of The parallax is represented by a visual difference, Representing the angle of the parallax of the person, Representing the image point to imaging plane distance, z represents the image point to microlens array distance, and f 1 is the microlens focal length.
5. The method for absolute depth measurement based on polarized microlens light field image parallax angle according to claim 4, wherein in the obtaining of the relative depth information, a linear relation expression of depth distance information and parallax information is deduced and established from a hardware structure of the microlens camera, an accurate depth calibration curve is completed by using a small number of points, and the following formula (6) holds: (6) Wherein F 2 is the focal length of the main lens, depth0 represents parallax, and the linear model of absolute depth Z and relative depth is obtained by combining the equality of F# of the main lens and the micro lens as formula (7): (7) Where Z is the absolute depth, B is the baseline distance, and c 1 , c 2 is the definition of the weight coefficient.
6. The method for measuring absolute depth based on the parallax angle of the polarized microlens light field image according to claim 5, wherein the depth calibration in the third step is characterized in that the microlens light field camera is utilized to vertically shoot the checkerboard image, the chess and card images with different depths are respectively extracted to obtain relative depth information, meanwhile, the true distances are recorded, after a plurality of sets of data are obtained repeatedly, a scatter diagram of the relative depth information and the reciprocal of the true distances is built, and meanwhile, the specific parameters c 1 and c 2 are determined by adopting a least square method to carry out linear fitting.
7. The method of claim 6, wherein the absolute depth measurement of the angle of view of the polarized microlens light field image is characterized in that the absolute depth physical parameter of the observed object in the fourth step is obtained by photographing the checkerboard light field image at multiple angles, determining an internal reference matrix K according to an internal reference calibration method of the light field camera, determining the absolute depth by combining the relative depth map and the calibration curve, further calculating to obtain the absolute three-dimensional information of the photographed object, and obtaining parameters for each pixel on the image The calculation formulas of the space three-dimensional coordinates are formulas (8), (9): (8) (9)。

Description

Absolute depth measurement method based on polarized microlens light field image parallax angle Technical Field The invention relates to the technical field of light field imaging measurement, in particular to an absolute depth measurement method based on a polarized microlens light field image parallax angle. Background Along with the continuous development of social economy, the demand for three-dimensional remote sensing information is increasing, and the rapid, accurate and economic acquisition of three-dimensional information of a ground object is more and more important. The traditional remote sensing imaging devices all adopt digital cameras, and are composed of optical lens combinations and electronic photosensitive elements, and the imaging process is to convert three-dimensional information of an object into two-dimensional plane image information. The recovery of the surface information of the three-dimensional object from the image is the inverse of the imaging process, and a plurality of images with different angles are needed to be shot by a digital camera, and the three-dimensional information of the object is recovered through soft part calculation and processing. Because a large amount of information is lost in the process of projecting the three-dimensional world to the two-dimensional image, the problem of uncertainty of the inverse process is inherent, the three-dimensional reconstruction is complex in calculation, the reconstruction accuracy is influenced by various factors, and the acquisition of the three-dimensional remote sensing information is severely restricted. With the advent of light field imaging technology, a microlens light field camera has been able to acquire direction information of light rays inside the camera, and this characteristic enables a single Zhang Guangchang image to acquire three-dimensional information of a target, thereby providing a new direction for acquiring three-dimensional information of a scene. Compared with the traditional camera imaging mode, the three-dimensional reconstruction mode of light field imaging is more efficient and real, for a light field camera image, the key of stereo matching parallax calculation based on sub-aperture images is to find corresponding projection points in left and right images, the process needs to consider the problems of accuracy, processing speed and the like, and pixel matching is very key in the whole stereo matching process. If a shielding area exists in the light field image or textures are weaker, the traditional depth extraction method based on stereo matching can generate a mismatching phenomenon, so that how to adopt an effective means to reduce the mismatching problem is very important. Meanwhile, in order to acquire absolute depth information of a target, the extracted relative depth information is required to be calibrated, the invention takes a first generation micro lens light field camera as a hardware basis, and light field images acquired by the camera are calculated, and relates to an absolute depth measurement method based on the parallax angle of the polarized micro lens light field images. Disclosure of Invention The invention aims to provide an absolute depth measurement method based on a polarized microlens light field image parallax angle, which can acquire absolute depth information of a target to be detected in a light field image, and particularly can realize absolute depth information acquisition of a weak texture and a shielding target. The invention aims to solve the problems, and provides a method for measuring absolute depth based on the parallax angle of a polarized microlens light field image, which comprises the following steps: step one, providing a reference for optimizing a parallax map based on image pixel-level classification of a multi-view sub-aperture depth map; step two, constructing a matching cost equation based on pixel classification results, and realizing global optimization relative depth map from thick to thin; thirdly, based on a hardware structure of the micro-lens light field camera, establishing a linear mapping model from a relative depth map to an absolute depth map, so as to realize depth calibration; And step four, combining the depth calibration with the internal parameters of the light field camera, and calculating to obtain the absolute depth physical parameters of the observed target. Physical measurement and characterization of the three-dimensional absolute size of the real observation object are realized. Firstly, classifying all pixel points in an image, and for pixels with weaker shielding and texture of a polarized microlens light field image, obtaining a pathological matching cost equation by using the conventional method, wherein the depth corresponding to the minimum matching cost value is often not the correct depth, and the phenomenon is called mismatching. Attempts to eliminate mismatch are difficult, but the correct depth of non-occluded, n