CN-116596773-B - ToF depth image denoising method based on graph Laplace regularization network

CN116596773BCN 116596773 BCN116596773 BCN 116596773BCN-116596773-B

Abstract

The application provides a ToF depth image denoising method based on a graph Laplace regularization network, which comprises the steps of acquiring raw data acquired by a ToF sensor; and removing noise of the raw data based on the graph Laplace regularization network to obtain a denoising depth map, and removing a region corresponding to the region with the confidence coefficient smaller than a threshold value by post-processing of the denoising depth map to obtain a final depth map. According to the scheme, a better denoising effect is obtained by using the graph Laplace regularization network, the acquisition of actual data is not needed for training, the training can be directly performed by using simulated Gaussian noise, and the method has good generalization capability.

Inventors

ZENG JIN
JIA JINGWEI
HE CHANGYONG
WANG JIANHUI
YANG CHEN
ZHAO SHENGJIE

Assignees

同济大学

Dates

Publication Date: 20260505
Application Date: 20230406

Claims (7)

1. A method for denoising a ToF depth image based on a graph laplace regularization network, the method comprising: acquiring raw data acquired by a ToF sensor; if the raw data is single-frequency raw data, the single-frequency raw data is converted into corresponding IQ data through the following formula: Wherein, the Output signals when the single frequency raw data corresponds to different phase shifts k=1, 2, 3..n; The raw data is subjected to noise removal based on a graph Laplace regularization network to obtain a denoising depth map, and the method comprises the following steps of: The single-frequency raw data are converted into corresponding IQ data; Denoising the IQ data based on a graph Laplace regularization network to obtain denoised IQ data; the de-noised IQ data is converted into the de-noised depth map; if the raw data is multi-frequency raw data, the multi-frequency raw data at least comprises low-frequency raw data and high-frequency raw data; The raw data is subjected to noise removal based on a graph Laplace regularization network to obtain a denoising depth map, and the method comprises the following steps of: the low-frequency raw data are converted into corresponding low-frequency IQ data, and the high-frequency raw data are converted into corresponding high-frequency IQ data; Denoising the low-frequency IQ data based on a graph Laplace regularization network to obtain denoised low-frequency IQ data, and denoising the high-frequency IQ data based on the graph Laplace regularization network to obtain denoised high-frequency IQ data; the de-noised low-frequency IQ data is converted into a low-frequency de-noising depth map, and the de-noised high-frequency IQ data is converted into a high-frequency de-noising depth map; and removing the region corresponding to the denoising depth map with the confidence coefficient smaller than the threshold value through post-processing to obtain a final depth map.
2. The method of claim 1, wherein the denoising the IQ data based on a graph laplace regularization network, to obtain denoised IQ data, comprises: the IQ data is input into a feature extraction network to obtain a feature map; determining a graph adjacency matrix according to the characteristic graph; determining a Laplace matrix according to the graph adjacency matrix; Inputting the IQ data into a pre-filtering network to obtain pre-filtered IQ data; the IQ data input coefficient prediction network obtains a prediction coefficient; And determining the de-noised IQ data according to the Laplacian matrix, the pre-filtered IQ data and the prediction coefficient.
3. The method of claim 2, wherein said determining said de-noised IQ data from said laplace matrix, said pre-filtered IQ data, said prediction coefficients comprises: constructing an equation based on the Laplace matrix, the pre-filtered IQ data, the prediction coefficient and the denoised IQ data; And solving the equation by a QP solver to obtain the de-noised IQ data.
4. The method of claim 1, wherein the converting the denoised IQ data to the denoised depth map comprises: Converting the de-noised IQ data into phase data; and calculating the depth of the denoising depth map according to the phase data.
5. The method of claim 1, wherein the denoising depth map removes, by post-processing, a region corresponding to a confidence level less than a threshold value, to obtain a final depth map, comprising: determining confidence according to the de-noised IQ data; and setting the region corresponding to the confidence coefficient smaller than the threshold value as an invalid value to obtain the final depth map.
6. The method of claim 1, wherein the denoising depth map removes, by post-processing, a region corresponding to a confidence level less than a threshold value, to obtain a final depth map, comprising: The low-frequency denoising depth map and the high-frequency denoising depth map are fused through phase expansion, so that a fused depth map is obtained; And removing the region corresponding to the confidence coefficient smaller than the threshold value by post-processing of the fusion depth map to obtain the final depth map.
7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements a tou depth image denoising method based on a graph laplace regularization network as claimed in any one of claims 1-6 when the program is executed by the processor.

Description

ToF depth image denoising method based on graph Laplace regularization network Technical Field The invention belongs to the technical field of computer vision, and particularly relates to a ToF depth image denoising method based on a graph Laplace regularization network. Background The three-dimensional imaging technology has wide application prospect and commercial value in the fields of mobile robots, remote medical treatment, three-dimensional televisions and the like. Accurate acquisition of depth information is a critical technique necessary for three-dimensional research and three-dimensional practical application. Thus, fast and high quality depth sensors are very popular, and Time-of-Flight (ToF) cameras have become an efficient, low cost, multi-purpose depth imaging solution in a wide variety of depth acquisition technologies (binocular, structured light, lidar, etc.). Noise exists in the depth image, the noise in the image is derived from various processes such as image acquisition, compression, transmission and the like, and the image denoising is the process of reducing the noise in the image. The image denoising algorithm follows the principle that some details and edge information of the original image can be kept as much as possible while noise is separated, i.e. useful information and noise data in the image can be separated. In addition, when the neural network is adopted for denoising, the generalization capability is limited, and the acquisition of actual data is required. Disclosure of Invention An object of an embodiment of the present disclosure is to provide a ToF depth image denoising method based on a graph laplace regularization network. In order to solve the technical problems, the embodiment of the application is realized by the following steps: In a first aspect, the present application provides a method for denoising a ToF depth image based on a graph laplace regularization network, the method comprising: acquiring raw data acquired by a ToF sensor; the raw data is subjected to noise removal based on a graph Laplace regularization network to obtain a denoising depth graph; and removing the region corresponding to the confidence coefficient smaller than the threshold value from the denoising depth map through post-processing to obtain a final depth map. In one embodiment, if the raw data is single frequency raw data; the raw data is subjected to noise removal based on a graph Laplace regularization network to obtain a denoising depth map, and the method comprises the following steps of: Converting the single-frequency raw data into corresponding IQ data; Denoising the IQ data based on a graph Laplace regularization network to obtain denoised IQ data; and (5) converting the de-noised IQ data into a de-noised depth map. In one embodiment, the single frequency raw data is converted into corresponding IQ data by the following formula: Wherein V k is the output signal of the single frequency raw data corresponding to different phase shifts k=1, 2,3. In one embodiment, denoising the IQ data based on a graph laplace regularization network to obtain denoised IQ data, including: Inputting IQ data into a feature extraction network to obtain a feature map; determining a graph adjacency matrix according to the feature graph; determining a Laplace matrix according to the graph adjacency matrix; inputting the IQ data into a pre-filtering network to obtain pre-filtered IQ data; inputting IQ data into a coefficient prediction network to obtain a prediction coefficient; And determining the de-noised IQ data according to the Laplace matrix, the pre-filtered IQ data and the prediction coefficient. In one embodiment, determining the de-noised IQ data according to the laplace matrix, the pre-filtered IQ data, and the prediction coefficients comprises: Constructing an equation based on the Laplace matrix, the pre-filtered IQ data, the prediction coefficient and the denoised IQ data; and solving an equation by a QP solver to obtain de-noised IQ data. In one embodiment, the denoising method for converting the IQ data into the denoising depth map comprises the following steps: The de-noised IQ data is converted into phase data; and calculating the depth of the denoising depth map according to the phase data. In one embodiment, removing, by post-processing, a region corresponding to a confidence level less than a threshold value from the denoising depth map to obtain a final depth map, including: Determining confidence according to the de-noised IQ data; and setting the region corresponding to the confidence coefficient smaller than the threshold value as an invalid value to obtain a final depth map. In one embodiment, if the raw data is multi-frequency raw data, the multi-frequency raw data at least includes low-frequency raw data and high-frequency raw data; the raw data is subjected to noise removal based on a graph Laplace regularization network to obtain a denoising depth map, and the method compris