CN-121999322-A - Data quality parameterization evaluation method based on improved convolutional neural network

CN121999322ACN 121999322 ACN121999322 ACN 121999322ACN-121999322-A

Abstract

The invention discloses a data quality parameter evaluation method based on an improved convolutional neural network, and belongs to the technical field of optical remote sensing quantification. The method comprises the steps of firstly carrying out image enhancement and feature refinement on a reference spectrum image and a spectrum image to be evaluated, then carrying out robust sparse feature analysis, completing independent subspace transformation of data, and finally constructing a quality evaluation measurement function to carry out image quality evaluation. The method can effectively process the problems of complex image distortion and noise, improves the accuracy and the instantaneity of spectrum data quality evaluation, has strong adaptability and application potential, can not only cope with data with large noise, but also provide real-time optimization, is suitable for quality assurance of large-scale data, and promotes the wide application of spectrum technology in various fields.

Inventors

Tong Chiming
JIANG CHENG
MA ZHONGQI
XU PENGMEI
Jie Yongshi
WANG YU
CUI ZIHAN

Assignees

北京空间机电研究所

Dates

Publication Date: 20260508
Application Date: 20251212

Claims (10)

1. The data quality parameterization evaluation method based on the improved convolutional neural network is characterized by comprising the following steps of: step one, respectively carrying out image enhancement processing on a noiseless reference spectrum image and a spectrum image to be evaluated; Generating a high-dimensional feature matrix by feature expansion of the enhanced spectral image to be evaluated and the enhanced reference spectral image respectively, and then performing robust sparse feature analysis on the high-dimensional feature matrix to highlight key features, and outputting a spectral feature matrix of the spectral image to be evaluated and a spectral feature matrix of the reference spectral image reconstructed by the sparse features; Step three, based on the two spectrum feature matrixes obtained in the step two, an improved convolutional neural network is established, and the improved convolutional neural network performs multi-layer feature mapping to obtain a low-dimensional feature matrix of the spectrum image to be evaluated and a low-dimensional feature matrix of the reference spectrum image after mapping; And step four, constructing a quality evaluation metric function based on the two low-dimensional feature matrixes obtained in the step three, and comprehensively evaluating the spectrum data quality of the spectrum image to be evaluated.
2. The data quality parameterization evaluation method based on the improved convolutional neural network according to claim 1 is characterized in that robust sparse feature analysis is conducted on a high-dimensional feature matrix to highlight key features, an optimized model of the robust sparse feature analysis is established, an optimal solution of a spectrum feature matrix reconstructed by the sparse features is obtained, and the optimized model of the robust sparse feature analysis is: in the formula, The spectrum data matrix of the enhanced image to be evaluated comprises n samples and d spectrum features; for the spectrum characteristic matrix reconstructed by the sparse characteristic, reducing the high-dimensional spectrum characteristic from d dimension to k dimension; correcting the mean deviation of the spectrum data as a deviation vector; Is a parameter for controlling sensitivity to outliers; Is a unit matrix; Constraint conditions of the optimization model are as follows: And is also provided with , =I constrained sparse projection matrix Has the advantages of the orthogonality, the good compatibility, And limiting the sparse projection matrix P to enter an optimization model by only h rows of features, wherein I is a low latitude identity matrix.
3. The data quality parameterization evaluation method based on the improved convolutional neural network according to claim 2, wherein an iterative re-weighting algorithm is adopted to obtain an optimal solution of the sparse projection matrix P, specifically: Converting an optimized model of the robust sparse feature analysis, wherein the optimized model satisfies the following conditions in each iteration: in the formula, Representing the trace of the matrix; in the formula, Representing that the deviation vector b is copied and expanded on all sample dimensions to form a matrix consistent with the input data matrix dimension; d is a diagonal matrix, and the ith diagonal element of D is: 。
4. The data quality parameterization evaluation method based on the improved convolutional neural network according to claim 1, wherein the multi-layer feature perception of the improved convolutional neural network is expressed as: in the formula, Indicating the number of convolutional layers in the select modified convolutional neural network, Is the first The number of feature maps for the individual convolutional layers, Representing that the image to be evaluated is contained in the perceptual representation as a feature map of layer 0; the multi-layer convolution feature joint modeling and fusion feature of the sparse spectrum feature and the improved convolution neural network is expressed as follows: in the formula, Representing a spectrum feature matrix obtained by robust sparse feature analysis, wherein the symbol [: ] represents feature level stitching; The mapping process obtains an optimal transformation matrix through iterative optimization of the depth network model to realize mapping from the high-dimensional feature space to the low-dimensional independent subspace: in the formula, Representing the mapped low-dimensional feature matrix, Is based on improving the mapping of convolutional neural networks.
5. The method for parameterized evaluation of data quality based on an improved convolutional neural network of claim 4, wherein the quality evaluation metric function is constructed based on the overall metric function based on the texture similarity function and the structural similarity metric function according to the mapped low-dimensional feature matrix and by weighting and summing the texture and structural similarity of all convolutional layers: in the formula, And The weighting coefficients of texture similarity and structural similarity respectively, 、 Low-dimensional feature matrix respectively representing reference image extraction Low-dimensional feature matrix extracted from mean value and image to be evaluated Is the average value of (2); is a constant; 、、 The variance of the low-dimensional feature matrix x extracted from the reference image, the variance of the low-dimensional feature matrix y extracted from the image to be evaluated and the covariance of the two are respectively.
6. The method for parameterized evaluation of data quality based on improved convolutional neural network of claim 5, wherein the texture similarity function and the structural similarity metric function are respectively: Texture similarity measurement function : In the formula, And Feature mapping on each channel vector of the data for the improved convolutional neural network, And Respectively spatial average value, i is the convolution layer number of the improved convolution neural network, Is a constant term; Structural similarity metric function : In the formula, For the covariance of each channel, And As a function of the variance of the values, Is a small constant (e.g ) For avoiding the case where the denominator is zero.
7. The improved convolutional neural network-based data quality parameterized evaluation method of claim 6, wherein the overall metric function The method comprises the following steps: in the formula, And Respectively, the weighting coefficients of texture similarity and structural similarity; And Respectively representing the feature matrix extracted from the reference image and the image to be evaluated after the processing of the step three.
8. The improved convolutional neural network-based data quality parameterization evaluation method according to claim 5 is characterized in that the quality evaluation metric function is based on comprehensive evaluation of the spectrum data quality of the spectrum image to be evaluated, the comprehensive evaluation method comprises the steps of substituting low-dimensional feature matrixes of the spectrum image to be evaluated and the reference spectrum image into the quality evaluation metric function, calculating the texture and structure similarity among feature graphs of each layer, weighting and fusing feature responses of different layers, and finally mapping the weighted overall similarity result into a quality evaluation value between 0 and 1.
9. The method for parameterized evaluation of data quality based on improved convolutional neural network of claim 5, wherein training the improved convolutional neural network to obtain weighting coefficients for texture similarity and structural similarity And 。
10. The data quality parameterization evaluation method based on the improved convolutional neural network according to claim 1, wherein the improved convolutional neural network performs the following critical structural modification on a standard VGG16 network model, wherein all maximum pooling layers in the standard VGG16 network model are replaced by anti-aliasing filtering downsampling layers, and a low-pass filter kernel is introduced to prevent spectrum aliasing in the downsampling process, and the modification model is defined as: In the formula, the product of points by points is as follows, Is a fuzzy kernel implemented by using a Hanning window, and the cut-off frequency of the fuzzy kernel is constrained to avoid aliasing; The improved convolutional neural network obtained after correction keeps the output structure of the convolutional layer, ensures that complete characteristic information from the bottom texture to the high-level semantic can be captured, and simultaneously keeps the original information of an input image as the 0 th layer characteristic.

Description

Data quality parameterization evaluation method based on improved convolutional neural network Technical Field The invention relates to a data quality parameter evaluation method based on an improved convolutional neural network, and belongs to the technical field of optical remote sensing quantification. Background With the rapid development of optical imaging technology, the application of spectrum data in the fields of environmental monitoring, geological exploration and the like is increasing. However, the high dimensionality and complexity of the spectral data, as well as the effects of noise, distortion, and sensor errors, lead to data quality instability, which presents challenges for subsequent processing and analysis. Most of the existing spectrum data quality evaluation methods depend on single indexes (such as signal-to-noise ratio, mean square error and the like), but the methods have limitations in multi-source data fusion, high-dimension data processing and noise removal, and lack self-adaptive capability, so that the requirements of high precision and real-time performance in complex application scenes can not be met. Therefore, there is a need for a multi-dimensional, comprehensive and adaptable method for evaluating the quality of spectral data. Disclosure of Invention The technical problem solved by the invention is to overcome the limitation of the traditional spectrum data quality evaluation method, particularly the defects in high-dimensional data processing and noise removal, and provide a data quality parameterization evaluation method based on an improved convolutional neural network, so as to realize efficient and accurate spectrum data quality analysis and optimization. The technical scheme of the invention is as follows: a data quality parameterization evaluation method based on an improved convolutional neural network comprises the following steps: step one, respectively carrying out image enhancement processing on a noiseless reference spectrum image and a spectrum image to be evaluated; Generating a high-dimensional feature matrix by feature expansion of the enhanced spectral image to be evaluated and the enhanced reference spectral image respectively, and then performing robust sparse feature analysis on the high-dimensional feature matrix to highlight key features, and outputting a spectral feature matrix of the spectral image to be evaluated and a spectral feature matrix of the reference spectral image reconstructed by the sparse features; Step three, based on the two spectrum feature matrixes obtained in the step two, an improved convolutional neural network is established, and the improved convolutional neural network performs multi-layer feature mapping to obtain a low-dimensional feature matrix of the spectrum image to be evaluated and a low-dimensional feature matrix of the reference spectrum image after mapping; And step four, constructing a quality evaluation metric function based on the two low-dimensional feature matrixes obtained in the step three, and comprehensively evaluating the spectrum data quality of the spectrum image to be evaluated. Further, carrying out robust sparse feature analysis on the high-dimensional feature matrix to highlight key features, establishing an optimized model of the robust sparse feature analysis, and obtaining an optimal solution of the spectrum feature matrix reconstructed by the sparse features, wherein the optimized model of the robust sparse feature analysis is as follows: in the formula, The spectrum data matrix of the enhanced image to be evaluated comprises n samples and d spectrum features; for the spectrum characteristic matrix reconstructed by the sparse characteristic, reducing the high-dimensional spectrum characteristic from d dimension to k dimension; correcting the mean deviation of the spectrum data as a deviation vector; Is a parameter for controlling sensitivity to outliers; Is a unit matrix; Constraint conditions of the optimization model are as follows: And is also provided with ,=I constrained sparse projection matrixHas the advantages of the orthogonality, the good compatibility,And limiting the sparse projection matrix P to enter an optimization model by only h rows of features, wherein I is a low latitude identity matrix. Further, an iterative re-weighting algorithm is adopted to obtain an optimal solution of the sparse projection matrix P, specifically: Converting an optimized model of the robust sparse feature analysis, wherein the optimized model satisfies the following conditions in each iteration: in the formula, Representing the trace of the matrix; in the formula, Representing that the deviation vector b is copied and expanded on all sample dimensions to form a matrix consistent with the input data matrix dimension; d is a diagonal matrix, and the ith diagonal element of D is: 。 further, improving the multi-layer feature perception of convolutional neural networks is expressed as: in the formula, I