CN-121746538-B - Multi-layer lookup table image mapping method based on condition sensing and mixed expert mechanism

CN121746538BCN 121746538 BCN121746538 BCN 121746538BCN-121746538-B

Abstract

The invention belongs to the technical field of computer vision and image processing, and provides a multi-layer lookup table image mapping method based on a condition sensing and mixing expert mechanism, aiming at the technical problem that the mapping behavior cannot be adaptively adjusted according to the characteristics of different image areas, and the linear expansion and feature diversification of a receptive field are realized through a space progress fusion framework; and finally, carrying out deep compression on all expert lookup tables through a diagonal priority compression algorithm, and remarkably improving the space adaptability and the storage efficiency of image mapping while keeping extremely low reasoning time delay.

Inventors

CHEN TIANXIANG
HU JIKANG
ZHOU QIUZHAN

Assignees

吉林省日月智感互联科技有限公司

Dates

Publication Date: 20260512
Application Date: 20260226

Claims (7)

1. The multi-layer lookup table image mapping method based on the condition sensing and mixed expert mechanism is characterized by comprising the following steps: S1, calculating roughness information used for representing regional characteristics in a target image and generating a gating modulation coefficient, wherein the gating modulation coefficient is used for controlling expert branch deflection; S2, inputting a target image into a cascade mapping model, wherein each layer at least comprises a basic expert branch and a detail expert branch which are arranged in parallel; S3, performing storage optimization on the expert lookup table by adopting a diagonal preferential compression strategy for each layer of lookup table mapping module in the cascade mapping model; s4, based on the gating modulation coefficient in the S1, carrying out weighted fusion on the outputs of all branch experts to obtain corresponding output characteristics Y of each layer; S5, executing cross-layer feature transfer operation on the output feature Y obtained in the S4 through channel segmentation operation; s6, repeating the steps S3-S5 to the final layer, and carrying out feature fusion on all cross-layer retention features and the final layer output to obtain fusion features; s7, outputting a reconstructed final image according to the fusion characteristics; the S1 is specifically that a local maximum value of each pixel position is obtained through a local sliding window And local minimum Gating modulation factor , Wherein a and b are independently set and learnable affine parameters, and roughness Is a roughness threshold; gating modulation coefficients when the target image area is in a stationary background The model tends to call the underlying expert to eliminate quantization noise, gating the modulation coefficients when the target image region is at texture edges The model tends to call detail specialists to enhance high frequency details.
2. The multi-layer lookup table image mapping method based on the condition sensing and mixing expert mechanism as claimed in claim 1, wherein the cascade mapping model comprises 4 mapping blocks, wherein the number of output characteristic channels is 2 in the first three mapping blocks, the number of output characteristic channels is 1 in the fourth mapping block, the output characteristic channels are directly stored into a definition_list, the characteristics in the definition_list are subjected to spatial information fusion through a channel convolution layer, the channel convolution layer is realized by adopting a lookup table, and the fused characteristics are restored to target resolution through an up-sampling block.
3. The multi-layer lookup table image mapping method based on the conditional sensing and blending expert mechanism as claimed in claim 1, wherein said S3 is specifically based on calculating absolute values of differences of pairs of input neighboring pixels , ; If it is Judging that the current pixel pair belongs to a diagonal region with high probability distribution, and acquiring a one-dimensional index by using a preset index mapping matrix To reduce the dimensionality, the output is obtained by four-wire interpolation in a compressed look-up table, , Indexing the compressed one-dimensional memory physical address; If it is Judging the region belonging to the non-diagonal line region, classifying the region into a non-diagonal line table, and adopting a large step size And acquiring output from the sparse table subjected to downsampling.
4. The multi-layer lookup table image mapping method based on the conditional sensing and mixing expert mechanism as claimed in claim 1, wherein each layer corresponds to an output feature , wherein, In order to gate the modulation factor(s), The output value is branched off for the basic expert, The output value branches for the detail expert.
5. The multi-layer lookup table image mapping method based on the condition sensing and mixing expert mechanism according to claim 1, wherein the step S5 is specifically that each layer of output features Y obtained in the step S4 is processed through channel segmentation, a part of features X is used as index input features of a next layer of lookup table mapping module, and a part of features S is stored in a refine_list to be used as cross-layer feature information reserved in the present layer.
6. The multi-layer lookup table image mapping method based on a conditional sensing and hybrid expert mechanism as claimed in claim 1, wherein a multi-mode sampling mechanism is adopted inside a private branch in the cascade mapping model: Mode s, adopt 2 2 Standard neighborhood sampling, sampling coordinate offset is Capturing a base texture; Mode d, adopting expansion neighborhood sampling with interval of 2, sampling coordinate offset being For expanding receptive fields; the mode Y adopts asymmetric Y-type sampling, and the sampling coordinate offset is as follows For capturing edge features with directionality.
7. The multi-layer lookup table image mapping method based on the conditional access and hybrid expert system as claimed in any one of claims 1-6, wherein a comprehensive loss function is used Training a cascade mapping model: , wherein the main reconstruction is lost For constraining fundamental pixel differences between final image and ground truth, flat region residual regularization penalty The gating state is utilized to carry out inverse constraint on the detail expert output, when the gating module judges that the area is flat, the loss term will strongly restrain the response of the detail expert, the gating regularization loss By introducing roughness threshold From the target offset coefficient, pilot-gated modulation coefficients Is a distribution of (a).

Description

Multi-layer lookup table image mapping method based on condition sensing and mixed expert mechanism Technical Field The invention relates to the technical field of computer vision and image processing, in particular to a multi-layer lookup table image mapping method based on a condition sensing and mixed expert mechanism, which is particularly suitable for tasks such as image super-resolution reconstruction, denoising, deblocking effect, image enhancement and the like. Background The image mapping method based on the lookup table is widely applied to related tasks of image processing and image reconstruction due to low calculation complexity and high reasoning efficiency. The method realizes nonlinear image transformation or feature reconstruction by carrying out lookup table mapping on input pixels or local features. On the one hand, the existing image mapping method based on the lookup table mostly adopts a unified mapping function to process different space regions. However, in the actual image, there is a significant difference between the flat region and the edge or texture region in terms of gradient distribution, spectral characteristics, noise sensitivity, and the like, and the same mapping manner is adopted to easily cause over-fitting of the flat region or insufficient expression of details in the complex texture region. This is particularly true in the case of look-up tables where the dimensions are limited and the effective receptive field is small. In addition, the prior lookup table mapping method is limited by the input dimension of the lookup table, and generally only relies on local pixel values for modeling, and lacks explicit perceptibility of conditional information such as regional structure complexity, so that the mapping function can only take uniform or approximately uniform mapping behaviors when facing different regional characteristics, and self-adaptive adjustment is difficult to realize. On the other hand, in the deep neural network, the convolution layer and the full connection layer perform linear or affine mapping on the input features, the activation function performs nonlinear modulation on the mapping result, and a selection or suppression mechanism for the feature response is implied. In the existing look-up table based modeling mode, the mapping and activating modulation behaviors are usually combined or approximated as a unified look-up table mapping process, and explicit modeling of a mapping modulation and selection mechanism is lacked, so that flexibility and expression capability of the look-up table mapping under different areas are limited. Therefore, how to improve the adaptive mapping capability of the lookup table to different image region characteristics while maintaining the efficient reasoning characteristics becomes a technical problem to be solved in the art. Disclosure of Invention The invention provides a multi-layer lookup table image mapping method based on a conditional sensing and mixing expert mechanism, which aims to solve the technical problems that the storage cost of a cascade lookup table is huge and the mapping behavior can not be adaptively adjusted according to different image region characteristics in the prior art. The core technical scheme of the invention comprises the following steps: S1, calculating roughness information used for representing regional characteristics in a target image and generating a gating modulation coefficient, wherein the gating modulation coefficient is used for controlling expert branch deflection; S2, inputting a target image into a cascade mapping model, wherein each layer at least comprises a basic expert branch and a detail expert branch which are arranged in parallel; S3, performing storage optimization on the expert lookup table by adopting a diagonal preferential compression strategy for each layer of lookup table mapping module in the cascade mapping model; s4, based on the gating modulation coefficient in the S1, carrying out weighted fusion on the outputs of all branch experts to obtain corresponding output characteristics of each layer; S5, executing cross-layer feature transfer operation on the output features obtained in the S4 through channel segmentation operation; s6, repeating the steps S3-S5 to the final layer, and carrying out feature fusion on all cross-layer retention features and the final layer output to obtain fusion features; And S7, outputting the reconstructed final image according to the fusion characteristics. The technical effects are as follows: The invention provides a multi-layer lookup table image processing scheme which is self-adaptively adjusted according to regional characteristics and has high-efficiency storage characteristics, and integrates an integral framework of 'space progress fusion', 'hybrid special control' and 'diagonal preferential compression'. Firstly, realizing linear expansion and feature diversification of a receptive field through a 'space progress fusion'