CN-121486579-B - Image compression domain self-adaption method and system based on space-channel perception

CN121486579BCN 121486579 BCN121486579 BCN 121486579BCN-121486579-B

Abstract

The invention discloses a space-channel perception-based image compression domain self-adaption method and a system, wherein the method comprises the steps of constructing a double-branch space-channel adapter for a transformation network to independently capture space layout and channel correlation based on a deep learning-based leavable image compression model; and integrating the dual-branch space-channel adapter and the space-channel interaction low-rank self-adaptive module into a deep learning leavable image compression model, and performing fine tuning by using a small amount of samples to realize domain self-adaptation to a specific domain. The invention is used for the learner image compression and realizes the high-efficiency domain self-adaption for specific domains.

Inventors

LIANG YONGSHENG
Lu Xuanrui
TAN WEN
MENG FANYANG
WANG GENHONG

Assignees

哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院)

Dates

Publication Date: 20260512
Application Date: 20260112

Claims (8)

1.A spatial-channel perception based image compression domain adaptation method, the method comprising: deep learning-based learner image compression model: Constructing a dual-branch space-channel adapter for a transformation network to independently capture space layout and channel correlation; constructing a space-channel interaction low-rank adaptive module for an entropy model to jointly model space and channel contexts; Integrating the dual-branch space-channel adapter and the space-channel interaction low-rank adaptive module into a deep learning leachable image compression model, and performing fine adjustment by using a small amount of samples to realize domain adaptation to a specific domain; the dual-branch space-channel adapter specifically includes: For input features Outputting self-adaptive characteristics of the aggregated space operation and the channel operation, wherein the space operation is the depth separable convolution, and the channel operation comprises the steps of performing dimension reduction through one 1x1 convolution, enabling the output to activate a function through LeakyReLU, and restoring the dimension to be consistent with the input dimension of the channel branch through one 1x1 convolution; initializing a dual-branch space-channel adapter using an identity map, including initializing a space branch to a pass-through output and a channel branch output to zero; integrating the dual-branch space-channel adapter and the space-channel interaction low-rank adaptive module into a deep learning leachable image compression model specifically comprises: Serial insertion of dual-branch space-channel adapters into analysis changes in a deep-learned, learnable image compression model And synthetic changes After the nonlinear transformation in (a), adding a space-channel interaction low-rank adaptive module to a super-synthetic transformation network of an entropy model Is included in the convolution layer of (a).
2. The image compression domain adaptive method based on space-channel perception according to claim 1, wherein the specific expression of the space-channel interaction low-rank adaptive module is: , The input characteristics are represented as such, For the original pre-training weights, For the updated convolution weights, 、 For projection along the channel dimension, For spatial operation in low rank dimension independent application for achieving fine-grained local filtering and enriching spatial information, wherein With the use of a gaussian initialization, Is set to be zero and is set to be zero, Uniformly initializing by using an Xavier.
3. The image compression domain adaptive method based on space-channel perception according to claim 1, wherein fine tuning is performed with a small amount of samples, and a two-stage fine tuning training strategy is adopted to fine tune only the total parameters of the dual-branch space-channel adapter and the space-channel interaction low-rank adaptive module , wherein, The first stage fine tuning concrete expression is: wherein Representing a rate function Distortion function The trade-off coefficient between these is that, A potential representation of the representation quantization, The quantized side information is represented by a picture of the picture, The input samples are represented as such, Representing an output; The second stage fine-tuning freezes the parameters of each encoder-side component and entropy model in the learnable image compression model and changes the composition Divided into four stacks, only the last stack at the decoder side is trimmed using distortion loss.
4. The spatially-channel aware based image compression domain adaptation method according to claim 1, wherein the depth learning based learner image compression model is selected from any one of Cheng2020, ELIC, TCM and MLICpp.
5. The adaptive method of image compression domain based on space-channel perception according to claim 3, wherein only the adapter parameters are stored in the fine tuning process, and the analysis change of the image compression model can be learned after one fine tuning is completed The middle dual-branch space-channel adapter parameters are stored in the encoder and the remaining adapter parameters are transmitted to the decoder.
6. An image compression domain adaptive system based on space-channel perception, the system comprising: DBSCA a construction module for constructing a dual-branch space-channel adapter for transforming a network based on a deep-learned image compression model to independently capture spatial layout and channel correlation; the SCI-LoRA building module is used for building a space-channel interaction low-rank self-adaptive module for the entropy model based on the deep learning image compression model so as to jointly model space and channel context; The module integration and fine adjustment module is used for integrating the dual-branch space-channel adapter and the space-channel interaction low-rank self-adaptation module into a deep learning leachable image compression model, and performing fine adjustment by using a small amount of samples, so as to realize domain self-adaptation to a specific domain; the dual-branch space-channel adapter specifically includes: For input features Outputting self-adaptive characteristics of the aggregated space operation and the channel operation, wherein the space operation is the depth separable convolution, and the channel operation comprises the steps of performing dimension reduction through one 1x1 convolution, enabling the output to activate a function through LeakyReLU, and restoring the dimension to be consistent with the input dimension of the channel branch through one 1x1 convolution; initializing a dual-branch space-channel adapter using an identity map, including initializing a space branch to a pass-through output and a channel branch output to zero; integrating the dual-branch space-channel adapter and the space-channel interaction low-rank adaptive module into a deep learning leachable image compression model specifically comprises: Serial insertion of dual-branch space-channel adapters into analysis changes in a deep-learned, learnable image compression model And synthetic changes After the nonlinear transformation in (a), adding a space-channel interaction low-rank adaptive module to a super-synthetic transformation network of an entropy model Is included in the convolution layer of (a).
7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the spatial-channel aware image compression domain adaptation method steps of any one of claims 1 to 5 when the program is executed.
8. A non-transitory computer readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the spatial-channel aware image compression domain adaptation method steps of any one of claims 1 to 5.

Description

Image compression domain self-adaption method and system based on space-channel perception Technical Field The invention relates to the technical field of image processing, in particular to an image compression domain self-adaption method and system based on space-channel perception. Background Image compression is a fundamental technology in the field of computer vision. Traditional compression methods such as JPEG, JPEG2000, BPG, and the latest VVC have accumulated a mature technological system in long-term applications, but their fixed transformation and prediction mechanisms have increasingly revealed limitations in the face of complex and diverse image scenes. With the development of deep learning in recent years, many of the Learning Image Compression (LIC) methods based on deep learning achieve better performance than these conventional image compression methods. However, the performance of the existing LIC model depends heavily on training data, and cannot flexibly adapt to new scenes and expand their functions, resulting in poor generalization in practical applications. Specifically, LIC models are typically trained on natural image datasets, but practical applications typically involve data such as cartoons, screen content, and pixel images, which have large field gaps that can lead to significant degradation in compression performance of off-domain data, sometimes even behind conventional codecs. This limitation underscores the need to develop efficient domain adaptation techniques to improve the generalization of pre-trained LIC models in different image content domains. Specifically, LIC models are typically trained on natural image datasets, but practical applications typically involve data such as cartoons, screen content, and pixel images, which have large field gaps that can lead to significant degradation in compression performance of off-domain data, sometimes even behind conventional codecs. This limitation underscores the need to develop efficient domain adaptation techniques to improve the generalization of pre-trained LIC models in different image content domains. The IA method can improve the compression effect of a single image, but the calculation and time cost of the IA method make the IA method difficult to deploy in practical application, and in practical application, the DA method is a more ideal solution. However, the conventional DA method requires updating and storing a large number of parameters. Whereas the most advanced small sample DA method only considers the problem from the channel dimension, in fact spatial information is also crucial in domain adaptation. Because distribution shifts typically involve variations in cross-domain layout, texture, and local patterns, lack of spatial modeling can lead to under-representation and poor compression performance. In order to solve the above problems, we have designed a domain adaptive scheme based on spatial-channel aware fine tuning to further improve the generalization of LIC. Disclosure of Invention In view of the above problems, the present invention provides a spatial-channel perception based image compression domain adaptive method and system, which aim to improve generalization of a depth learning based image compression (LIC) method. According to a first aspect of embodiments of the present disclosure, there is provided a spatial-channel aware based image compression domain adaptation method, the method comprising the steps of: deep learning-based learner image compression model: Constructing a dual-branch space-channel adapter for a transformation network to independently capture space layout and channel correlation; constructing a space-channel interaction low-rank adaptive module for an entropy model to jointly model space and channel contexts; And integrating the dual-branch space-channel adapter and the space-channel interaction low-rank adaptive module into a deep learning leachable image compression model, and performing fine adjustment by using a small amount of samples to realize domain adaptation of a specific domain. In some embodiments, the dual branch space-channel adapter specifically includes: For input features Outputting self-adaptive characteristics of the aggregated space operation and the channel operation, wherein the space operation is the depth separable convolution, and the channel operation comprises the steps of performing dimension reduction through one 1x1 convolution, enabling the output to activate a function through LeakyReLU, and restoring the dimension to be consistent with the input dimension of the channel branch through one 1x1 convolution; The dual-branch space-channel adapter is initialized using an identity map, including the space branch initializing to a pass-through output and the channel branch output initializing to zero. In some embodiments, the spatial-channel interaction low-rank adaptation module is specifically expressed as: , The input characteristics are represented a