CN-121998824-A - Scene self-adaption-based image super-processing method and system

CN121998824ACN 121998824 ACN121998824 ACN 121998824ACN-121998824-A

Abstract

The invention discloses an image super-processing method and system based on scene self-adaption, wherein the method comprises the following steps of extracting multichannel characteristics of a low-resolution image, determining scene types corresponding to the low-resolution image based on the multichannel characteristics, loading a super-processing model corresponding to the scene types, and performing super-processing on the low-resolution image corresponding to the scene types through the super-processing model to obtain the super-resolution image of the low-resolution image. According to the method, the special super-division model is deployed for different scene categories, so that the system can identify the scene categories in real time and adaptively switch to the super-division model of the corresponding scene, and the image is subjected to targeted super-division processing, so that the super-processing effect of the images of different scenes can be remarkably improved.

Inventors

HUANG CHENG
XIONG XIAOYI
CUI CHANGHAO

Assignees

武汉高德智感科技有限公司

Dates

Publication Date: 20260508
Application Date: 20251219

Claims (10)

1. The scene self-adaption-based image super-division processing method is characterized by comprising the following steps of: Extracting multichannel characteristics of the low-resolution image; determining a scene category corresponding to the low-resolution image based on the multi-channel features; And loading a super-processing model corresponding to the scene category, and performing super-processing on the low-resolution image corresponding to the scene category through the super-processing model to obtain a super-resolution image of the low-resolution image.
2. The image super processing method as claimed in claim 1, wherein the multi-channel features of the low resolution image are extracted based on a lightweight convolutional neural network.
3. The image super processing method as claimed in claim 1, wherein determining a scene category of the low resolution image based on the multi-channel feature comprises the steps of: performing dimension compression on the input multichannel characteristics through a 1X 1 convolution layer; inputting the multi-channel characteristics subjected to dimension compression into a global average pooling layer, and compressing the multi-channel characteristic space dimension into 1 multiplied by 1 through the global average pooling layer; Inputting the multichannel characteristics with the space dimension compressed into a first full-connection layer, and outputting characteristic vectors after activation; the feature vector is input into a second full-connection layer, and a prediction score corresponding to the total number of scene categories is output; and outputting probability distribution of each scene category, and determining the scene category according to the probability distribution.
4. The image super processing method as claimed in claim 1, wherein the scene category determination process is trained based on a scene classification loss function L cls , and the scene classification loss function L cls is as follows: ; Wherein N is the number of batch training samples, C is the total number of scene categories, w ci is the scene category weight coefficient, and w ci = total number of samples/number of samples of scene category C i , i is the ith sample in scene category C i , z i,j is the prediction score of the ith sample on the jth scene category, C i is the real scene category label of the ith sample, and z i,ci is the prediction score of the ith sample on its real scene category.
5. The image super processing method as claimed in claim 1, wherein a super processing model corresponding to at least one scene category is trained based on a super loss function L SR , and the super loss function L SR is as follows: L SR =α·L L1 +β·L SSIM; Wherein L L1 、L SSIM is L1 loss and structure similarity loss, alpha and beta are weighting coefficients of L1 loss L L1 and structure similarity loss L SSIM , and alpha+beta=1.
6. The image super processing method as claimed in claim 5, wherein, , Watch (watch) Showing the superscore image; And N is the total number of pixels of the super-resolution image output by the super-processing model.
7. The image super processing method as claimed in claim 5, wherein L SSIM =1-SSIM(I SR ,I HR ) wherein, SSIM is a structural similarity index that, Representing the superdivision image; representing a true high resolution image.
8. The image super processing method as claimed in claim 5, wherein the scene category classification result and the super processing result are optimized by a total loss function L total , and L total =α·L SR +β·L cls .
9. The image super-processing method according to claim 1, wherein if the current low resolution image is a non-first frame image, a super-processing model corresponding to the scene category is loaded, and the low resolution image corresponding to the scene category is super-processed by the super-processing model, comprising the steps of: Receiving scene categories of continuous K frames of low-resolution images including the current low-resolution image; If the scene types of the continuous K frames of low-resolution images are the same as the scene types corresponding to the currently loaded superdivision processing model, superdivision processing is directly carried out on at least one frame of low-resolution images in the continuous K frames of low-resolution images through the currently loaded superdivision processing model so as to obtain superdivision images; if at least one scene category in the scene categories of the continuous K-frame low-resolution images is different from the scene category corresponding to the currently loaded superdivision processing model, loading the superdivision processing model corresponding to the different scene categories, and performing superdivision processing on the low-resolution images of the different scene categories through the superdivision processing model to obtain the superdivision images.
10. An image superdivision processing system, comprising: a basic feature extraction module for extracting multi-channel features of the low resolution image; a scene classification module that determines a scene category of the low resolution image based on the multi-channel features; And the super processing module is used for loading a super processing model corresponding to the scene category and performing super processing on the low-resolution image corresponding to the scene category through the super processing model so as to obtain a super-resolution image of the low-resolution image.

Description

Scene self-adaption-based image super-processing method and system Technical Field The invention relates to the technical field of image processing, in particular to an image super-processing method and system based on scene self-adaption. Background In the field of image processing, due to limitation of image Resolution, tasks that are dependent on image completion, such as target recognition, cannot meet expected requirements, so that image Resolution (i.e., image Super-Resolution processing) needs to be improved by a Super-Resolution technology (SR), and image quality is improved. In the prior art, a single lightweight CNN model (such as MobileNet, shuffleNet variant) is mostly adopted to perform super-processing on the image, but the rising lightweight CNN model has limited parameters, so that various scene characteristics, such as fine textures of a forest, regular edges of urban buildings and the like, are difficult to optimize simultaneously, further the super-division effect is poor, such as excessive smooth artifact generated in a vegetation area, ringing effect easily occurs on the edges of the buildings, and details of low-contrast targets (such as human bodies in grasslands) are lost. Disclosure of Invention Aiming at the defects of the prior art, the invention provides a scene self-adaption-based image super-processing method and a scene self-adaption-based image super-processing system, which enable the system to identify scene types in real time and switch to super-division models of corresponding scenes in a self-adaption manner by deploying special super-division models for different scene types so as to perform targeted super-division processing on images, thereby remarkably improving the super-processing effect of images of different scenes. In order to achieve the above purpose, the present invention provides the following technical solutions: In one aspect, a scene-adaptive-based image super-processing method is provided, which includes the following steps: Extracting multichannel characteristics of the low-resolution image; determining a scene category corresponding to the low-resolution image based on the multi-channel features; And loading a super-processing model corresponding to the scene category, and performing super-processing on the low-resolution image corresponding to the scene category through the super-processing model to obtain a super-resolution image of the low-resolution image. Preferably, the multichannel features of the low resolution image are extracted based on a lightweight convolutional neural network. Preferably, determining a scene category of the low resolution image based on the multi-channel feature includes the steps of: performing dimension compression on the input multichannel characteristics through a 1X 1 convolution layer; inputting the multi-channel characteristics subjected to dimension compression into a global average pooling layer, and compressing the multi-channel characteristic space dimension into 1 multiplied by 1 through the global average pooling layer; Inputting the multichannel characteristics with the space dimension compressed into a first full-connection layer, and outputting characteristic vectors after activation; the feature vector is input into a second full-connection layer, and a prediction score corresponding to the total number of scene categories is output; and outputting probability distribution of each scene category, and determining the scene category according to the probability distribution. Preferably, the scene category determination process is trained based on a scene classification loss function L cls, and the scene classification loss function L cls is as follows: Wherein N is the number of batch training samples, C is the total number of scene categories, w ci is the scene category weight coefficient, and w ci = total number of samples/number of samples of scene category C i, i is the ith sample in scene category C i, z i,j is the prediction score of the ith sample on the jth scene category, C i is the real scene category label of the ith sample, and z i,ci is the prediction score of the ith sample on its real scene category. Preferably, the super-processing model corresponding to at least one scene category is trained based on a super-loss function L SR, and the super-loss function L SR is as follows: LSR=α·LL1+β·LSSIM Wherein L L1、LSSIM is L1 loss and structure similarity loss, alpha and beta are weighting coefficients of L1 loss L L1 and structure similarity loss L SSIM, and alpha+beta=1. Preferably, the method comprises the steps of,,Representing the superdivision image; And N is the total number of pixels of the super-resolution image output by the super-processing model. Preferably, L SSIM=1-SSIM(ISR,IHR), wherein SSIM is a structural similarity index,Representing the superdivision image; representing a true high resolution image. Preferably, the scene category classification result and the super processing resu