CN-122023152-A - High dynamic range video reconstruction method and system based on hybrid expert system

CN122023152ACN 122023152 ACN122023152 ACN 122023152ACN-122023152-A

Abstract

The invention discloses a high dynamic range video reconstruction method and system based on a hybrid expert system, which comprises the following steps of obtaining SDR video to be processed and decoding the SDR video into a continuous SDR image frame sequence, extracting degradation characteristics for representing insufficient image quality from the SDR image frame sequence frame by frame through a degradation coding network and coding the degradation characteristics, generating probability weight vectors corresponding to all SDR images in the SDR image frame sequence through a routing network based on degradation coding output by the degradation coding network, carrying out characteristic enhancement and weighted fusion of the enhanced characteristics from the SDR image frame to frame through a video frame enhancement network integrating a plurality of parallel expert sub-networks based on the probability weight vectors to form a preliminary HDR video frame sequence, carrying out highlight region restoration on the preliminary HDR video frame sequence frame by frame through a highlight region restoration network to obtain a target HDR video frame sequence, and encoding and packaging the target HDR video frame sequence to obtain a reconstructed HDR video file.

Inventors

ZHAO YANG
ZHANG WENYOU
LI XINJIE
JIA WEI
ZHANG ZHAO
LIU XIAOPING

Assignees

合肥工业大学

Dates

Publication Date: 20260512
Application Date: 20251226

Claims (10)

1. A high dynamic range video reconstruction method based on a hybrid expert system comprises the following steps: S1, acquiring SDR video to be processed and decoding the SDR video into a continuous SDR image frame sequence; S2, extracting degradation characteristics for representing insufficient image quality from SDR images in the SDR image frame sequence frame by frame through a degradation coding network and coding the degradation characteristics; S3, generating a probability weight vector corresponding to each SDR image in the SDR image frame sequence through a routing network based on the degradation coding output by the degradation coding network; S4, based on the probability weight vector, carrying out characteristic enhancement of different scenes and weighted fusion of the enhanced characteristics on SDR images in an SDR image frame sequence frame by frame through a video frame enhancement network integrating a plurality of parallel expert sub-networks, and forming a preliminary HDR video frame sequence; S5, performing highlight region restoration on the HDR video frames in the preliminary HDR video frame sequence frame by frame through a highlight region restoration network to obtain a target HDR video frame sequence; S6, encoding and packaging the target HDR video frame sequence to obtain a reconstructed HDR video file.
2. The hybrid expert system-based high dynamic range video reconstruction method according to claim 1, wherein the degenerate encoding network comprises: An adjustment layer configured to adjust SDR images in the SDR image frame sequence to a preset size; a convolution feature extraction network, formed by sequentially connecting at least one convolution block, configured to extract degradation features from the SDR image and output a two-dimensional feature map; the global average pooling layer is configured to carry out global average pooling on the two-dimensional feature map to obtain an aggregate global feature vector; and the feature code full-connection network is configured to map the aggregate global feature vector to a preset low-dimensional space and output the degradation code.
3. The hybrid expert system-based high dynamic range video reconstruction method according to claim 1, wherein the routing network comprises: the first processing unit comprises a first full-connection layer and a first activation function layer; The first full-connection layer is configured to perform linear transformation on the degradation code, the first activation function layer is a ReLU activation function and is configured to perform nonlinear activation on the characteristics after linear transformation and output intermediate characteristic vectors; the second processing unit comprises a second full-connection layer and a second activation function layer; The second full connection layer is configured to map the intermediate feature vectors into a plurality of initial weights equal to the number of expert sub-networks, and the second activation function layer is a Softmax activation function and is configured to normalize the plurality of initial weights and output a probabilistic weight vector.
4. The hybrid expert system-based high dynamic range video reconstruction method according to claim 1, wherein the video frame enhancement network employs a U-Net-based architecture, comprising: An initial feature extraction network configured to extract base features for an input SDR image frame; An encoder network configured to downsample and abstract the base features and output an encoded feature map; The mixed expert bottleneck network comprises a plurality of parallel expert sub-networks, is configured to carry out self-adaptive enhancement of different scene feature enhancement on the coding feature map by using the plurality of expert sub-networks, takes the probability weight vector corresponding to the SDR image as a gating parameter to carry out weighted fusion on the enhancement features output by the expert sub-networks, and outputs bottleneck layer features; A decoder network configured to upsample and reconstruct features of the bottleneck layer feature; an HDR reconstruction output network configured to map features of the decoder network output to the preliminary HDR video frames.
5. The hybrid expert system-based high dynamic range video reconstruction method according to claim 4, wherein the expert subnetwork comprises: an exposure estimation module configured to generate an exposure map with the same size as the encoded feature map and a numerical range within [0,1] ; A dual-branch residual processing module comprising two independent branches, each branch containing at least one residual block configured to extract a bright region feature from the encoded feature map, respectively And dark area features ; An exposure guidance fusion module configured to Computing and outputting enhanced features 。
6. The hybrid expert system-based high dynamic range video reconstruction method according to claim 1, wherein the highlight region restoration network comprises: A mask generation unit configured to generate a high-light mask by comparing a luminance channel of the preliminary HDR video frame with a preset luminance threshold; a region separation unit configured to separate a highlight region image from the preliminary HDR video frame using the highlight mask; The highlight generation network is configured to repair and enhance details of the highlight region image and output a repaired highlight image; and the image synthesis unit is configured to synthesize the restored highlight image with a non-highlight region in a preliminary HDR video frame and output a target HDR video frame.
7. The hybrid expert system-based high dynamic range video reconstruction method according to claim 6, wherein the highlight region restoration network further comprises a highlight discrimination network; the highlight judging network is configured to respectively process the highlight region of the input target HDR video frame and the highlight region of the real HDR image and output judging results; and calculating the countermeasures according to the discrimination results, and optimizing parameters of the high-light generation network by utilizing the countermeasures.
8. The hybrid expert system-based high dynamic range video reconstruction method as recited in claim 7, wherein, The high light generation network comprises: at least one sequentially concatenated convolution block, each said convolution block comprising a3 x 3 convolution layer and a ReLU activation function layer; an output layer comprising a 3 x 3 convolution layer and a Sigmoid activation function layer; the highlight discrimination network comprises: The feature extraction layer comprises a convolution layer with a convolution kernel size of 3 multiplied by 3 and a step length of 1; The depth characteristic judging module comprises at least one downsampling convolution block, wherein each downsampling convolution block comprises a convolution layer with a convolution kernel size of 3 multiplied by 3 and a step length of 2 and a LeakyReLU activation function layer; The discrimination output layer comprises a convolution layer, a full connection layer and a Sigmoid activation function layer.
9. A hybrid expert system-based high dynamic range video reconstruction system, comprising: the data preprocessing module is configured to acquire and decode SDR video to be processed to obtain a continuous SDR image frame sequence; the degradation coding module is configured to process the SDR image frame sequence frame by frame, extract degradation characteristics for representing insufficient image quality and code the degradation characteristics into degradation codes; a routing module configured to generate a probabilistic weight vector corresponding to each SDR image in a sequence of SDR image frames according to the degenerate encoding; the video frame enhancement module is configured to utilize the probability weight vector to perform feature enhancement of different scenes and weighted fusion of the enhanced features on the SDR images through a plurality of integrated special sub-networks so as to generate a preliminary HDR video frame sequence; The highlight restoration module is configured to restore the highlight areas of the HDR video frames in the preliminary HDR video frame sequence to generate a target HDR video frame sequence; A video output module configured to encode and encapsulate the target HDR video frame sequence into a reconstructed HDR video file.
10. An electronic device comprising a processor and a memory; the memory stores a computer program; The processor is configured to execute the computer program to implement the hybrid expert system based high dynamic range video reconstruction method as defined in any one of claims 1 to 8.

Description

High dynamic range video reconstruction method and system based on hybrid expert system Technical Field The invention relates to the field of image processing and computer vision, in particular to a high dynamic range video reconstruction method and system based on a hybrid expert system. Background High Dynamic Range (HDR) video reconstruction techniques aim at recovering HDR video content with a higher luminance range, richer color levels, and clearer details from Standard Dynamic Range (SDR) video, which is of great importance for improving the visual experience. Currently, most methods in this field are typically trained based on a single degradation model, with video conversion done by building a fixed mapping from SDR to HDR. However, the fundamental limitation of this type of approach is its inadequate generalization capability. As the video content in practical application becomes diversified, the illumination condition is complex and various, and a single degradation model is difficult to cover all scenes. This results in a significant reduction in reconstruction efficiency and unstable behavior of the model in the face of video with a different distribution than the training data. Conventional approaches such as those based on luminance mapping curves or local filtering also face challenges in terms of reconstruction quality. These methods often lack deep understanding and modeling of image semantic information, regional luminance distribution, and complex texture features. Therefore, the reconstructed HDR video is easy to have the problems of insufficient color restoration, loss or overexposure of the details of a highlight area, blurry details of a dark part and the like. In particular when dealing with complex scenes or non-standard exposure video, existing methods have difficulty in achieving a good balance between color fidelity, detail retention, and overall visual quality. In summary, the prior art fails to effectively solve the contradiction between the model adaptation capability and the high quality reconstruction requirement. Disclosure of Invention Aiming at the problems in the prior art, the invention provides a high dynamic range video reconstruction method and a system based on a hybrid expert system, which can realize the self-adaptive enhancement of input SDR video and reconstruct High Dynamic Range (HDR) video with high quality and rich details. The aim of the invention is achieved by the following technical scheme. The summary of the application is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. The summary of the application is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Some embodiments of the present application provide a high dynamic range video reconstruction method and system based on a hybrid expert system, which solve the technical problems mentioned in the background section above. As a first aspect of the present application, some embodiments of the present application provide a high dynamic range video reconstruction method based on a hybrid expert system, comprising the steps of: S1, acquiring SDR video to be processed and decoding the SDR video into a continuous SDR image frame sequence; S2, extracting degradation characteristics for representing insufficient image quality from SDR images in an SDR image frame sequence frame by frame through a degradation coding network and coding the degradation characteristics; S3, generating a probability weight vector corresponding to each SDR image in the SDR image frame sequence through a routing network based on the degradation coding output by the degradation coding network; S4, based on the probability weight vector, carrying out characteristic enhancement of different scenes and weighted fusion of the enhanced characteristics on SDR images in an SDR image frame sequence frame by frame through a video frame enhancement network integrating a plurality of parallel expert sub-networks, and forming a preliminary HDR video frame sequence; S5, performing highlight region restoration on the HDR video frames in the preliminary HDR video frame sequence frame by frame through a highlight region restoration network to obtain a target HDR video frame sequence; s6, encoding and packaging the target HDR video frame sequence to obtain a reconstructed HDR video file. Further, the degenerate coding network comprises: An adjustment layer configured to adjust SDR images in the SDR image frame sequence to a preset size; a convolution feature extraction network, formed by sequentially connecting at least one convolution block, configured to extract degradation features from the SDR image and output a two-dimensional feature map; The global average pooling layer is configured to carry out global average pooling on the two-dimensional feature