CN-116309046-B - Underwater image cooperative enhancement and super-resolution method

CN116309046BCN 116309046 BCN116309046 BCN 116309046BCN-116309046-B

Abstract

The invention discloses an underwater image collaborative enhancement and super-resolution method, which belongs to the technical field of computer vision and comprises the steps of utilizing an underwater image dataset with a visual enhancement reference image to construct a training dataset required in a model training process, wherein the training dataset comprises a low-resolution original underwater degradation image, a low-resolution reference truth image, a high-resolution original degradation underwater image and a high-resolution reference truth image, constructing an underwater image collaborative enhancement and super-resolution deep learning network, inputting paired original underwater degradation images into the deep network, conducting constraint optimization on a predicted image output by a model through the corresponding low-resolution reference truth image, conducting constraint optimization on a model training process through minimized pixel value difference loss and structure similarity loss, training the underwater image collaborative enhancement and super-resolution deep learning network, taking any pair of original underwater degradation images as model input, and obtaining output through model calculation.

Inventors

LI KUNQIAN
QI QI
LI CHUNYAN
LIU WENJIE
SONG DALEI

Assignees

中国海洋大学

Dates

Publication Date: 20260505
Application Date: 20230216

Claims (2)

1. An underwater image cooperative enhancement and super-resolution method is characterized by comprising the following steps: s1, constructing a training data set required in a deep learning neural network model training process by utilizing an underwater image data set with a visual enhancement reference picture, wherein the training data set comprises an original underwater degradation image with low resolution, and Corresponding low resolution reference truth image High resolution raw degraded underwater image And (3) with Corresponding high resolution reference truth image ; S2, building an underwater image cooperative enhancement and super-resolution deep learning network, and pairing the underwater images As the deep network input, use is made of its correspondence Constraint optimization is carried out on the predicted image output by the deep learning neural network model; S3, performing constraint optimization on a deep learning neural network model training process by using minimized pixel value difference loss and structural similarity loss, and training the underwater image cooperative enhancement and super-resolution deep learning network to obtain a deep learning neural network model; S4, any pair of As the input of the model, the model is output after the deep learning neural network model ; S2 comprises the following steps: S2.1, constructing an underwater image cooperative enhancement and super-resolution deep learning network structure, wherein the network structure is composed of a cascaded underwater image cooperative enhancer module CoE-M and an underwater image cooperative super-resolution sub-module CoSR-M; S2.2, building CoE-M and CoSR-M based on a double-branch twin depth network structure, wherein the twin structure is connected by a feature matching association submodule to realize feature information transmission; CoE-M consists of a twin structure depth coding-decoding structure and a feature association matching module, wherein the twin structure depth coding-decoding structure is used for obtaining an initial enhancement result; CoSR-M consists of a twin residual error module, a feature association matching module and an up-sampling-convolution module, wherein the twin residual error module is used for acquiring depth feature expression facing a super-resolution task, the feature association matching module calculates double-branch feature association similarity by using cross correlation and performs feature channel combination according to nearest neighbor relation so as to realize feature information transmission and collaborative association on an image super-resolution level; S3 comprises the following steps: S3.1. Pair of Is marked as And As the input of the underwater image cooperative enhancement and super-resolution deep learning network; s3.2, outputting a pair of model predicted low-resolution clear underwater images by the underwater image cooperative enhancement and super-resolution deep learning network, and marking the low-resolution clear underwater images as And Respectively calculate And Between which, And Pixel value difference loss between; S3.3. Pair of Is marked as And As the input of the underwater image cooperative enhancement and super-resolution deep learning network; S3.4, outputting a pair of model predicted high-resolution clear underwater images by the underwater image cooperative enhancement and super-resolution deep learning network, and marking as And Respectively calculate And Between which, And (3) with Pixel value difference loss and structural similarity loss between the pixel values; S3.5, inputting the training data set obtained in the step S1 into an underwater image cooperative enhancement and super-resolution deep learning network in a form of pairing any two groups of samples according to the step S3.1 and the step S3.3, and respectively calculating the difference loss of the prediction output and the reference result of the two sub-modules according to the step S3.2 and the step S3.4; s3.6, iteratively optimizing model parameters by minimizing the difference loss.
2. The method for collaborative enhancement and super-resolution of an underwater image according to claim 1, wherein S1 comprises: S1.1, utilizing an image downsampling method to perform downsampling on Downsampling to obtain the corresponding one ; S1.2, utilizing an image downsampling method to perform downsampling on Downsampling to obtain the corresponding one 。

Description

Underwater image cooperative enhancement and super-resolution method Technical Field The invention discloses an underwater image cooperative enhancement and super-resolution method, and belongs to the technical field of computer vision. Background In recent years, autonomous underwater vehicles (Autonomous Underwater Vehicle, AUV) and remotely operated vehicles (Remote Operated Vehicle, ROV) have been rapidly developed as major underwater observation and operation platforms. In order to obtain underwater visual data, most underwater vehicles are equipped with high performance cameras. As one of the main sources of underwater information, vision can provide visual observation and guidance for underwater operations. Good visual quality and high resolution are increasingly becoming important requirements for image acquisition modules for underwater vehicles. However, underwater imaging conditions are dynamically changing and more complex than in an atmospheric imaging environment. Furthermore, degradation of the underwater image is often unavoidable due to wavelength dependent absorption, forward scattering, backward scattering, turbulence and interference with suspended matter. Accordingly, underwater image enhancement (Underwater IMAGE ENHANCEMENT, UIE) is aimed at improving the visual quality of Underwater images, and has recently received a lot of attention. The enhanced quality even determines success or failure of subsequent underwater vision tasks. Furthermore, although most underwater robots are equipped with high-performance cameras, data transmission of the acquired images is still a troublesome problem. For a cableless remote communication underwater robot, the high cost and narrow bandwidth of the cableless data transmission are limited, and in a remote underwater task, it is very difficult to transmit high-resolution images. Transmitting low Resolution images and then transmitting Super Resolution (SR) processed high Resolution images for subsequent tasks in the decoding stage is an effective tradeoff. The underwater image enhancement improves contrast and saturation at the same time, and the super-resolution technology recovers local details lost due to downsampling operation, so that the underwater image visual quality is improved together, and the underwater observation and operation experience of the underwater robot is better. Therefore, both underwater image enhancement and underwater image super-resolution are critical to underwater viewing and operation. Unfortunately, although underwater image enhancement and single image super-resolution have been widely studied separately, there are few approaches to discuss their coupling problem as a whole. These two tasks are consistent with the goal of improving the visual quality of the image, and the enhancement task can help super-resolution capture degradation information of the underwater image better. In addition, we also note that the existing underwater image enhancement algorithm is mostly designed for an isolated single image, is objectively hindered by the serious lack of underwater vision enhancement learning samples, and is difficult to learn an enhancement model with robust performance. Few researchers have solved the task of cooperative super resolution of non-overlapping images from related scenes. Even a few joint enhancement and super resolution algorithms whose algorithm inputs are isolated single images. Compared with the rich content diversity of land scenes, the underwater scenes have relatively limited content semantic types, mainly comprising water bodies, aquatic plants, ruins, reefs, fishes, seafloors, rocks and the like. Images taken in similar underwater scenes, i.e. scene-related images, typically contain a large amount of complementary information. Therefore, the collaborative processing strategy for the scene associated images is more beneficial to learning a unified and effective depth model with high visual uniformity and universality, and has very wide application prospect. Based on the method, the invention provides an underwater image cooperative enhancement and super-resolution method. Disclosure of Invention The invention discloses an underwater image collaborative enhancement and super-resolution method, which solves the problem that an integral joint calculation method is lacking in an image enhancement and super-resolution algorithm in the prior art. An underwater image cooperative enhancement and super-resolution method comprises the following steps: S1, constructing a training data set required in a deep learning neural network model training process by using an underwater image data set with a visual enhancement reference image, wherein the training data set comprises a low-resolution original underwater degradation image X l, a low-resolution reference true value image Y l corresponding to X l, a high-resolution original degradation underwater image X h and a high-resolution reference true value image Y h