US-12626335-B2 - Image processing method, electronic device, and storage medium

US12626335B2US 12626335 B2US12626335 B2US 12626335B2US-12626335-B2

Abstract

Provided are an image processing method, an electronic device, and a storage medium. In the method, an original image is obtained by means of an image sensor. A gain is applied to the original image by means of a Pre-ISP to obtain a first image. The first image is denoised by the Pre-ISP, to obtain a second image. The original image and the second image are fused by the Pre-ISP, to obtain a fused image.

Inventors

Fei Geng

Assignees

GUANGDONG OPPO MOBILE TELECOMMUNICATIONS CORP., LTD.

Dates

Publication Date: 20260512
Application Date: 20230714
Priority Date: 20210118

Claims (17)

1 . An image processing method, for an electronic device, the electronic device comprising an image sensor and a Pre-Image Signal Processor (Pre-ISP), and the method comprising: acquiring, by the image sensor, an original image; applying, by the Pre-ISP, a gain to the original image, to obtain a first image; denoising, by the Pre-ISP, the first image to obtain a second image; and fusing, by the Pre-ISP, the original image with the second image, to obtain a fused image, comprising: for a first region of the second image where a brightness value is greater than a first brightness threshold and a corresponding first region of the original image, determining only the corresponding first region of the original image as a first to-be-fused image; for a second region of the second image where the brightness value is less than a second brightness threshold and a corresponding second region of the original image, determining only the second region of the second image as a second to-be-fused image, the first brightness threshold being greater than the second brightness threshold; for a third region of the second image where the brightness value is between the second brightness threshold and the first brightness threshold and a corresponding third region of the original image, determining both the third region of the second image and the corresponding third region of the original image as a third to-be-fused image; and stitching the first to-be-fused image, the second to-be-fused image, and the third to-be-fused image, thereby obtaining the fused image.
2 . The method of claim 1 , wherein the Pre-ISP comprises a gain module, a denoising module and a fusion module; and wherein applying, by the Pre-ISP, the gain to the original image to obtain a first image, comprises: applying, by the gain module, the gain to the original image, to obtain the first image; wherein denoising, by the Pre-ISP, the first image to obtain the second image, comprises: denoising, by the denoising module, the first image to obtain the second image; and wherein fusing, by the Pre-ISP, the original image and the second image to obtain the fused image, comprises: fusing, by the fusion module, the original image with the second image, to obtain the fused image.
3 . The method of claim 2 , wherein applying, by the gain module, the gain to the original image to obtain the first image, comprises: determining, by the gain module, a brightness parameter of the original image; and applying, by the gain module and based on the brightness parameter, the gain to the original image, to obtain the first image; and wherein denoising, by the denoising module, the first image to obtain the second image, comprises: denoising, by the denoising module and based on the brightness parameter, the first image to obtain the second image.
4 . The method of claim 3 , wherein applying, based on the brightness parameter, the gain to the original image to obtain the first image, comprises: determining, based on a value of the brightness parameter, a magnitude of the gain for the original image, wherein the smaller the value of the brightness parameter, the larger the magnitude of the gain for the original image; and the larger the value of the brightness parameter, the smaller the magnitude of the gain for the original image.
5 . The method of claim 4 , wherein determining, based on the value of the brightness parameter, the magnitude of the gain for the original image, comprises: calculating, based on the value of the brightness parameter, a contrast of the original image; and determining, based on the contrast of the original image, a magnitude of the gain for the original image, wherein the larger the contrast of the original image, the larger the magnitude of the gain for the original image, and the smaller the contrast of the original image, the smaller the magnitude of the gain for the original image.
6 . The method of claim 3 , wherein the original image has a plurality of brightness parameters, the plurality of brightness parameters defines a brightness value interval, the broader the brightness value interval, the smaller a magnitude of the gain applied to the original image; and the narrower the brightness value interval, the larger the magnitude of the gain applied to the original image.
7 . The method of claim 2 , wherein denoising, by the denoising module, the first image to obtain the second image, comprises: denoising, by the denoising module, the first image based on a magnitude of the gain for the original image, to obtain the second image, wherein the larger the magnitude of the gain for the original image, the higher a degree of denoising the first image, and the smaller the magnitude of the gain for the original image, the lower the degree of denoising the first image.
8 . The method of claim 1 , wherein for the third region of the second image where the brightness value is between the second brightness threshold and the first brightness threshold and the corresponding third region of the original image, determining both the third region of the second image and the corresponding third region of the original image as the third to-be-fused image, comprises: determining a weight parameter for the corresponding third region of the original image, and determining a weight parameter for the third region of the second image, wherein the closer the brightness value of the third region of the second image is to the first brightness threshold, the larger the weight parameter for the corresponding third region of the original image; the closer the brightness value of the third region of the second image is to the second brightness threshold, the larger the weight parameter for the third region of the second image; and combining the corresponding third region of the original image and the third region of the second image, based on corresponding weight parameters, to obtain an image as the third to-be-fused image.
9 . The method of claim 1 , wherein acquiring, by the image sensor, the original image comprises: taking, as the original image, an image acquired by the image sensor, in response to detecting that the electronic device fails to support a high dynamic range function.
10 . The method of claim 1 , wherein a size of the second image is the same as a size of the original image.
11 . An electronic device, comprising: an image sensor, configured to acquire an original image, wherein the original image is acquired by the image sensor without using a high dynamic range function; and a Pre-Image Signal Processor (Pre-ISP), configured to apply a gain to the original image, to obtain a first image; denoise the first image to obtain a second image; denoise the original image to obtain a third image; and fuse the third image with the second image, to obtain a fused image; wherein the Pre-ISP is specifically configured to: for a first region of the second image where a brightness value is greater than a first brightness threshold and a corresponding first region of the third image, determine only the corresponding first region of the third image as a first to-be- fused image; for a second region of the second image where the brightness value is less than a second brightness threshold and a corresponding second region of the third image, determine only the second region of the second image as a second to-be- fused image, the first brightness threshold being greater than the second brightness threshold; for a third region of the second image where the brightness value is between the second brightness threshold and the first brightness threshold and a corresponding third region of the third image, determine both the third region of the second image and the corresponding third region of the third image as a third to-be-fused image; and stitch the first to-be-fused image, the second to-be-fused image, and the third to-be-fused image, thereby obtaining the fused image.
12 . The electronic device of claim 11 , wherein the Pre-ISP is configured to: determine a brightness parameter of the original image; apply, based on the brightness parameter, the gain to the original image, to obtain the first image; and denoise, based on the brightness parameter, the first image to obtain the second image.
13 . The electronic device of claim 11 , wherein the Pre-ISP is configured to: determine a weight parameter for the corresponding third region of the third image, and determine a weight parameter for the third region of the second image, wherein the closer the brightness value of the third region of the second image is to the first brightness threshold, the larger the weight parameter for the corresponding third region of the third image; the closer the brightness value of the third region of the second image is to the second brightness threshold, the larger the weight parameter for the third region of the second image; and combine the corresponding third region of the third image and the third region of the second image, based on corresponding weight parameters, to obtain an image as the third to-be-fused image.
14 . The electronic device of claim 11 , wherein a degree of denoising applied to the original image is equal to or less than a degree of denoising applied to the first image.
15 . A non-transitory computer-readable storage medium, having program codes stored therein, wherein the program codes, when being executed by a processor, cause an image processing method to be implemented, the method comprising: acquiring an original image, wherein the original image is acquired without using a high dynamic range function; applying a gain to the original image, to obtain a first image; denoising the first image to obtain a second image; and fusing the original image with the second image, to obtain a fused image, comprising: for a first region of the second image where a brightness value is greater than a first brightness threshold and a corresponding first region of the original image, determining only the corresponding first region of the original image as a first to-be-fused image; for a second region of the second image where the brightness value is less than a second brightness threshold and a corresponding second region of the original image, determining only the second region of the second image as a second to-be-fused image, the first brightness threshold being greater than the second brightness threshold; for a third region of the second image where the brightness value is between the second brightness threshold and the first brightness threshold and a corresponding third region of the original image, determining both the third region of the second image and the corresponding third region of the original image as a third to-be-fused image; and stitching the first to-be-fused image, the second to-be-fused image, and the third to-be-fused image, thereby obtaining the fused image.
16 . The non-transitory computer-readable storage medium of claim 15 , wherein for the third region of the second image where the brightness value is between the second brightness threshold and the first brightness threshold and the corresponding third region of the original image, determining both the third region of the second image and the corresponding third region of the original image as the third to-be-fused image, comprises: determining a weight parameter for the corresponding third region of the original image, and determining a weight parameter for the third region of the second image, wherein the closer the brightness value of the third region of the second image is to the first brightness threshold, the larger the weight parameter for the corresponding third region of the original image; the closer the brightness value of the third region of the second image is to the second brightness threshold, the larger the weight parameter for the third region of the second image; and combining the corresponding third region of the original image and the third region of the second image, based on corresponding weight parameters, thereby obtaining the third to-be-fused image.
17 . The non-transitory computer-readable storage medium of claim 15 , wherein the applying a gain to the original image to obtain a first image, comprises: calculating, based on a value of a brightness parameter of the original image, a contrast of the original image; determining, based on the contrast of the original image, a magnitude of the gain for the original image; and applying the gain of the determined magnitude to the original image, to obtain the first image, wherein the larger the contrast of the original image, the larger the magnitude of the gain for the original image, and the smaller the contrast of the original image, the smaller the magnitude of the gain for the original image.

Description

CROSS-REFERENCE OF RELATED APPLICATIONS This application is a continuation of International Application No. PCT/CN2021/134021, filed Nov. 29, 2021, which claims priority to Chinese Patent Application No. 202110064269.6, filed Jan. 18, 2021, the entire disclosures of which are incorporated herein by reference. TECHNICAL FIELD The present disclosure relates to the field of image processing technologies, and more particularly, to an image processing method, an electronic device, and a storage medium. BACKGROUND With the continuous development of smart terminal technology, the usage of electronic devices (such as smart phones and cameras) is becoming more and more popular, where the majority of users record their studies, work and life through images captured by electronic devices; also, the users put forward higher requirements for the quality of the captured images. In order to improve the quality of the captured images, a camera with an HDR sensor function are usually used to generate multiple images through multiple exposures, and an HDR image is formed by fusing the multiple images obtained from multiple exposures. However, due to time intervals between the multiple exposures, a problem of ghosting in image would be easily caused when the multiple images formed after exposures are fused. SUMMARY In view of the above, the present disclosure proposes an image processing method, an electronic device, and a storage medium. In a first aspect, an embodiment of the present disclosure provides an image processing method, for an electronic device. The electronic device includes an image sensor and a Pre-ISP. The method includes: acquiring, by the image sensor, an original image; applying, by the Pre-ISP, a gain to the original image to obtain a first image, and denoising, by the Pre-ISP, the first image to obtain a second image; and fusing the original image with the second image, to obtain a fused image. In a second aspect, an embodiment of the disclosure provides an electronic device. The electronic device includes an image sensor and a Pre-Image Signal Processor (Pre-ISP). The image sensor is configured to acquire an original image, where the original image is acquired by the image sensor without using a high dynamic range function. And the Pre-ISP is configured to apply a gain to the original image, to obtain a first image; denoise the first image to obtain a second image; and fuse the original image with the second image, to obtain a fused image. In a third aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium. The computer-readable storage medium stores therein program codes which, when being executed, cause an image processing method to be implemented. The method includes: acquiring an original image, where the original image is acquired without a high dynamic range function; applying a gain to the original image, to obtain a first image; denoising the first image to obtain a second image; and fusing the original image with the second image, to obtain a fused image. Other features and aspects of the disclosed features will become apparent from the following detailed description, take in conjunction with the accompanying drawings, which illustrate, by way of example, the features in accordance with embodiments of the disclosure. The summary is not intended to limit the scope of any embodiments described herein. BRIEF DESCRIPTION OF DRAWINGS In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, drawings needed to be used in the description of the embodiments are briefly described below. Apparently, the drawings below are just some embodiments of the present disclosure, and other drawings can also be obtained by those skilled in the art based on these drawings without paying any creative effort. FIG. 1 illustrates a method flowchart of an image processing method provided by an embodiment of the present disclosure. FIG. 2 illustrates a method flowchart of an image processing method provided by another embodiment of the present disclosure. FIG. 3 illustrates an example diagram of determining to-be-fused images as provided by an embodiment of the present disclosure. FIG. 4 illustrates a schematic diagram of an image processing procedure provided by an embodiment of the present disclosure. FIG. 5 illustrates a method flowchart of an image processing method provided by another embodiment of the present disclosure. FIG. 6 illustrates a schematic diagram of another image processing procedure provided by an embodiment of the present disclosure. FIG. 7 illustrates a structural block diagram of an image processing apparatus provided by an embodiment of the present disclosure. FIG. 8 illustrates a schematic architecture diagram of an image processing system provided by an embodiment of the present disclosure. FIG. 9 illustrates a schematic architecture diagram of another image processing system provided by an embodimen