CN-122023179-A - Method executed by electronic device, storage medium, and program product

Abstract

The embodiment of the disclosure provides a method executed by electronic equipment, the electronic equipment, a storage medium and a program product, and relates to the fields of video processing, artificial intelligence and the like. The method comprises the steps of determining a first motion track of pixels between at least two first images of a video, determining motion track points corresponding to at least one second image of the video in the first motion track, obtaining a second motion track based on the motion track points, wherein the ambiguity of the second image is greater than that of the first image, and performing deblurring processing on the at least one second image based on the second motion track. Alternatively, the above-described methods performed by the electronic device may be performed using an artificial intelligence model.

Inventors

• WANG FAN
• ZHANG YING
• LIU ZIKUN
• LI JIA
• ZHANG JIANXING
• PU XIANXI

Assignees

• 北京三星通信技术研究有限公司
• 三星电子株式会社

Dates

Publication Date

20260512

Application Date

20241112

Claims (20)

1. 1. A method performed by an electronic device, comprising: determining a first motion trail of pixels between at least two first images of the video; Determining a motion track point corresponding to at least one second image of the video based on the first motion track, and obtaining a second motion track based on the motion track point, wherein the ambiguity of the second image is greater than that of the first image; and performing deblurring processing on the at least one second image based on the second motion trail.
2. 2. The method as recited in claim 1, further comprising: And in the video acquisition process, if detecting that the third image ambiguity acquired based on the first exposure parameter is greater than a threshold value, acquiring at least one frame of image after the third image based on the second exposure parameter, and taking the image acquired based on the second exposure parameter as the first image, wherein the exposure time corresponding to the second exposure parameter is smaller than the exposure time corresponding to the first exposure parameter.
3. 3. The method of claim 2, wherein before the acquiring at least one frame of image after the third image based on the second exposure parameter, comprising: the second exposure parameter is determined based on the third image and the first exposure parameter.
4. 4. A method according to claim 3, wherein said determining said second exposure parameter based on said third image and said first exposure parameter comprises: determining a lower bound of exposure time based on the brightness of the third image and the first exposure parameter; determining an upper bound of exposure time based on the first exposure parameter; Determining an adjustment factor based on the degree of blur of the third image, the image characteristics of the third image, the upper bound and the lower bound; The second exposure parameter is determined based on the first exposure parameter, the upper bound, the lower bound, and the adjustment coefficient.
5. 5. The method of any of claims 1-4, wherein determining a first motion profile for a pixel between at least two first images of the video comprises: Determining the offset of the at least two first images from a predetermined first image respectively; And determining the first motion trail based on the offset.
6. 6. The method according to any one of claims 1-5, wherein determining a motion trajectory point corresponding to at least one second image of the video based on the first motion trajectory, and obtaining a second motion trajectory based on the motion trajectory point, comprises: Inserting a motion trail point corresponding to the at least one second image into the first motion trail; and adjusting the position of the inserted motion track point in the first motion track based on the ambiguity and the ambiguity direction of the at least one second image, and obtaining the second motion track based on the motion track point after the position adjustment.
7. 7. The method according to any one of claims 1-6, wherein said deblurring the at least one second image based on the second motion profile comprises: Determining displacement information and scale change information between images of the video based on the second motion trail; and performing deblurring processing on the at least one second image based on the displacement information and the scale change information.
8. 8. The method of claim 7, wherein the deblurring the at least one second image based on the displacement information and the scale change information comprises: the at least one second image is subjected to noise adding to obtain at least one fourth image; And performing at least one first denoising process on the at least one fourth image based on the at least two first images, the displacement information and the scale change information to obtain a deblurring process result of the at least one second image.
9. 9. The method of claim 8, wherein performing a first denoising process on the at least one fourth image based on the at least two first images, the displacement information, and the scale change information comprises: Based on the phase characteristics of the at least two first images, adjusting the phase characteristics of the at least one fourth image to obtain updated phase characteristics; Obtaining first guiding information based on the updated phase characteristics and the amplitude characteristics of the at least one fourth image; and performing first denoising processing on the at least one fourth image based on the first guide information, the displacement information and the scale change information.
10. 10. The method of claim 8, wherein performing a first denoising process on the at least one fourth image based on the at least two first images, the displacement information, and the scale change information comprises: Clustering pixels in the at least one fourth image based on the displacement information and the scale change information to obtain a clustering result; Processing the at least one fourth image using at least one attention network in cascade based on the at least two first images and the clustering result.
11. 11. The method of claim 10, wherein the at least one attention network of the cascade comprises at least one self-attention network, and wherein using the self-attention network to process the at least one fourth image comprises: Performing first attention calculation on pixels belonging to the same type in at least one fifth image by using a self-attention network based on the clustering result to obtain a first attention result, wherein the at least one fifth image is obtained based on the at least one fourth image or the at least one fifth image is obtained based on the at least one fourth image and the at least two first images; Processing the at least one fourth image based on the first attention result.
12. 12. The method of claim 10, wherein the at least one attention network of the cascade comprises at least one inter-frame attention network, and wherein processing the at least one fourth image using the inter-frame attention network comprises: Scaling the at least two first images based on the scale change information and the clustering result, and fusing the at least two scaled first images to obtain a sixth image; Performing second attention calculation on the sixth image and the at least one fourth image by using an inter-frame attention network to obtain a second attention result; Processing the at least one fourth image based on the second attention result.
13. 13. The method of claim 10, wherein the video comprises at least one segment, each segment comprising at least two first images and at least one second image, the cascaded at least one attention network comprising at least one inter-segment attention network, the at least one fourth image being processed using the inter-segment attention network, comprising: Performing third attention calculation on at least one first segment subjected to deblurring processing and a second segment to be subjected to deblurring processing by using an inter-segment attention network to obtain a third attention result; And processing at least one fourth image corresponding to the second segment based on the third attention result.
14. 14. The method according to any one of claims 1-13, further comprising: and performing brightness adjustment processing on the at least two first images based on the at least one second image.
15. 15. The method of claim 14, wherein the performing brightness adjustment processing on the at least two first images based on the at least one second image comprises: adjusting the amplitude characteristics of the at least two seventh images based on the amplitude characteristics of the at least one second image to obtain updated amplitude characteristics; obtaining second guide information based on the updated amplitude characteristics and the phase characteristics of the at least two seventh images; And carrying out brightness adjustment processing on the at least two first images based on the second guide information.
16. 16. A method performed by an electronic device, comprising: In the video acquisition process, if detecting that the second image ambiguity acquired based on the first exposure parameter is greater than a threshold value, acquiring at least one frame of first image after the second image based on the second exposure parameter, wherein the exposure time corresponding to the second exposure parameter is smaller than the exposure time corresponding to the first exposure parameter; and performing deblurring processing on the second image based on the first image.
17. 17. A method performed by an electronic device, comprising: noise is added to at least one second image of the video to obtain at least one fourth image; Performing at least one first denoising treatment on the at least one fourth image based on the at least two first images to obtain a deblurring treatment result of the at least one second image; Performing brightness adjustment processing on the at least two first images based on the at least one second image; wherein the second image has a greater degree of blur than the first image.
18. 18. An electronic device comprising a memory, a processor and a computer program stored on the memory, characterized in that the processor executes the computer program to implement the method of any one of claims 1-17.
19. 19. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the method of any of claims 1-17.
20. 20. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the method of any of claims 1-17.

Description

Method executed by electronic device, storage medium, and program product Technical Field The present disclosure relates to the field of video processing technology, and in particular, to a method performed by an electronic device, a storage medium, and a program product. Background With the improvement of hardware such as a sensor and a lens, the video quality of the camera of the electronic device is better and better, and meanwhile, the requirement of a user on the video quality of the electronic device is higher and higher. However, the shake of the electronic equipment held by the photographer, the movement of the photographed object and the like can cause motion blur of the recorded video, and seriously affect the user experience. Currently, most prior art solutions to video motion blur are almost hardware improvements. For example, with anti-shake sensors, smooth, stable video is captured by compensating for significant shock and motion. However, this approach relies on expensive hardware costs and is still less effective for shooting of sports scenes. Disclosure of Invention The aim of the embodiment of the disclosure is to solve the problem of how to improve the deblurring effect of video aiming at a motion scene. According to one aspect of an embodiment of the present disclosure, there is provided a method performed by an electronic device, the method comprising: determining a first motion trail of pixels between at least two first images of the video; Determining a motion trail point corresponding to at least one second image of the video based on the first motion trail, and obtaining a second motion trail based on the motion trail point, wherein the ambiguity of the second image is greater than that of the first image; and performing deblurring processing on at least one second image based on the second motion trail. Optionally, in the video acquisition process, if detecting that the ambiguity of the third image acquired based on the first exposure parameter is greater than the threshold value, acquiring at least one frame of image after the third image based on the second exposure parameter, and taking the image acquired based on the second exposure parameter as the first image, wherein the exposure time corresponding to the second exposure parameter is smaller than the exposure time corresponding to the first exposure parameter. Optionally, before acquiring at least one frame of image after the third image based on the second exposure parameter, the method includes: A second exposure parameter is determined based on the third image and the first exposure parameter. Optionally, determining the second exposure parameter based on the third image and the first exposure parameter includes: Determining a lower bound of exposure time based on the brightness of the third image and the first exposure parameter; Determining an upper bound of the exposure time based on the first exposure parameter; determining an adjustment coefficient based on the degree of blur of the third image, the image characteristics of the third image, the upper bound and the lower bound; a second exposure parameter is determined based on the first exposure parameter, the upper bound, the lower bound, and the adjustment coefficient. Optionally, determining a first motion trajectory of a pixel between at least two first images of the video includes: Determining the offset of at least two first images from a preset first image respectively; Based on the offset, a first motion profile is determined. Optionally, determining the offset of the at least two first images from the predetermined first image respectively includes: And determining the offset of at least two first images and a preset first image respectively based on the IMU parameters of the inertial measurement unit of the acquisition equipment during video acquisition. Optionally, determining a motion trail point corresponding to at least one second image of the video based on the first motion trail, and obtaining a second motion trail based on the motion trail point includes: inserting at least one motion trail point corresponding to the second image into the first motion trail; And adjusting the position of the inserted motion track point in the first motion track based on the ambiguity and the ambiguity direction of at least one second image, and obtaining a second motion track based on the motion track point after the position adjustment. Optionally, the direction of adjustment of the motion profile point corresponds to the direction of blurring, and/or, The adjustment distance of the motion trail point is proportional to the ambiguity. Optionally, based on the second motion trajectory, deblurring at least one second image, including: Determining displacement information and scale change information between images of the video based on the second motion trail; and performing deblurring processing on at least one second image based on the displacement information and t