CN-115239777-B - Video processing method and device

CN115239777BCN 115239777 BCN115239777 BCN 115239777BCN-115239777-B

Abstract

The application discloses a video processing method and a device thereof, belonging to the technical field of video processing. The method comprises the steps of obtaining a target frame of a target video and depth information of the target frame, carrying out motion estimation on the target frame based on a historical frame relative to the target frame in the target video, determining a first motion vector of the target frame, wherein the resolution of the motion estimation is smaller than that of the target frame, updating the first motion vector based on the depth information of the target frame to obtain a second motion vector of the target frame, and registering the target frame with the historical frame based on the second motion vector.

Inventors

TANG CHAOYING
LI CHANGYANG

Assignees

维沃移动通信有限公司
维沃移动通信有限公司

Dates

Publication Date: 20260421
Application Date: 20220715
Priority Date: 20220715

Claims (8)

1. A video processing method, comprising: acquiring a target frame of a target video and depth information of the target frame; Performing motion estimation on the target frame based on a historical frame relative to the target frame in the target video, and determining a first motion vector of the target frame, wherein the resolution of the motion estimation is smaller than that of the target frame; updating the first motion vector based on the depth information of the target frame to obtain a second motion vector of the target frame; Registering the target frame with the history frame based on the second motion vector; the updating the first motion vector based on the depth information of the target frame to obtain a second motion vector of the target frame includes: determining a foreground vector and a background vector in the first motion vector; Determining a weighting coefficient according to the depth information of the target frame; And respectively carrying out weighting processing on the foreground vector and the background vector according to the weighting coefficient to obtain a second motion vector.
2. The video processing method according to claim 1, wherein the determining a weighting coefficient from the depth information of the target frame comprises: converting the depth information into a depth value in a target depth range according to the target depth range; and determining the weighting coefficient based on the depth value in the target depth range, wherein the value of the weighting coefficient is 0 to 1, and the larger the depth value is, the larger the value of the corresponding weighting coefficient is.
3. The method according to claim 1, wherein the weighting the foreground vector and the background vector according to the weighting coefficients, respectively, to obtain a second motion vector, comprises: determining a first average value of the foreground vector and a second average value of the background vector; weighting the foreground vector and the second average value according to the weighting coefficient to obtain a second motion vector corresponding to the foreground vector; And weighting the background vector and the first average value based on the weighting coefficient to obtain a second motion vector corresponding to the background vector.
4. The method of claim 1, wherein the registering the target frame with the history frame based on the second motion vector comprises: determining an offset vector for each pixel point in the target frame based on the second motion vector; and moving the pixel points based on the offset vector to obtain a registered target frame.
5. A video processing apparatus, comprising: The first acquisition module is used for acquiring a target frame of a target video and depth information of the target frame; A first determining module, configured to perform motion estimation on the target frame based on a history frame relative to the target frame in the target video, and determine a first motion vector of the target frame, where a resolution of the motion estimation is smaller than that of the target frame; A first updating module, configured to update the first motion vector based on depth information of the target frame, and obtain a second motion vector of the target frame; a first registration module for registering the target frame with the history frame based on the second motion vector; The first updating module includes: a first segmentation module, configured to determine a foreground vector and a background vector in the first motion vector; A second determining module, configured to determine a weighting coefficient according to the depth information of the target frame; And the first weighting module is used for respectively carrying out weighting processing on the foreground vector and the background vector according to the weighting coefficient to obtain a second motion vector.
6. The video processing apparatus of claim 5, wherein the second determining module comprises: the first conversion module is used for converting the depth information into a depth value in the target depth range according to the target depth range; And a third determining module, configured to determine the weighting coefficient based on a depth value in the target depth range, where the weighting coefficient takes a value from 0 to 1, and the larger the depth value is, the larger the corresponding weighting coefficient takes.
7. The video processing apparatus of claim 5, wherein the first weighting module comprises: a fourth determining module, configured to determine a first average value of the foreground vector and a second average value of the background vector; the second weighting module is used for carrying out weighting processing on the foreground vector and the second average value according to the weighting coefficient to obtain a second motion vector corresponding to the foreground vector; and the third weighting module is used for carrying out weighting processing on the background vector and the first average value based on the weighting coefficient to obtain a second motion vector corresponding to the background vector.
8. The video processing apparatus of claim 5, wherein the first registration module comprises: A fifth determining module, configured to determine an offset vector of each pixel point in the target frame based on the second motion vector; And the first moving module is used for moving the pixel points based on the offset vector to obtain a registered target frame.

Description

Video processing method and device Technical Field The application belongs to the technical field of video processing, and particularly relates to a video processing method and a device thereof. Background Since video requires at least 25 frames per second, the exposure time of each frame of image cannot exceed 40 milliseconds, and the exposure time is directly related to the image signal-to-noise ratio, image noise is a major problem affecting the video image quality. Video noise reduction typically uses frame-to-frame temporal information, requiring multiple frames of images to be processed together. During the shooting process of video, a photographer walks or hands shake to generate motion, so that spatial offset exists between frames, and therefore, before the video is subjected to noise reduction, spatial registration is required for each frame of image. The registration algorithm generally adopts an optical flow method and a block matching method, the methods generally divide an image into a plurality of image blocks, the finer the block is, the higher the resolution of registration is, the higher the requirement on equipment performance is, and the application range is limited. Disclosure of Invention The embodiment of the application aims to provide a video processing method and a device thereof, which can reduce the resolution of image registration and solve the problem that an algorithm has high requirements on equipment performance. In a first aspect, an embodiment of the present application provides a video processing method, including: acquiring a target frame of a target video and depth information of the target frame; Performing motion estimation on the target frame based on a historical frame relative to the target frame in the target video, and determining a first motion vector of the target frame, wherein the resolution of the motion estimation is smaller than that of the target frame; updating the first motion vector based on the depth information of the target frame to obtain a second motion vector of the target frame; registering the target frame with the history frame based on the second motion vector. In a second aspect, an embodiment of the present application provides a video processing apparatus, including: The first acquisition module is used for acquiring a target frame of a target video and depth information of the target frame; A first determining module, configured to perform motion estimation on the target frame based on a history frame relative to the target frame in the target video, and determine a first motion vector of the target frame, where a resolution of the motion estimation is smaller than that of the target frame; A first updating module, configured to update the first motion vector based on depth information of the target frame, and obtain a second motion vector of the target frame; a first registration module for registering the target frame with the history frame based on the second motion vector. In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, where the memory stores a program or instructions executable on the processor, the program or instructions implementing the video processing method according to the first aspect when executed by the processor. In a fourth aspect, embodiments of the present application provide a readable storage medium having stored thereon a program or instructions which, when executed by a processor, implement the video processing method according to the first aspect. In a fifth aspect, an embodiment of the present application provides a chip, where the chip includes a processor and a communication interface, where the communication interface is coupled to the processor, and the processor is configured to execute a program or instructions to implement the video processing method according to the first aspect. In a sixth aspect, embodiments of the present application provide a computer program product stored in a storage medium, the program product being executable by at least one processor to implement the video processing method according to the first aspect. In the embodiment of the application, fine blocking is not needed when the video is aligned, only the motion vector of the target frame is determined by motion estimation on low resolution, and then the registration is assisted by depth information, so that the calculated amount of a motion estimation algorithm can be reduced on the basis of ensuring the registration accuracy, the complexity of the algorithm is reduced, the requirement of image registration on the equipment performance is reduced, the application scene of the registration is increased, and the image quality of the video is improved. Drawings Fig. 1 is a flowchart of a video processing method according to an embodiment of the present application; Fig. 2 is a schematic diagram of an image in a video processing method according to an embo