CN-122027898-A - Video processing method, image pickup apparatus, and computer-readable storage medium

Abstract

The application provides a video processing method, image pickup equipment and a computer readable storage medium, wherein the method comprises the steps of a, sequentially identifying each frame image of a video to be processed to determine a reference frame, b, detecting the position of an observation target in the reference frame to determine the reference key position of the reference frame, c, taking a frame image after the reference frame as a current frame, d, detecting the position of the observation target in the current frame to determine the current key position of the current frame, e, determining the difference between the current key position and the reference key position to determine the offset of the observation target in the current frame, f, calibrating the current frame based on the offset, reading the next frame of the current frame from the video to be processed under the condition that the current frame is not the last frame of the video to be processed, determining the next frame of the current frame as a new current frame, and realizing the calibration of each frame image by the steps d to f.

Inventors

• MA SASA
• MENG DI
• WEN JIA

Assignees

• 苏州振旺光电有限公司

Dates

Publication Date

20260512

Application Date

20260319

Claims (19)

1. 1. A video processing method, comprising: Step a, sequentially identifying each frame of image of a video to be processed to determine a reference frame, wherein the reference frame is an image which is identified to be in accordance with a preset condition for the first time in the video to be processed, and the preset condition at least comprises an observation target; Step b, detecting the position of an observation target in the reference frame, and determining the reference key position of the reference frame; c, taking a frame of image after the reference frame as a current frame; Step d, detecting the position of an observation target in the current frame, and determining the current key position of the current frame; Step e, determining the offset of the observation target in the current frame by the difference between the current key position and the reference key position; step f, calibrating the current frame based on the offset; and under the condition that the current frame is not the last frame of the video to be processed, reading the next frame of the current frame from the video to be processed, determining the next frame of the current frame as a new current frame, and using the steps d to f to realize the calibration of each frame of image.
2. 2. The method according to claim 1, wherein the method further comprises: Carrying out morphological recognition on an observation target in any frame of target image in the video to be processed to obtain a first recognition result; and determining the center of a fitting graph aiming at the observation target in the target image as a key position under the condition that the first recognition result meets a first morphological condition, wherein the key position represents a reference key position of the reference frame under the condition that the target image is the reference frame, and the key position represents a current key position of the current frame under the condition that the target image is the current frame.
3. 3. The method according to claim 2, wherein the method further comprises: Under the condition that the first recognition result meets a second form condition, determining the key position of the target image from the edge information of the fitting graph of the target image aiming at the observation target; the second morphological condition represents that the duty ratio of the observation target in the fitting graph is smaller than that of the first morphological condition represents that the observation target in the fitting graph.
4. 4. The method of claim 1, wherein the performing position detection on the observation target in the reference frame to determine the reference key position of the reference frame comprises: preprocessing the reference frame, removing interference elements in the reference frame, and obtaining a first preprocessed image; performing contour detection on the first preprocessed image to determine a first contour of an observation target; fitting the first contour to determine a first fitting geometry; in the first fitting geometry, a reference key location of the reference frame is determined.
5. 5. The method of claim 4, wherein the performing position detection on the observation target in the current frame to determine the current key position of the current frame comprises: preprocessing the current frame, removing interference elements in the current frame, and obtaining a second preprocessed image; performing contour detection on the second preprocessed image to determine a second contour of the observation target; fitting the second contour to determine a second fitting geometry; and determining the current key position of the current frame based on the second fitting geometric shape.
6. 6. The method of claim 5, wherein the observation target comprises a moon or a sun, the first fitting geometry comprises a first circle, the reference key location comprises a center of the first circle; The second fitting geometry comprises a second circle, and the current key position comprises the center of the second circle.
7. 7. The method of claim 5, wherein the observation target comprises a moon or a sun, the first fitting geometry comprises a first circle and a first bounding rectangle circumscribing the first circle, the reference key location comprises a vertex of the first bounding rectangle; the second fitting geometry comprises a second circle and a second external rectangle circumscribing the second circle, and the current key position comprises a vertex of the second external rectangle.
8. 8. The method of claim 7, wherein the observed target comprises a moon in a non-full month state or a sun in a solar food scene, wherein the determining the current key location of the current frame based on the second fitting geometry comprises: Determining the convex surface orientation of the observation target based on image recognition in the observation target in the current frame; And determining the current key position of the current frame in the second fitting geometrical shape based on the convex surface orientation of the observation target.
9. 9. The method of claim 8, wherein the convex surface is represented toward a designated point in the convex surface using the observation target; the determining, based on the convex orientation of the observation target, the current key position of the current frame in the second fitting geometry includes: and taking the vertex closest to the appointed point from among the vertices of the second externally connected rectangle as the current key position.
10. 10. The method of claim 7, wherein the observed target comprises a moon in a non-full month state or a sun in a solar food scene, wherein the determining the current key location of the current frame based on the second fitting geometry comprises: fitting the second contour to determine a minimum circumscribed rectangle under the condition that the ratio of the area of the second contour of the current frame to the circumscribed circle of the first contour of the reference frame is smaller than a set ratio; And determining that one vertex of the second circumscribed rectangle is the current key position based on the included angle between the minimum circumscribed rectangle and the horizontal direction of the current frame under the condition that the long side of the minimum circumscribed rectangle is smaller than the diameter of the circumscribed circle of the first outline of the reference frame.
11. 11. The method of claim 10, wherein determining one vertex of a second bounding rectangle as the current key location based on the angle between the minimum bounding rectangle and the horizontal direction of the current frame comprises: determining a horizontal reference point and a vertical reference point according to the included angle between the minimum circumscribed rectangle and the horizontal direction of the current frame; And determining a current key position in the vertexes of the second circumscribed rectangle based on the horizontal reference point and the vertical reference point, wherein the current key position is the vertex adjacent to the horizontal reference point and the vertical reference point.
12. 12. The method of claim 10, wherein the second fitting geometry comprises a second circle, wherein determining a horizontal reference point and a vertical reference point based on an angle of the minimum bounding rectangle with a horizontal direction of the current frame comprises: Under the condition that the included angle between the long side of the minimum circumscribed rectangle and the horizontal direction of the current frame is in a first range, determining a first intersection point of the second circle and the second circumscribed rectangle in the horizontal direction as a horizontal reference point, and determining a second intersection point of the second circle and the second circumscribed rectangle in the vertical direction as a vertical reference point; Determining a third intersection point of the second circle and the second circumscribed rectangle in the horizontal direction as a horizontal reference point and determining a second intersection point of the second circle and the second circumscribed rectangle in the vertical direction as a vertical reference point under the condition that the included angle between the long side of the minimum circumscribed rectangle and the horizontal direction of the current frame is in a second range; Determining a third intersection point of the second circle and the second circumscribed rectangle in the horizontal direction as a horizontal reference point and determining a fourth intersection point of the second circle and the second circumscribed rectangle in the vertical direction as a vertical reference point under the condition that the included angle between the long side of the minimum circumscribed rectangle and the horizontal direction of the current frame is in a third range; And under the condition that the included angle between the long side of the minimum circumscribed rectangle and the horizontal direction of the current frame is in a fourth range, determining that a first intersection point of the second circle and the second circumscribed rectangle in the horizontal direction is a horizontal reference point, and determining that a fourth intersection point of the second circle and the second circumscribed rectangle in the vertical direction is a vertical reference point.
13. 13. The method of claim 7, wherein the observed target comprises a moon in a non-full month state or a sun in a solar food scene, wherein the determining the current key location of the current frame based on the second fitting geometry comprises: fitting the second contour under the condition that the ratio of the area of the second contour of the current frame to the circumscribed circle of the first contour of the reference frame is smaller than a set ratio, and determining a minimum circumscribed triangle and a minimum circumscribed rectangle; And determining one vertex of the second circumscribed rectangle as the current key position based on the position of the maximum included angle in the minimum circumscribed triangle under the condition that the long side of the minimum circumscribed rectangle is not smaller than the diameter of the circumscribed circle of the first outline of the reference frame.
14. 14. The method of claim 13, wherein determining, based on the location of the largest included angle in the smallest bounding triangle, one vertex of the second bounding rectangle as the current critical location comprises: and taking the vertex closest to the position of the maximum included angle in the minimum circumscribed triangle in the vertices of the second circumscribed rectangle as the current key position.
15. 15. The method of claim 1, wherein the calibrating the current frame based on the offset comprises: Based on the offset, performing translation operation on the current frame to obtain a translation image; And carrying out background filling on the edge of the translation image.
16. 16. The method according to any one of claims 1 to 15, wherein the video to be processed is a video being photographed, and wherein after the calibrating the current frame based on the offset, the method further comprises: Replacing a corresponding frame in the video being photographed with the calibrated image; In the case where an operation to end video recording is obtained, the calibrated video is saved.
17. 17. An image capturing apparatus comprising a processor, a memory storing machine-readable instructions executable by the processor, which when executed by the processor perform the steps of the method according to any one of claims 1 to 16.
18. 18. A computer-readable storage medium, characterized in that it has stored thereon a computer program which, when executed by a processor, performs the steps of the method according to any of claims 1 to 16.
19. 19. A computer program product, characterized in that the computer program product comprises a computer program which, when executed by a processor, implements the method of any one of claims 1 to 16.

Description

Video processing method, image pickup apparatus, and computer-readable storage medium Technical Field The present application relates to the field of image processing technology, and in particular, to a video processing method, an image capturing apparatus, and a computer-readable storage medium. Background During shooting by the image pickup apparatus, there may be some uncontrolled shake of the image pickup apparatus, and a small movement of the camera of the image pickup apparatus may cause a shift between frames of the shot video in the vertical and horizontal directions. Especially when a camera uses a tele lens or digital zooming, the phenomenon is obvious, and the shot video jitter is obvious. Disclosure of Invention An object of the present application is to provide a video processing method, an image capturing apparatus, and a computer-readable storage medium capable of improving the offset between frames of a video captured by the image capturing apparatus due to shake. The invention provides a video processing method, which comprises the steps of a, sequentially identifying each frame image of a video to be processed to determine a reference frame, wherein the reference frame is an image which is identified in the video to be processed for the first time and accords with preset conditions, the preset conditions at least comprise an observation target, b, detecting the position of the observation target in the reference frame to determine the reference key position of the reference frame, c, taking a frame image after the reference frame as a current frame, d, detecting the position of the observation target in the current frame to determine the current key position of the current frame, e, determining the difference between the current key position and the reference key position to determine the offset of the observation target in the current frame, f, calibrating the current frame based on the offset, reading the next frame of the current frame from the video to be processed under the condition that the current frame is not the last frame of the video to be processed, and determining the next frame of the current frame as a new current frame, and using the steps d to realize the calibration of each frame. In the implementation manner, the reference frame is selected from each frame of image of the video to be processed, and the offset can be adjusted by taking the reference frame as a reference for the subsequent other images of the video to be processed, so that the whole video can be integrally adjusted by taking one frame of image as a reference, the video can be aligned with the reference frame, and the whole video can be relatively more stable. In an optional implementation manner, the method further comprises the steps of carrying out morphological recognition on an observed target in any frame of target image in the video to be processed to obtain a first recognition result, determining the center of a fitting graph aiming at the observed target in the target image as a key position when the first recognition result meets a first morphological condition, wherein the key position represents a reference key position of the reference frame when the target image is the reference frame, and the key position represents a current key position of the current frame when the target image is the current frame. In the implementation manner, when the key positions of the reference frame and the current frame to be calibrated are selected, the representative position (central position) in the observation target in the image is used as the key position, so that each frame of image is relatively easier to locate and offset is also easier to determine. In an optional implementation mode, the method further comprises determining a key position of the target image from edge information of a fitting graph of the target image for the observation target under the condition that the first recognition result meets a second morphological condition, wherein the second morphological condition represents that the occupancy rate of the observation target in the fitting graph is smaller than that of the first morphological condition. In the implementation manner, when the key positions of the reference frame and the current frame to be calibrated are selected, the position determined by representative edge information in the observation target in the image is used as the key position, so that each frame of image is relatively easier to position and offset is also easier to determine. In an alternative embodiment, the detecting the position of the observed target in the reference frame to determine the reference key position of the reference frame includes preprocessing the reference frame, removing interference elements in the reference frame to obtain a first preprocessed image, detecting the contour of the first preprocessed image to determine a first contour of the observed target, fitting the first contour to dete