CN-121861078-B - Motion estimation-oriented curvature enhancement high-displacement image variation optical flow method

CN121861078BCN 121861078 BCN121861078 BCN 121861078BCN-121861078-B

Abstract

The invention provides a curvature enhancement strong displacement image variable light flow method for motion estimation, which relates to the field of image processing and aims to describe the complexity of a local structure of an image by introducing the curvature of an image contour line, and carry out limited self-adaptive weighted adjustment on the constant-brightness constraint and the constant-gradient constraint on the data item level of a light flow variable model based on the curvature, so as to inhibit the interference of unreliable matching on the light flow estimation in a complex structure area, improve the robustness of the light flow estimation in illumination change, complex texture and large displacement scenes and ensure the numerical stability and convergence of the model in the multi-scale calculation process.

Inventors

LIU BOWEN
LI HUANTAO
ZHANG YULIN
WANG KAI
KONG XIANGYU
PAN YUE
LI ZHONGTAO

Assignees

济南大学

Dates

Publication Date: 20260508
Application Date: 20260318

Claims (8)

1. The curvature enhancement high-displacement image variation optical flow method for motion estimation is characterized by comprising the following steps of: s1, acquiring two adjacent frames of input images, and constructing an image pyramid for the two adjacent frames of input images; S2, selecting a pyramid layer currently used for optical flow estimation from an image pyramid, initializing an optical flow field and an optical flow increment of the pyramid layer, and constructing a variable optical flow energy model containing brightness invariant constraint, gradient invariant constraint and curvature weight factors ; S3, in the variable optical flow energy model Introducing an adaptive weight adjustment mechanism based on image curvature, representing the complexity of a local structure of an image through the image curvature, and performing differential weighting adjustment on a brightness invariable constraint item and a gradient invariable constraint item; S4, constructing a curvature weight function which monotonically decreases and is bounded with respect to the curvature of the image, introducing a self-adaptive adjusting factor, and dynamically restraining and adjusting the curvature weight; s5, aiming at the nonlinear variation optical flow model with curvature weighting introduced, carrying out variable splitting and collaborative optimization on the data item and the total variation regular item by adopting an alternate direction multiplier method, so as to realize model solving; S6, after the current pyramid layer obtains an optical flow updating result, interpolating and transmitting the optical flow updating result to a higher resolution layer, and repeating the steps S2 to S5 until the highest resolution layer finishes calculation; And S7, after the optical flow increment iteration update is completed on the highest resolution layer, outputting a final optical flow field for representing pixel-level motion information between two adjacent frames of input images.
2. The method for enhancing the variable optical flow of a large-displacement image with curvature oriented to motion estimation according to claim 1, wherein in the step S1, two adjacent frames of input images are acquired And Preprocessing the two adjacent frames of input images to construct an image pyramid composed of N layers of images with different resolutions, wherein the image with the lowest resolution is positioned at the top layer of the pyramid and the image with the highest resolution is positioned at the bottom layer of the pyramid, and the image pyramid is formed on the pyramid layer Initializing a horizontal component of the optical flow field corresponding to the current pyramid layer And a vertical component And corresponding optical flow delta And After the optical flow estimation of the initial optical flow calculation layer is completed, the optical flow field obtained by the layer is used as an initial value of optical flow calculation for the image with the higher resolution of the layer, and the optical flow estimation is carried out layer by layer according to the sequence from the low resolution layer to the high resolution layer, so that the gradual estimation of the large displacement optical flow is realized.
3. The method for enhancing the variable optical flow of the large-displacement image with curvature facing motion estimation according to claim 2, wherein in the step S2, a variable optical flow energy model comprising both a luminance invariant constraint and a gradient invariant constraint is constructed, and the curvature weight factor is calculated The specific steps of (a) are as follows: s21, initializing optical flow fields of two frames of input images at the current image pyramid layer to obtain optical flow horizontal components And a vertical component And optical flow delta corresponding thereto And Respectively carrying out partial derivative calculation on the two frames of input images of the current pyramid layer, wherein the first-order spatial partial derivative of the input image in the horizontal direction is calculated based on the constant constraint of brightness First order spatial partial derivative in vertical direction And a luminance variation partial derivative in the time axis direction Based on the invariable constraint of the gradient, calculating the spatial gradient of the input image to obtain the first Gradient partial derivative of frame input image in horizontal direction Gradient partial derivative in vertical direction And a corresponding gradient change term of the gradient with time Wherein =1, 2 Represents a first frame input image and a second frame input image, respectively; S22, introducing brightness invariant constraint and gradient invariant constraint into the current pyramid layer to construct a variable optical flow energy function containing curvature weight factors For measuring motion consistency between adjacent image pixels, the functional form is defined as: , wherein, A gradient vector representing the luminance of the input image spatially, Representing the magnitude of the gradient vector, The operator of the degree of divergence is represented, Curvature information representing the contour of the image, The weighting function is constructed based on image curvature and is used for carrying out differential weighting adjustment on the constant brightness constraint item and the constant gradient constraint item according to the complexity of the local structure of the image; the image definition field is represented as such, For the weight coefficient of the total variation regular term, constructing a mixed data item containing a brightness invariable constraint term and a gradient invariable constraint term on the basis Expressed as The blended data item is used to enhance the robustness of optical flow matching under illumination variations and complex texture conditions.
4. A method for enhancing a variable optical flow of a large-displacement image with curvature oriented to motion estimation according to claim 3, wherein in the step S3, an adaptive weight adjustment mechanism based on image curvature is introduced into a data item of a variable optical flow energy model, and specifically includes: s31, constructing an adaptive weight function based on image curvature Wherein The method is used for representing curvature information of brightness contour lines of the input image and describing geometric complexity of local structures of the image; S32, self-adapting curvature to weight function Multiplying said data item as an integral coefficient Before, performing differential weighting adjustment on the brightness invariant constraint term and the gradient invariant constraint term, thereby obtaining an improved variable optical flow energy model: When the local curvature of the image is large, the self-adaptive weight function Taking smaller values to reduce the influence of data items in complex structure regions on the light flow estimation, and adaptive weighting functions when the local curvature of the image is smaller Take larger values to enhance the constraint ability of data items in flat areas of the structure on the light flow estimation.
5. The method for enhancing the variable optical flow of a large-displacement image with curvature oriented to motion estimation according to claim 4, wherein in the step S4, a curvature weight function which monotonically decreases and is bounded with respect to the curvature of the image is constructed, and an adaptive adjustment factor is introduced, so as to dynamically constrain and adjust the curvature weight, and the method specifically comprises: S41, calculating image curvature according to the input image at the current pyramid layer The image curvature is defined as: wherein ε is a normal number constant that prevents denominator from being zero; s42, based on the image curvature Respectively constructing curvature weight functions acting on the brightness invariable constraint term and the gradient invariable constraint term And (3) with The curvature weight function is a function with monotonically decreasing absolute value of curvature and bounded value, and mathematical models thereof are respectively expressed as: , and satisfy 0< ≤1,0< Is less than or equal to 1 percent, wherein, And (3) with Curvature adjustment coefficients corresponding to the brightness-invariant constraint term and the gradient-invariant constraint term respectively; S43, introducing an adaptive factor into the curvature adjustment coefficient And dynamically adjusting the curvature adjustment coefficient according to the curvature of the image, wherein a mathematical model is expressed as follows: , , wherein, And (3) with On the basis, a curvature weight function is respectively introduced into a brightness invariable constraint item and a gradient invariable constraint item, and the original mixed data item is subjected to weighted correction so as to obtain a curvature enhanced data item The data item weight can be dynamically adjusted along with the local curvature of the image, and the mathematical model is as follows: 。
6. the motion estimation-oriented curvature enhancement strong displacement image variation optical flow method according to claim 5, wherein in the step S5, for a nonlinear variation optical flow energy model with curvature self-adaptive weighting introduced, an alternate direction multiplier method is adopted to solve the energy model, and specifically comprises the steps of performing variable splitting on gradient constraint in a total variation regularization term in the variation optical flow energy model, and introducing auxiliary variables Lagrange multiplier corresponding to the Lagrange multiplier And constructs the following constraint relation: Wherein And (3) representing the index of the optical flow component, and constructing a data item with enhanced curvature and a total variation regularization item together into an enhanced Lagrange energy function on the basis, wherein a mathematical model is as follows: , wherein, The weight parameters are regularized for the total variation, To constrain penalty parameters, the optical flow increment variables are sequentially optimized by using an alternative optimization strategy Auxiliary variable Lagrangian multiplier And performing iterative updating to realize collaborative optimization among the curvature enhancement data item, the total variation regularization item and the constraint condition.
7. The method for motion estimation-oriented curvature enhancement of large-displacement image variation optical flow according to claim 6, wherein in said step S6, the optical flow increment is And On the basis of meeting the Euler-Lagrange optimality condition, adopting a gradient descent mode to carry out iterative updating, wherein the updating mode meets the following relation: , wherein, And Respectively representing the spatial coordinates of the images And Is used for the bias operator of the (c), In order to constrain the penalty parameters, As an auxiliary variable of the horizontal optical flow gradient, As an auxiliary variable of the vertical optical flow, The corresponding lagrangian multiplier is constrained for horizontal optical flow, Lagrangian multiplier corresponding to vertical optical flow constraint For the preset iteration step parameters, Is the first And iterating for a plurality of times to ensure the numerical stability and convergence of the optical flow increment updating process.
8. The method for enhancing the variable optical flow of a large-displacement image for motion estimation according to claim 7, characterized in that in step S7, at the highest resolution image pyramid layer Acquiring corresponding final optical flow delta And And the final optical flow increment is respectively corresponding to the optical flow horizontal component of the layer And a vertical component Performing superposition updating to obtain horizontal component of final optical flow field And a vertical component Wherein the update relationship satisfies: And taking the final optical flow field as an output result of a large displacement variation optical flow calculation method based on curvature enhancement, and using the final optical flow field as pixel-level motion information between two adjacent frames of input images.

Description

Motion estimation-oriented curvature enhancement high-displacement image variation optical flow method Technical Field The invention belongs to the field of image processing, and particularly relates to a curvature enhancement strong displacement image variation optical flow method for motion estimation. Background Optical flow estimation is a basic task in computer vision and is used for estimating pixel-level motion information from a continuous image sequence, a traditional variable optical flow method generally constructs an energy function based on brightness invariant assumption or gradient invariant assumption and introduces a regular term to smooth an optical flow field, however, when a scene with large displacement motion, a complex texture structure or obvious illumination change is processed, the traditional method is easy to generate mismatching in a complex structure area, so that the optical flow estimation accuracy is reduced. The existing method usually adopts image pyramid structure layer-by-layer optimization or introduces local feature enhancement constraint, but still does not fully consider the influence of image local geometry on matching reliability; curvature is used as a characteristic for describing the local geometric complexity of an image and can be used for distinguishing a flat area from a complex-structure area, but no method exists at present for introducing the system into a weighting mechanism of an optical flow data item so as to adaptively adjust the constraint intensity of different areas; in order to solve the problems, the invention provides the method for describing the local structure complexity of the image by introducing the curvature of the contour line of the image, and carrying out limited self-adaptive weighted adjustment on the constant-brightness constraint and the constant-gradient constraint on the data item level of the optical flow variation model based on the curvature, thereby inhibiting the interference of unreliable matching on the optical flow estimation in the complex structure area, improving the robustness of the optical flow estimation in the illumination variation, complex texture and large displacement scene, and ensuring the numerical stability and convergence of the model in the multi-scale calculation process. Disclosure of Invention The invention aims to provide a curvature enhancement strong displacement image variation optical flow method for motion estimation, which is characterized by introducing image contour curvature to describe the local structure complexity of an image, and carrying out limited self-adaptive weighted adjustment on brightness invariable constraint and gradient invariable constraint on the data item level of an optical flow variation model based on the curvature, so that interference of unreliable matching on the optical flow estimation in a complex structure area is inhibited, the robustness of the optical flow estimation in illumination change, complex texture and large displacement scenes is improved, and the numerical stability and convergence of the model in a multi-scale calculation process are ensured. In order to achieve the purpose, the invention adopts the following technical scheme that the curvature enhancement high-displacement image variation optical flow method for motion estimation comprises the following steps. S1, acquiring two adjacent frames of input images, and constructing an image pyramid for the two adjacent frames of input images. S2, selecting a pyramid layer currently used for optical flow estimation from an image pyramid, initializing an optical flow field and an optical flow increment of the pyramid layer, and constructing a variable optical flow energy model containing brightness invariant constraint, gradient invariant constraint and curvature weight factors。 S3, in the variable optical flow energy modelIntroducing an adaptive weight adjustment mechanism based on image curvature, representing the complexity of the local structure of the image through the image curvature, and performing differential weighting adjustment on the constant brightness constraint item and the constant gradient constraint item. S4, constructing a curvature weight function which monotonically decreases and is bounded with respect to the curvature of the image, introducing an adaptive adjustment factor, and dynamically restraining and adjusting the curvature weight. S5, aiming at the nonlinear variation optical flow model with curvature weighting introduced, adopting an alternate direction multiplier method to carry out variable splitting and collaborative optimization on the data item and the total variation regular item, and realizing model solving. And S6, after the current pyramid layer obtains an optical flow updating result, interpolating and transmitting the optical flow updating result to a higher resolution layer, and repeating the steps S2 to S5 until the highest resolution layer finishes calculation.