CN-121978703-A - Vibrating mirror type vision sensor capable of dynamically imaging compensation and dynamic imaging compensation method

CN121978703ACN 121978703 ACN121978703 ACN 121978703ACN-121978703-A

Abstract

The invention provides a vibrating mirror type vision sensor capable of dynamically imaging compensation and a dynamic imaging compensation method, belongs to the technical field of sensors, and aims to track, lock and cooperatively move in a dynamic imaging process by utilizing the high dynamic characteristic of a vibrating mirror to realize high-definition imaging under a dynamic condition. The invention mainly comprises three parts of a vibrating mirror type vision sensor system design capable of dynamic imaging compensation, a mathematical model and a tracking compensation method, and provides a hardware design, a software framework, a modeling operation basis and a dynamic imaging compensation method for the sensor design. The invention can realize clear imaging in the dynamic measurement process and provide image guarantee for improving the dynamic measurement precision.

Inventors

LIU ZHEN
YAN FENG
PAN XIAO

Assignees

北京航空航天大学

Dates

Publication Date: 20260505
Application Date: 20260127

Claims (10)

1. The vibrating mirror type vision sensor capable of dynamically imaging compensation is characterized by comprising a high-speed camera, a laser range finder, a beam splitting prism and a two-dimensional vibrating mirror, wherein the high-speed camera and the laser range finder are respectively positioned on a transmission surface and a reflection surface of the beam splitting prism to form a set of coaxial optical system, the laser range finder solves the problem that a monocular vision sensor is free of scale, meanwhile, the precise control of a visual angle is realized by combining a mathematical model, and the two-dimensional vibrating mirror is arranged behind the beam splitting prism, reflects a coaxial visual field and controls the mirror to rotate to rapidly adjust the visual field angle.
2. The dynamically image-compensated galvanometer vision sensor of claim 1, wherein during operation, a system view angle follows a target rotation, and a moving speed of the target can be estimated, and during exposure, the view angle is controlled to continuously follow the target movement, so that the target is always imaged at the same pixel position to prevent motion blur.
3. The dynamically image-compensated galvanometer vision sensor of claim 1, wherein after the image is acquired, an image tracking detection algorithm is first performed to obtain a target position, and the galvanometer rotation is calculated and controlled according to a deviation of the target from the center of the image, so as to keep the target always in the center region of the image, thereby realizing tracking of the target.
4. The dynamically image-compensated galvanometer vision sensor of claim 3, wherein the motion speed of the target is estimated, the galvanometer is controlled to move synchronously with the target during the next camera exposure process, and the target is kept to be imaged at the same position of the sensitive element all the time, thereby realizing the dynamic image compensation of the hardware level.
5. The dynamically image-compensable galvanometer vision sensor of claim 1, wherein a mathematical model of the dynamically image-compensable galvanometer vision sensor describes a relationship between a galvanometer corner and a change in a viewing angle, a reflection transformation is established by analysis, a complete three-dimensional coordinate transformation model is established, and simultaneously, a three-dimensional position of a target is reconstructed in combination with depth information provided by a laser range finder.
6. The dynamically image-compensable galvanometer vision sensor of claim 5, wherein the imaging process of the galvanometer is described by formula (1): (1) Wherein, the Is any scale factor that is not zero, Undistorted two-dimensional image point being an object point Is used for the matching of the coordinate system, Is an internal parameter matrix of the camera, Is a mirror parameter The derived matrix of external parameters is used, Is the object point In a two-dimensional galvanometer camera coordinate system Lower three-dimensional homogeneous coordinates; Mirror parameters Is a set of parameters describing the state of the mirror surface of the galvanometer, wherein 、、 And Respectively an X-axis motor corner, a Y-axis motor corner camera-to-X axis mirror distance and distance between X axis mirror and Y axis mirror; mapping the galvanometer camera G into a virtual camera C through reflection transformation; for primary reflection transformation, determining a primary reflection transformation matrix by knowing the normal direction of the reflecting surface and a point, as shown in a formula (2); (2) Wherein, the To rotate matrix for three-dimensional homogeneous coordinates of reflected object points , Is a mirror normal vector, a translation vector , Fixed point for the passing mirror; Representing one The superscript T represents the transpose of the matrix; according to mirror parameters, namely the transformation relation between the vibrating mirror camera and the first reflection virtual camera which are established in sequence And a transformation relationship between the first reflective virtual camera and the second reflective virtual camera And deriving an external parameter matrix according to formula (3) : (3)。
7. The dynamically image-compensated galvanometer vision sensor of claim 6, Based on the formula (3), under the condition of determining the mirror parameters, calculating the projection process under any view angle, and further, establishing a calculation model from the image to the rotation angle, comprising: for points to be pointed Let its two-dimensional image homogeneous coordinates be To point to Three-dimensional coordinates in the current virtual camera coordinate system The calculation is as follows: (4) Wherein z is the result obtained by measuring by a laser range finder, The undistorted two-dimensional image homogeneous coordinates of the object point; According to And precisely calculating to obtain a coordinate system transformation relation, and further calculating to obtain the required rotation angle of the vibrating mirror.
8. A dynamic imaging compensation method of a vibrating mirror type vision sensor capable of dynamically imaging compensation is characterized by comprising the following steps: step1, tracking and detecting the target position in an image obtained from the current visual angle by using a tracker, and based on an external parameter matrix Calculating to obtain the coordinate system of the virtual camera in the current Three-dimensional coordinates of (a); Step 2, unifying the coordinates to a reference camera coordinate system according to the mathematical model and through the coordinate transformation relation among different visual angles In (a) and (b); Step3, in a coordinate system The result of Kalman filtering is combined with a model of image deviation to camera rotation angle, so that the rotation angle of the vibrating mirror required by pointing to the latest position of the target can be calculated; step4, using a PID control strategy, and estimating overshoot by introducing integral quantity and differential quantity, so as to realize accurate tracking; And 5, estimating the movement amount of the target in the exposure process by using the speed obtained by Kalman filtering calculation in the exposure process, and controlling the vibrating mirror to continuously rotate according to the estimated speed in the camera exposure process, so that synchronous movement with the target is realized, dynamic imaging compensation is completed, and clear imaging is obtained.
9. The dynamic imaging compensation method of a galvanometer vision sensor capable of dynamic imaging compensation according to claim 8, wherein in said step 1, the transformation relationship between the galvanometer camera and the first reflection virtual camera is sequentially established And a transformation relationship between the first reflective virtual camera and the second reflective virtual camera And deriving an external parameter matrix M according to the formula (3): (3)。
10. The dynamic imaging compensation method of the vibrating mirror type vision sensor capable of dynamic imaging compensation according to claim 8, wherein the mathematical model in the step 2 describes the relation between the corner of the vibrating mirror and the change of the visual angle; The imaging process of the galvanometer is described as: (1) Wherein, the Is any scale factor that is not zero, Undistorted two-dimensional image point being an object point Is used for the matching of the coordinate system, Is an internal parameter matrix of the camera, Is a mirror parameter The derived matrix of external parameters is used, Is the object point In a two-dimensional galvanometer camera coordinate system Lower three-dimensional homogeneous coordinates; Mirror parameters Is a set of parameters describing the state of the mirror surface of the galvanometer, wherein 、、 And The method comprises the steps of respectively obtaining an X-axis motor corner, a Y-axis motor corner, a distance from a camera to an X-axis mirror surface and a distance between the X-axis mirror surface and the Y-axis mirror surface, mapping a galvanometer camera G into a virtual camera C through reflection transformation, and determining a single reflection transformation matrix by knowing the normal direction of a reflection surface and a point for primary reflection transformation, wherein the single reflection transformation matrix is shown as a formula (2); (2) Wherein, the To rotate matrix for three-dimensional homogeneous coordinates of reflected object points , Is a mirror normal vector, a translation vector , Fixed point for the passing mirror; Representing one The superscript T represents the transpose of the matrix; according to mirror parameters, namely the transformation relation between the vibrating mirror camera and the first reflection virtual camera which are established in sequence And a transformation relationship between the first reflective virtual camera and the second reflective virtual camera And deriving an external parameter matrix M according to the formula (3): (3) Thus, based on equation (3), the projection process at any viewing angle is calculated under the condition of determining the mirror parameters, and further, a calculation model from the image to the rotation angle is established, comprising: for points to be pointed Let its two-dimensional image homogeneous coordinates be To point to Three-dimensional coordinates in the current virtual camera coordinate system The calculation is as follows: (4) Wherein z is the result obtained by measuring by a laser range finder, The undistorted two-dimensional image homogeneous coordinates of the object point; According to And precisely calculating to obtain a coordinate system transformation relation, and further calculating to obtain the required rotation angle of the vibrating mirror.

Description

Vibrating mirror type vision sensor capable of dynamically imaging compensation and dynamic imaging compensation method Technical Field The invention belongs to the technical field of sensors, and particularly relates to a vibrating mirror type vision sensor capable of dynamically imaging compensation and a dynamic imaging compensation method. Background Dynamic vision measurement refers to an observation process of relative movement between an observation target and a measurement system, when exposure time is long or target movement speed is high, for example, motion blur is easy to occur in low-illumination scenes such as dark night or under the condition that jolt vibration of a moving carrier and the like is serious, and more serious image distortion is caused if a rolling shutter door camera is used, so that the measurement is difficult to realize by the existing algorithm. How to realize high-definition imaging under the dynamic measurement condition, and solving image distortion and blurring are key to realizing high-precision dynamic vision measurement. The existing mode for solving the distortion and the blurring of the image mainly comprises a software method, wherein the software method is used for performing post-processing after the image is acquired through an image deblurring algorithm so as to eliminate the distortion and the blurring, and the hardware method is used for solving the influence of motion on measurement by using a visual stabilization platform. The software method generally relies on prior or estimated fuzzy kernels to perform operations such as deconvolution, so that the definition of the image is improved, and the software method is widely applied to sensing tasks such as recognition, segmentation and the like. However, since the projection correspondence between the spatial object and the image pixels cannot be strictly restored, it is difficult to overcome the precision measurement task. The hardware method can solve the problems of distortion and blurring in the measuring process, and is more suitable for measuring tasks than the software method. The existing visual stable platform mainly comprises an inertial platform and a PTZ (Pan-Tilt-Zoom) camera, wherein the inertial platform can only sensitively and compensate the motion of the inertial platform, and cannot actively capture the motion of an observation target, so that distortion and blurring caused by the motion of the target cannot be compensated, and in contrast, the PTZ camera can actively sense and follow the rotation of the target, but the mechanical turntable is slower in speed, so that tracking of a high-motion target is difficult to realize, and imaging compensation is more difficult to realize. Therefore, if a high-speed visual angle conversion device can be used and a dynamic imaging compensation mechanism is established, the problems of distortion and blurring in the dynamic imaging process can be effectively solved. The vibrating mirror is an actuating mechanism which consists of a high-speed motor and a light lens and can deflect rapidly, and a common laser is matched with the actuating mechanism for laser marking, laser radar and other scenes. In recent years, a scholars have combined a galvanometer and a camera to form a galvanometer type vision sensor, and the direction of a field of view is changed by rotation of a mirror surface. However, the above solution fails to build an accurate view angle control model, so that the solution is only used for target tracking, and dynamic imaging compensation of flexible and maneuvering targets cannot be realized. Disclosure of Invention In order to solve the problem of distortion and blurring which are easy to occur in the dynamic vision measurement process, the invention provides the vibrating mirror type vision sensor capable of dynamic imaging compensation and the dynamic imaging compensation method, and the high-dynamic characteristic of the vibrating mirror is utilized to realize target tracking locking and cooperative movement in the dynamic imaging process, so that high-definition imaging under the dynamic condition is realized. The invention mainly comprises three parts of a vibrating mirror type vision sensor system design capable of dynamic imaging compensation, a mathematical model and a tracking compensation method, and provides a hardware design, a software framework, a modeling operation basis and a dynamic imaging compensation method for the sensor design. The invention can realize clear imaging in the dynamic measurement process and provide image guarantee for improving the dynamic measurement precision. In order to achieve the above purpose, the invention adopts the following technical scheme: The vibrating mirror type vision sensor capable of dynamically imaging compensation comprises a high-speed camera, a laser range finder, a beam splitting prism and a two-dimensional vibrating mirror, wherein the high-speed camera and the laser range finde