CN-121977436-A - Binocular vision sensor of double-view single-camera and measuring method

Abstract

The invention discloses a binocular vision sensor of a double-view single-camera and a measuring method thereof, belonging to the technical field of computer vision and measurement. The sensor aims to solve the problems that the common view field of the traditional binocular vision system is limited, and the front-back bidirectional wide view field three-dimensional measurement is difficult to realize simultaneously. The sensor consists of an industrial camera, two plane mirrors, a triangular prism and a sensor bracket, wherein the reflecting surfaces on two sides of the two plane mirrors and the triangular prism are respectively combined with the same camera to form 4 virtual cameras, two pairs of virtual binocular units are formed, and the optical axes of the two virtual cameras of the same virtual binocular are parallel. The measuring method comprises a calibration stage and a measuring stage, and three-dimensional coordinate measurement of the target characteristic points is realized by establishing a virtual camera imaging model and a sensor three-dimensional measuring model, so that the measuring method is suitable for high-precision three-dimensional measurement of the target in real-time sensing and positioning of an autonomous unmanned system in a complex environment.

Inventors

• ZHOU FUQIANG
• Xing Ruidan

Assignees

• 北京航空航天大学

Dates

Publication Date

20260505

Application Date

20251202

Claims (2)

1. 1. The binocular vision sensor of the double-view single-camera is characterized by comprising a plane folding mirror group (1), an industrial camera (2), a sensor bracket (3) and a camera supporting structure (4), wherein the plane folding mirror group (1) comprises a plane mirror A (5), a plane mirror B (6), a triangular prism (7) and a mirror group supporting structure (10), the triangular prism (7) is an isosceles triangular prism, two side surfaces of the triangular prism are reflecting mirror surfaces, namely a triangular prism A surface (8) and a triangular prism B surface (9), the bottom surface is a common plane, and the plane mirror A (5), the plane mirror B (6) and the triangular prism (7) are connected and fixed through the mirror group supporting structure (10); 1.1. The binocular vision sensor of the double-view single-camera is characterized in that the industrial camera (2) forms a virtual camera A (13) through mirror image of a plane mirror A (5) and forms a virtual camera B (14) through reflection of a triangular prism A surface (8), a virtual binocular unit A (11) is formed by the virtual camera A (13) and the virtual camera B (14), optical axes of the two virtual cameras are parallel to each other and have equal-sized field angles so as to ensure that the overlapping area of a public field of view is constant, and the virtual binocular unit A (11) performs three-dimensional measurement on a target object in the field of view A (20); 1.2. The binocular vision sensor of the double-view single-camera is characterized in that the industrial camera (2) forms a virtual camera D (16) through mirror image of a plane mirror B (6) and forms a virtual camera C (15) through reflection of a triangular prism B surface (9), a virtual binocular unit B (12) is formed by the virtual camera C (15) and the virtual camera D (16), the structure of the virtual binocular unit B (12) is the same as that of a virtual binocular unit A (11), and the virtual binocular unit B (12) is installed in the opposite direction, and the three-dimensional measurement is carried out on a target object in a view field B (21); 1.3. The binocular vision sensor of the double-view single-camera is characterized in that the virtual binocular unit A (11) and the virtual binocular unit B (12) are installed into a whole through the sensor bracket (3); 1.4. The binocular vision sensor of the double-view single-camera is characterized in that the focal length of the industrial camera (2) is 5-12 mm; 1.5. The binocular vision sensor of the double-view single-camera is characterized in that the vertical view angles of the view field A (20) and the view field B (21) are 15-20 degrees, the horizontal view angles are 50-75 degrees, and the sum of the horizontal view angles of the two view angles is 100-150 degrees, so that a wide view field range is formed.
2. 2. A measurement method for three-dimensional measurement using the binocular vision sensor of a dual view single camera of claim 1, comprising the steps of: Setting a calibration target A (17), a calibration target B (18) and a global camera (19), and calibrating the binocular vision sensor of the double-view single-camera to obtain imaging model parameters of each virtual camera, an essential matrix and structural parameters of a virtual binocular unit and structural parameters of the whole sensor; A measurement stage, namely utilizing a binocular vision sensor of a double-view single-camera to simultaneously perform three-dimensional measurement on a target object in a visual field A (20) and a visual field B (21) according to the sensor parameters acquired in the calibration stage; 2.1 the measurement method according to claim, wherein the three-dimensional coordinate system involved in the calibration and measurement phase of the binocular vision sensor of the dual-view single-camera comprises: Defining a three-dimensional coordinate system of a global camera (19) as a measurement coordinate system O-xyz of a binocular vision sensor of a double-view single-camera, and realizing unification of three-dimensional measurement coordinates of two virtual binocular units; The three-dimensional coordinate system of the virtual camera A (13) is O c1 -x c1 y c1 z c1 , the three-dimensional coordinate system of the virtual camera B (14) is O c2 -x c2 y c2 z c2 , the three-dimensional coordinate system of the virtual camera C (15) is O c3 -x c3 y c3 z c3 , and the three-dimensional coordinate system of the virtual camera D (16) is O c4 -x c4 y c4 z c4 ; Selecting a three-dimensional coordinate system O c2 -x c2 y c2 z c2 of a virtual camera B (14) as a three-dimensional coordinate system of a virtual binocular unit A (11), and selecting a three-dimensional coordinate system O c3 -x c3 y c3 z c3 of a virtual camera C (15) as a three-dimensional coordinate system of a virtual binocular unit B (12); 2.2 the method according to the preceding claim, wherein the structural parameters of the virtual binocular unit comprise: The transformation of O c1 -x c1 y c1 z c1 to O c2 -x c2 y c2 z c2 , the structural parameters of the virtual binocular unit A (11), are represented by a 3×3 rotation matrix R 1 and a 3×1 translation vector T 1 ; The transformation of O c4 -x c4 y c4 z c4 to O c3 -x c3 y c3 z c3 , the structural parameters of the virtual binocular unit B (12), are represented by a 3×3 rotation matrix R 2 and a 3×1 translation vector T 2 ; 2.3 the measurement method according to claim, wherein the structural parameters of the binocular vision sensor of the dual-view single-camera include: The transformation of O c2 -x c2 y c2 z c2 to O-xyz is represented by a 3×3 rotation matrix R 3 and a 3×1 translation vector T 3 , and the transformation of O c3 -x c3 y c3 z c3 to O-xyz is represented by a 3×3 rotation matrix R 4 and a 3×1 translation vector T 4 ; 2.4 the method according to the preceding claim, characterized in that the calibration phase comprises the following steps: Step S101, a calibration target A (17) and a calibration target B (18) are respectively arranged in a field A (20) and a field B (21) of an industrial camera (2), a global camera (19) is arranged, and the global camera (19) is ensured to be capable of shooting complete images of the calibration target A (17) and the calibration target B (18); step S102, freely moving at least six positions of a calibration target A (17) and a calibration target B (18) from near to far, wherein each time when one position is moved, the calibration target A (17) and the calibration target B (18) are ensured to be in the fields of view of an industrial camera (2) and a global camera (19) and simultaneously shoot an image, the image shot by the industrial camera (2) is represented by f (x, y), and the image shot by the global camera (19) is represented by g (x, y); Step S103, setting f (x, y) as m×n pixels, generating 4 virtual camera calibration images with f (x, y), and using f i (x, y), i=1,..4, and corresponding to virtual cameras A, B, C and D, respectively, where: Step S104 comprises calibrating imaging model parameters of the virtual camera A (13) by using at least six f 1 (x, y), calibrating imaging model parameters of the virtual camera B (14) by using at least six f 2 (x, y), calibrating structural parameters and an essential matrix of the virtual binocular unit A (11) by using at least six f 1 (x, y) and f 2 (x, y), calibrating imaging model parameters of the virtual camera C (15) by using at least six f 3 (x, y), calibrating imaging model parameters of the virtual camera D (16) by using at least six f 4 (x, y), and calibrating structural parameters and an essential matrix of the virtual binocular unit B (12) by using at least six f 3 (x, y) and f 4 (x, y) images; S105, calibrating imaging model parameters of a global camera (19) and structural parameters of a binocular vision sensor of a double-view single-camera by using at least six g (x, y) calibration images; 2.5 the measuring method according to claim, characterized in that the measuring phase comprises the steps of: Step S201, shooting target object images of a field A (20) and a field B (21) by using an industrial camera (2), namely a measurement image, wherein the measurement image is represented by S (x, y), the image size is set to be M multiplied by N pixels, 4 virtual camera measurement images are generated by S (x, y), and the virtual cameras A, B, C and D are respectively corresponding to the virtual cameras represented by S i (x, y), i=1. Step S202, respectively extracting characteristic points of the S 1 (x,y),s 2 (x, y) measurement image, and calculating three-dimensional coordinates of the characteristic points on the target object in the view field A (20) under O c2 -x c2 y c2 z c2 according to the parameters of the virtual binocular unit A (11) obtained in the step S104; Step S203, extracting the characteristic points of the S 3 (x,y),s 4 (x, y) measurement image respectively, and calculating the three-dimensional coordinates of the characteristic points on the target object in the field of view B (21) under O c3 -x c3 y c3 z c3 according to the parameters of the virtual binocular unit B (12) obtained in the step S104; and step S204, calculating the three-dimensional coordinates of the characteristic points in the O-xyz obtained in the steps S102 and S103 according to the structural parameters of the binocular vision sensor of the double-view single-camera obtained in the step S105, and realizing simultaneous three-dimensional measurement of the characteristic points of the target object in the double view.

Description

Binocular vision sensor of double-view single-camera and measuring method Technical Field The invention relates to the technical field of vision three-dimensional measurement, in particular to a binocular vision sensor of a double-view single-camera and a measurement method. Background With the wide application of autonomous unmanned systems in the fields of industry, traffic, security, etc., the environmental awareness becomes one of the core technologies. Conventional unmanned systems typically employ multiple camera combinations to achieve multi-view environmental awareness, such as binocular vision sensors, panoramic camera arrays, and the like. However, the multi-camera system has the problems of huge volume, high cost, complex calibration, difficult image synchronization and the like, and is difficult to meet the requirements of miniaturization and integration. Particularly in terms of wide-field dual-view perception, the prior art generally employs a physical arrangement of multiple cameras, resulting in a system that is bulky, heavy, and requires complex synchronization mechanisms and data transmission systems. In addition, the multi-camera system needs to frequently adjust the camera structure and carry out complex calibration under different working distances, and the complexity and the use difficulty of the system are increased. In recent years, virtual visual sensing based on a catadioptric mirror becomes a research hot spot, and double-view sensing is realized by using a single camera through an optical catadioptric principle. However, the existing virtual vision sensor still has the defects in the aspects of view field range, measurement precision and calibration convenience, and is difficult to meet the three-dimensional measurement requirements of wide view field and high precision. Therefore, development of a wide-view-field double-view-angle sensing sensor with small volume, low cost and simple calibration is needed to meet the requirement of an autonomous unmanned system for high-precision three-dimensional measurement of a target in real-time sensing and positioning in a complex environment. Disclosure of Invention The invention aims to solve the problems of large volume, high cost, complex calibration, difficult image synchronization and the like in wide-view-field multi-view-angle sensing of the existing unmanned system, and provides a binocular vision sensor of a double-view-angle single-camera and a measuring method. The technical scheme of the invention is that the binocular vision sensor of the double-view single-camera and the measuring method are characterized in that: 1. The invention provides a binocular vision sensor of a double-view single-camera, which consists of a plane folding mirror group (1), an industrial camera (2) and a sensor bracket (3). The plane folding mirror group (1) comprises a plane mirror A (5), a plane mirror B (6), a triangular prism (7) and a mirror group supporting structure (10), wherein the triangular prism (7) is an isosceles triangular prism, two side surfaces of the triangular prism are mirror surfaces, namely a triangular prism A surface (8) and a triangular prism B surface (9), and the triangular prism A surface and the triangular prism B surface are connected and fixed through the mirror group supporting structure (10). The industrial camera (2) forms a virtual camera A (13) through mirror image of a plane mirror A (5), forms a virtual camera B (14) through reflection of a triangular prism A surface (8), a virtual binocular unit A (11) is composed of the virtual camera A (13) and the virtual camera B (14), optical axes of the two virtual cameras are parallel and have equal-sized field angles so as to ensure that the overlapping area of a public field of view is constant, the virtual binocular unit A (11) performs three-dimensional measurement on a target object in the field of view A (20), the virtual binocular unit B (12) is composed of a virtual camera C (15) and a virtual camera D (16), the structure of the virtual binocular unit B is identical to that of the virtual binocular unit A (11) and is installed in the opposite direction, and the virtual binocular unit B (12) performs three-dimensional measurement on the target object in the field of view B (21). The virtual binocular unit A (11) and the virtual binocular unit B (12) are integrally installed through the sensor bracket (3), the focal length of the industrial camera (2) is 5-12 mm, the vertical view angles of the view field A (20) and the view field B (21) are 15-20 degrees, the horizontal view angles are 50-75 degrees, and the sum of the horizontal view angles of the two view angles is 100-150 degrees, so that a wide view field range is formed. 2. The invention provides a measuring method for three-dimensional measurement by using the binocular vision sensor of the double-view single-camera, which comprises the following steps: setting a calibration target A (17), a calibration targe