BR-102025012161-A2 - METHOD AND SYSTEM FOR VERTICAL IMAGE STITCHING

BR102025012161A2BR 102025012161 A2BR102025012161 A2BR 102025012161A2BR-102025012161-A2

Abstract

The present invention relates to a method for a camera monitoring system (CMS), which includes: obtaining a first image from a first camera and a second image from a second camera. The first image and the second image show a side view of a commercial vehicle and its surrounding environment. The first camera and the second camera have different, overlapping fields of view and have different respective optical axes, which intersect on a ground plane at different respective optical angles. The method also includes: performing a perspective transformation on at least one of the first and second images to obtain an updated image set, such that at least one of the first and second images is updated in the updated image set; and vertically stitching the images of the updated image set together to form a combined image. A camera monitoring system (CMS) is also described.

Inventors

Song Wang
Utkarsh SHARMA
Liang Ma
Troy Cooprider

Assignees

Stoneridge Electronics, AB

Dates

Publication Date: 20260310
Application Date: 20250613
Priority Date: 20240701

Claims (20)

1. A method for a camera monitoring system (CMS), characterized in that it comprises: obtaining a first image from a first camera and a second image from a second camera, the first and second images showing a side view of a commercial vehicle and its surrounding environment, the first and second cameras having different fields of view, superimposed, and having different respective optical axes, which intersect on a ground plane at different respective optical angles; performing a perspective transformation on at least one of the first and second images to obtain an updated image set, such that at least one of the first and second images is updated in the updated image set; vertically stitching the images of the updated image set together to form a combined image.
2. Method according to claim 1, characterized in that: the respective different optical angles include a first optical angle of the first camera and a second optical angle of the second camera; the first optical angle is less than 90°; and the second optical angle is equal to or less than 90° and is greater than the first optical angle.
3. A method according to claim 2, characterized in that the first camera has a first focal length, and the second camera has a second focal length, which is shorter than the first focal length.
4. Method according to claim 1, characterized in that the first image is shown above the second image.
5. Method, according to claim 1, characterized in that said execution of a perspective transformation is performed for both the first image and the second image, so that both the first image and the second image are updated in the updated image set.
6. Method, according to claim 1, characterized in that it comprises, before said execution of the perspective transformation, performing at least one of: a distortion correction for the first image, from the first camera, to attenuate the image distortion caused by a lens or sensor of the first camera; and a distortion correction for the second image, from the second camera, to attenuate the image distortion caused by a lens or sensor of the second camera.
7. A method according to claim 1, characterized in that it comprises, before said execution of the perspective transformation, performing both: a distortion correction for the first image, from the first camera, to attenuate the image distortion caused by a lens or sensor of the first camera; and a distortion correction for the second image, from the second camera, to attenuate the image distortion caused by a lens or sensor of the second camera.
8. Method, according to claim 1, characterized in that it comprises, prior to said vertical stitching: performing at least one cutout and enlargement of at least one of the first image and the second image.
9. Method according to claim 1, characterized in that said vertical stitching of the images of the updated image set to form a combined image comprises: mapping several of the first points of the first image, in the updated image set, to several second points in the second image, in the updated image set; aligning the first image, in the updated image set, and the second image, in the updated image set, based on the mapping; and combining the first image, in the updated image set, and the second image, in the updated image set, to form the combined image.
10. Method according to claim 1, characterized in that it comprises: displaying the combined image on an electronic display in the commercial vehicle in a first mode; displaying at least one of the first and second images separately without combination in a second mode; and switching between the first mode and the second mode in response to receiving a switch command from an occupant of the commercial vehicle.
11. Camera monitoring system (CMS), characterized by comprising: a first camera and a second camera, both configured to record respective images of a side view of a commercial vehicle and its surrounding environment, wherein the first camera and the second camera have different overlapping fields of view and different respective optical axes that intersect at different respective optical angles; and a set of processing circuits, operationally connected to memory and configured to: obtain a first image from the first camera and a second image from the second camera; perform a perspective transformation on at least one of the first and second images to obtain an updated image set, such that at least one of the first and second images is updated in the updated image set; vertically stitch the images of the updated image set together to form a combined image.
12. CMS, according to claim 11, characterized in that: the different respective optical angles comprise a first optical angle of the first camera and a second optical angle of the second camera; the first optical angle is less than 90°; and the second optical angle is equal to or less than 90° and is greater than the first optical angle.
13. CMS, according to claim 12, characterized in that the first camera has a first focal length, and the second camera has a second focal length, which is shorter than the first focal length.
14. CMS, according to claim 11, characterized in that, in the combined image, the first image is shown below the second image.
15. CMS, according to claim 11, characterized in that the processing circuitry is configured to perform perspective transformation for both the first and second images, such that both the first and second images are updated in the updated image set.
16. CMS, according to claim 11, characterized in that the processing circuitry is configured to, before performing the perspective transformation, perform at least one of: a distortion correction for the first image, from the first camera, to mitigate image distortion caused by a lens or sensor of the first camera; and a distortion correction for the second image, from the second camera, to mitigate image distortion caused by a lens or sensor of the second camera.
17. CMS, according to claim 11, characterized in that the processing circuitry is configured to, before performing the perspective transformation, perform both: a distortion correction for the first image, from the first camera, to mitigate image distortion caused by a lens or sensor of the first camera; and a distortion correction for the second image, from the second camera, to mitigate image distortion caused by a lens or sensor of the second camera.
18. CMS, according to claim 11, characterized in that the processing circuitry is configured to, before vertical stitching: perform at least one cropping and one enlargement of at least one of the first and second images.
19. CMS, according to claim 11, characterized in that to vertically stitch the images of the updated image set, the processing circuitry is configured to: map several of the first points of the first image, in the updated image set, to several of the second points of the second image, in the updated image set; align the first image, in the updated image set, and the second image, in the updated image set, based on the mapping; and combine the first image, in the updated image set, and the second image, in the updated image set, to form the combined image.
20. CMS, according to claim 11, characterized in that the processing circuitry is configured to: display the combined image on an electronic display in the commercial vehicle in a first mode; display at least one of the first and second images separately without combination in a second mode; and switch between the first mode and the second mode in response to receiving a switch command from an occupant of the commercial vehicle.

Description

TECHNICAL FIELD [001] The present invention relates to a camera monitoring system (CMS) and, more particularly, to a method and a system for providing vertical image stitching in a CMS. BACKGROUND [002] Vehicle camera systems for mirror replacement or to supplement mirror views are used in commercial vehicles to improve a vehicle operator's ability to see the surrounding environment of the commercial vehicle. Camera monitoring systems (CMSs) use one or more cameras to provide a better field of view for a vehicle operator. In some examples, mirror replacement systems cover a wider field of view than a conventional mirror, or include views that are not entirely obtainable through a conventional mirror. When images are provided to a vehicle occupant from multiple cameras, it can be difficult for the driver to efficiently analyze the images. SUMMARY [003] A method for a camera monitoring system (CMS), according to an exemplary embodiment of the present invention, includes obtaining a first image from a first camera and a second image from a second camera. The first image and the second image show a side view of a commercial vehicle and its surrounding environment. The first camera and the second camera have different, overlapping fields of view and have different respective optical axes, which intersect on a ground plane at different respective optical angles. The method also includes: performing a perspective transformation on at least one of the first and second images to obtain an updated image set, such that at least one of the first and second images is updated in the updated image set; and vertically stitching the images of the updated image set together to form a combined image. [004] In another embodiment of the previous embodiment, the respective different optical angles include a first optical angle of the first camera and a second optical angle of the second camera. The first optical angle is less than 90°. The second optical angle is equal to or less than 90° and is greater than the first optical angle. [005] In another embodiment of any of the previous embodiments, the first camera has a first focal length, and the second camera has a second focal length, which is shorter than the first focal length. The second camera is located closer to the front of the commercial vehicle than the first camera. [006] In another embodiment of any of the previous embodiments, the first image is shown above the second image. [007] In another embodiment of any of the previous embodiments, a perspective transformation is performed for both the first image and the second image, so that both the first image and the second image are updated in the updated image set. [008] In another embodiment of any of the previous embodiments, the method includes, before performing the perspective transformation, performing at least one distortion correction for the first image, from the first camera, to mitigate image distortion caused by a lens or sensor of the first camera, and a distortion correction for the second image, from the second camera, to mitigate image distortion caused by a lens or sensor of the second camera. [009] In another embodiment of any of the previous embodiments, the method includes, before performing the perspective transformation, performing both a distortion correction for the first image, from the first camera, to attenuate the image distortion caused by a lens or sensor of the first camera, and a distortion correction for the second image, from the second camera, to attenuate the image distortion caused by a lens or sensor of the second camera. [0010] In another embodiment of any of the previous embodiments, the method includes, before vertical stitching, performing at least one cutout and enlargement of at least one of the first and second images. [0011] In another embodiment of any of the preceding embodiments, the vertical stitching of the images in the updated image set to form a combined image includes: mapping multiple points from the first image, in the updated image set, to the second image, in the updated image set; aligning the first image, in the updated image set, and the second image, in the updated image set, based on the mapping; and combining the first image, in the updated image set, and the second image, in the updated image set, to form the combined image. [0012] In another embodiment of any of the preceding embodiments, the method includes: displaying the combined image on an electronic display in the commercial vehicle in a first mode; displaying at least one of the first image and the second image separately without combination in a second mode; and switching between the first mode and the second mode in response to receiving a switch command from an occupant of the commercial vehicle. [0013] A camera monitoring system (CMS), according to an exemplary embodiment of the present invention, includes: a first camera and a second camera, both configured to record respective images of a si