JP-7854913-B2 - Display system

JP7854913B2JP 7854913 B2JP7854913 B2JP 7854913B2JP-7854913-B2

Inventors

久保（葛谷）麻未
石塚博基
海野和也
山田貴洋

Assignees

株式会社東海理化電機製作所

Dates

Publication Date: 20260507
Application Date: 20220930

Claims (11)

A surrounding imaging unit that captures images of the area around a moving object to obtain an image of the area around the moving object, A lower imaging unit that photographs the lower side of the moving object and acquires a bottom image of the moving object, A mobile body includes a composite image generation unit that generates a composite image by combining an image of the mobile body's surroundings acquired by the surrounding imaging unit and an image of the mobile body's bottom surface acquired by the lower imaging unit, A display device, including a display unit, is mounted on an externally located remote monitoring device, A communication unit is included in the aforementioned mobile body and the aforementioned display device, respectively, which transmits or receives image data. Equipped with, The communication unit on the mobile side includes a transmission unit that transmits the composite image generated by the composite image generation unit, The communication unit on the display device side includes a receiving unit that receives the composite image transmitted from the transmitting unit, In the automatically operated mobile body , when the abnormal driving mode or the recovery driving mode, in which the mobile body starts driving after stopping, is selected, the display image generated from the composite image is displayed on the display device . A display system in which, when the abnormal driving mode is selected, the display image generated based on notification information from the automatic driving system and information detected by sensing by sensors mounted on the mobile body is displayed, and when the recovery driving mode is selected, the display image in which the lower side of the mobile body is visible is displayed .
A surrounding imaging unit that captures images of the area around a moving object to obtain an image of the area around the moving object, A mobile body including a lower imaging unit that photographs the lower side of the mobile body and acquires a bottom image of the mobile body, A display device including a display unit, A composite image generation unit is included in the moving body or the display device, which generates a composite image by converting the moving body's surrounding image acquired by the surrounding imaging unit and the moving body's bottom surface image acquired by the lower imaging unit into a planar 360-degree spherical image. A communication unit is included in the aforementioned mobile body and the aforementioned display device, respectively, which transmits or receives image data. Equipped with , The aforementioned composite image is a display system in which the image includes information in all directions that can be seen from the moving object which is the shooting location .
A surrounding imaging unit captures images of the entire surroundings of a moving object to acquire an image of the area around the moving object, A mobile body including a lower imaging unit that photographs the lower side of the mobile body and acquires a bottom image of the mobile body, A display device including a display unit, A composite image generation unit is included in the moving body or the display device, which generates a composite image by combining an image of the moving body's surroundings acquired by the surrounding imaging unit and an image of the moving body's bottom surface acquired by the lower imaging unit. A communication unit is included in the aforementioned mobile body and the aforementioned display device, respectively, which transmits or receives image data. Equipped with, The aforementioned images of the surroundings of the moving body and the aforementioned images of the bottom surface of the moving body include a portion of the moving body . The composite image generation unit is a display system that generates a composite image by overlapping the overlapping portions of adjacent images of the surroundings of the moving object and the bottom surface of the moving object .
The moving body includes the composite image generation unit, The communication unit on the mobile side includes a transmission unit that transmits the composite image generated by the composite image generation unit, The display system according to claim 2 or 3, wherein the communication unit on the display device side includes a receiving unit that receives the composite image transmitted from the transmitting unit.
The communication unit on the mobile body side includes a transmission unit that transmits the image of the mobile body's surroundings acquired by the surrounding imaging unit and the image of the mobile body's bottom surface acquired by the lower imaging unit. The communication unit on the display device side includes a receiving unit that receives the image of the surroundings of the moving object and the image of the bottom surface of the moving object transmitted from the transmitting unit. The display system according to claim 2 or 3, wherein the display device includes the composite image generation unit.
When the surrounding imaging unit acquires multiple images of the surroundings of the moving object, The display system according to any one of claims 1 to 3, wherein the composite image generation unit composites all adjacent images of the surroundings of the moving object and the bottom surface images of the moving object by overlapping them.
When the surrounding imaging unit acquires multiple images of the moving object's surroundings, and the lower imaging unit acquires one image of the moving object's bottom surface, The display system according to any one of claims 1 to 3, wherein the composite image generation unit composites the bottom surface image of the moving body by overlapping each of the surrounding images of the moving body.
When the surrounding imaging unit acquires multiple images of the surroundings of the moving object, and the lower imaging unit acquires two images of the bottom surface of the moving object, The display system according to any one of claims 1 to 3, wherein the composite image generation unit composites a plurality of images surrounding the moving body by overlapping each of them with at least one of the images of the bottom surface of the moving body.
When the lower imaging unit acquires multiple images of the bottom surface of the moving body, The display system according to any one of claims 1 to 3, wherein the composite image generation unit composites adjacent bottom surface images of the moving body by overlapping them.
The aforementioned display device is A display system according to any one of claims 1 to 3, comprising a display image generation unit that generates a display image to be displayed on the display unit from the composite image.
The display system according to claim 1 or claim 2, wherein the image of the surrounding area of the moving body and the image of the bottom surface of the moving body include a part of the moving body.

Description

This invention relates to a display system. Patent Document 1 discloses a vehicle-below-the-vehicle image display control device for displaying images based on images taken directly below the vehicle's underside on a display device. In this vehicle-below-the-vehicle image display control device, a bird's-eye view image of the vehicle's surroundings is generated using rearward-facing images captured by a rearward-facing camera, and a bird's-eye view image of the area directly below the vehicle is generated using downward-facing images captured by a downward-facing camera. Japanese Patent Publication No. 2006-279752 This is a block diagram showing the hardware configuration of a display system according to the first embodiment of the present invention.This is a top view showing the positions where cameras that photograph the area around the vehicle will be placed.This figure shows an example of an image of the area around a vehicle taken by a camera.This is a left side view showing the location where the camera that films the underside of the vehicle is positioned.This is an explanatory diagram for explaining the image synthesis method.This is an explanatory diagram for describing equirectangular composite images.This is an explanatory diagram illustrating the conversion from equirectangular to spherical format in a 360-degree spherical image.This figure shows an example of a display image shown on the display unit.This figure shows an example of a display image shown on the display unit.This figure shows an example of a display image shown on the display unit.This figure shows an example of a display image shown on the display unit.This figure shows an example of a display image shown on the display unit.This figure shows an example of a display image shown on the display unit.This is an explanatory diagram illustrating the method of combining an image of the vehicle's surroundings with two images of the vehicle's underside.This is a block diagram showing the hardware configuration of a display system according to a second embodiment of the present invention. The display system 10 according to the first embodiment of the present invention will be described using Figures 1 to 13. In the following description, when the directions of front, rear, left, right, up, and down are indicated, they refer to the front, rear, left, right, up, and down directions of the vehicle. Figure 1 is a block diagram showing the hardware configuration of the display system 10 according to the first embodiment of the present invention. As shown in Figure 1, the display system 10 of this embodiment includes a vehicle 20 as a mobile body and a display device 30 mounted on the vehicle 20. In this embodiment, the vehicle 20 is configured to be switchable between manual and automatic driving, and when the occupant manually drives the vehicle 20, known driving assistance functions are provided to support the occupant's driving operations. As shown in Figure 1, the vehicle 20 includes a surrounding imaging unit 22, a downward camera 23 as a downward imaging unit, an image processing device 24, a vehicle control device 26, and a communication unit 28. In this embodiment, the surrounding imaging unit 22 includes, as an example, eight cameras: a front camera 22A, a right front camera 22B, a right side camera 22C, a right rear camera 22D, a rear camera 22E, a left rear camera 22F, a left side camera 22G, and a left front camera 22H. These eight cameras 22A to 22H and the lower camera 23 each utilize, as an example, a circular fisheye lens with a field of view greater than 180 degrees. Images captured by a circular fisheye lens are fisheye images, characterized by subjects near the center of the shooting range appearing large, while subjects at the edges of the shooting range appear small. The shooting range is represented by a circular image. In this embodiment, the "captured image" may be a still image or a video. Figure 2 is a top view showing the positions of eight cameras 22A to 22H, which capture images around the vehicle 20, and Figure 3 is a diagram showing an example of vehicle surrounding images P1 to P8 captured by the eight cameras 22A to 22H. As shown in Figures 2 and 3, the front camera 22A is mounted on the front of the vehicle 20, and the vehicle surrounding image P1 captured by the front camera 22A shows the front of the vehicle 20. The right front camera 22B is mounted on the right front of the vehicle 20, and the vehicle surrounding image P2 captured by the right front camera 22B shows the right front of the vehicle 20. The right side camera 22C is mounted on the right side mirror of the vehicle 20, and the vehicle surrounding image P3 captured by the right side camera 22C shows the right side of the vehicle 20. The right rear camera 22D is mounted on the right rear of the vehicle 20, and the vehicle surrounding image P4 captured by the right rear camera 22D shows the right rear of the vehicle 20. The rear camera 22E is mount