JP-7855886-B2 - Display terminal, communication system, display method, communication method, and program

JP7855886B2JP 7855886 B2JP7855886 B2JP 7855886B2JP-7855886-B2

Inventors

澤田恭也
安中英邦
齊藤大輔
本間毅史

Assignees

株式会社リコー

Dates

Publication Date: 20260511
Application Date: 20220323

Claims (13)

A display terminal that plays and displays recorded videos when they are delivered to a communication terminal, A receiving unit that receives and displays a wide-field image having a wide field of view related to the recorded video, Based on the reception by the reception unit, a display control unit causes the display unit to reproduce and display the predetermined region image, which is the predetermined region in the wide-field image, based on viewpoint information for identifying a predetermined region in the wide-field image that was displayed by a predetermined communication terminal during the distribution. It has, The display control unit displays thumbnails of multiple predetermined region images, which are multiple predetermined regions of the wide-field image that were displayed by each of the multiple communication terminals. The receiving unit receives the selection of at least one of the thumbnails of the plurality of predetermined region images. The display terminal is characterized in that the display control unit reproduces and displays at least one predetermined region image relating to a thumbnail received by the reception unit as the predetermined region image which is a predetermined region in the wide-field image .
The display terminal according to claim 1, characterized in that the display control unit reproduces and displays the predetermined region image based on the viewpoint information transmitted by the predetermined communication terminal.
A display terminal according to claim 1 or 2, The reception unit receives screen operations on the predetermined area image that is being displayed during playback. The display terminal is characterized in that the display control unit moves the virtual viewpoint over the wide-field image based on the screen operation received by the reception unit, and then changes and displays the predetermined region image based on the modified viewpoint information corresponding to the viewpoint after the move.
The display terminal according to claim 3, characterized in that the display control unit reproduces and displays, on a single screen, the first predetermined region image, which is a first predetermined region of the wide-field image displayed by the first communication terminal, and the second predetermined region image, which is a second predetermined region of the wide-field image displayed by the second communication terminal.
The display terminal according to claim 4, characterized in that it plays back and displays the first predetermined region image and the second predetermined region image of the wide-field image relating to the video at the same playback elapsed time.
The display terminal according to claim 4 or 5, characterized in that the display control unit displays one of the first predetermined region image and the second predetermined region image at a larger size than the other.
The display terminal according to any one of claims 4 to 6, characterized in that the display control unit, in accordance with the content of the screen operation received by the reception unit, returns the first predetermined region image and the second predetermined region image to the original predetermined region image indicated by the viewpoint information, plays it back, plays it back, plays it in reverse, or pauses the playback display.
The display terminal according to any one of claims 1 to 7 , characterized in that the wide-field image has a viewing angle wider than the display range that can be displayed on the display unit at one time.
The display terminal according to claim 8 , characterized in that the wide-field image is an equirectangular full-sphere image, omnidirectional image, hemispherical image, three-dimensional panoramic image, two-dimensional panoramic image, or VR image.
A communication system comprising: an information processing system for distributing video; a communication terminal for receiving and displaying the distributed video; and a display terminal for playing back and displaying the video that was recorded when it was distributed, The aforementioned communication terminal is The system receives a video recorded when it is distributed from the aforementioned information processing system, and displays a predetermined region image, which is a predetermined area in a wide-field image having a wide field of view related to the video. The information processing system is provided with viewpoint information for identifying the predetermined area. The aforementioned information processing system is The viewpoint information is transmitted to the display terminal. The aforementioned display terminal is A receiving unit that receives and displays a wide-field image having a wide field of view related to the recorded video, Based on the viewpoint information, a display control unit plays back and displays a predetermined region image, which is a predetermined region of the wide-field image that was displayed by the communication terminal at the time of distribution , on a display unit. It has, The display control unit displays thumbnails of multiple predetermined region images, which are multiple predetermined regions of the wide-field image that were displayed by each of the multiple communication terminals. The receiving unit receives the selection of at least one of the thumbnails of the plurality of predetermined region images. The communication system is characterized in that the display control unit reproduces and displays at least one predetermined region image relating to a thumbnail received by the reception unit as the predetermined region image which is a predetermined region in the wide-field image .
A display method performed by a display terminal that plays back and displays a video that was recorded when it was delivered to a communication terminal, The aforementioned display terminal is A reception process for receiving and displaying a wide-field image having a wide field of view related to the recorded video, Based on the reception, a display control process is performed to reproduce and display the predetermined region image, which is the predetermined region in the wide-field image, on the display unit, based on viewpoint information for identifying a predetermined region in the wide-field image that was displayed by a predetermined communication terminal during the distribution. Execute, The display control process includes a process for displaying thumbnails of multiple predetermined region images, which are multiple predetermined regions of the wide-field image that were displayed by each of the multiple communication terminals. The reception process includes a process of receiving the selection of at least one thumbnail from the plurality of predetermined region images. The display method is characterized in that the display control process includes a process of reproducing and displaying at least one predetermined region image relating to a thumbnail received by the reception process as the predetermined region image which is a predetermined region in the wide-field image .
A communication method performed by a communication system having an information processing system for distributing video, a communication terminal for receiving and displaying the distributed video, and a display terminal for playing back and displaying the video that was recorded when it was distributed, The aforementioned communication terminal is The system receives a video recorded when it is distributed from the aforementioned information processing system, and displays a predetermined region image, which is a predetermined area in a wide-field image having a wide field of view related to the video. The information processing system is provided with viewpoint information for identifying the predetermined area. The aforementioned information processing system is The viewpoint information is transmitted to the display terminal. The aforementioned display terminal is A reception process for receiving and displaying a wide-field image having a wide field of view related to the recorded video, Based on the viewpoint information, a display control process is performed to play back and display on the display unit a predetermined region image, which is a predetermined region of the wide-field image that was displayed by the communication terminal at the time of distribution , from the recorded wide-field image. Execute, The display control process includes a process for displaying thumbnails of multiple predetermined region images, which are multiple predetermined regions of the wide-field image that were displayed by each of the multiple communication terminals. The reception process includes a process of receiving the selection of at least one thumbnail from the plurality of predetermined region images. The communication method is characterized in that the display control process includes a process of reproducing and displaying at least one predetermined region image relating to a thumbnail received by the reception process as the predetermined region image which is a predetermined region in the wide-field image .
When a video is recorded and delivered to a communication terminal, the computer that plays and displays the video will have the following settings: A reception process for receiving and displaying a wide-field image having a wide field of view related to the recorded video, Based on the reception, a display control process is performed to reproduce and display the predetermined region image, which is the predetermined region in the wide-field image, on the display unit, based on viewpoint information for identifying a predetermined region in the wide-field image that was displayed by a predetermined communication terminal during the distribution. Make it run, The display control process includes a process for displaying thumbnails of multiple predetermined region images, which are multiple predetermined regions of the wide-field image that were displayed by each of the multiple communication terminals. The reception process includes a process of receiving the selection of at least one thumbnail from the plurality of predetermined region images. The program is characterized in that the display control process includes a process of reproducing and displaying at least one predetermined region image relating to a thumbnail received by the reception process as the predetermined region image which is a predetermined region in the wide-field image .

Description

This disclosure relates to display terminals, information processing systems, communication systems, display methods, information processing methods, communication methods, and programs. There is a communication system that transmits images and audio in real time from one location to one or more other locations, enabling remote communication between users in different locations using images and audio. As for the images used, a wide field of view image (hereinafter referred to as "wide-field image") is known, which captures a wide area including parts that cannot be seen with a normal field of view, such as a 360-degree image (also called a 360-degree spherical image, omnidirectional image, or all-around image). Users can change their virtual viewpoint within a predetermined area of the wide-field image displayed on the communication terminal's screen by operating the communication terminal, thereby viewing a different predetermined area within the wide-field image. Furthermore, a technique has been disclosed for extracting a more suitable predetermined region from a wide-field image without requiring complicated operations (Patent Document 1). Furthermore, when wide-field images related to video are distributed to designated locations, users at those locations can independently modify and view their desired areas within the wide-field images. Additionally, if wide-field images related to video footage taken at a work site, etc., are recorded when they are distributed to the designated locations, users can later play back and view the recorded wide-field images. If users viewing the recording afterward can determine which areas of the wide-field image were displayed on the designated communication terminal at the designated location during recording, they can understand which areas were attracting attention at that location, which can then be used as a reference for subsequent thinking and actions. This figure illustrates an example of remote communication using wide-field imaging.This is an example of a schematic diagram of a communication system configuration.This is an example of a hardware configuration diagram for an imaging device.This is an example of a hardware configuration diagram for a communication terminal and an information processing system.(a) is a left side view of the imaging device, (b) is a front view of the imaging device, and (c) is a top view of the imaging device.This is a diagram illustrating the use of the imaging device.(a) is a hemispherical image (front) captured by the imaging device, (b) is a hemispherical image (back) captured by the imaging device, and (c) is an image represented by equirectangular projection.(a) A conceptual diagram showing the state of covering the sphere with an equirectangular projection image, and (b) A diagram showing a full-sphere image.This diagram shows the positions of a virtual camera and a predetermined region when a 360-degree image is treated as a three-dimensional sphere.(a) is a stereoscopic perspective view of Figure 5, (b) is a diagram showing the predetermined region image in the state of (a) on the display, (c) is a diagram showing the predetermined region after changing the viewpoint of the virtual camera IC in (a), and (d) is a diagram showing the predetermined region image in the state of (c) on the display.This figure shows the relationship between information about a predetermined region and an image of a predetermined region T.This is a diagram showing a point in three-dimensional Euclidean space using spherical coordinates.This is an example of a functional configuration diagram for a communication system.(a) and (b) are conceptual diagrams showing the image management information stored in the image management information storage unit.(a) A conceptual diagram showing virtual room information stored in the virtual room information storage unit, and (b) A conceptual diagram showing tenant information stored in the tenant information storage unit.This is a conceptual diagram showing the viewpoint-related information stored in the viewpoint-related information memory unit.(a) is a diagram showing an example of the room entry screen. (b) is a diagram showing an example of the image viewing screen displayed by the communication terminal when a user enters a virtual room.This is an example of a sequence diagram illustrating the process of a user (or communication terminal) entering a virtual room.This figure shows an example of a device registration screen displayed by a communication terminal.(a) is an example of a diagram showing the imaging device registration dialog. (b) is a diagram showing an example of a two-dimensional code screen.This figure shows an example of the VR goggles registration screen that appears when the VR goggles registration button is pressed.This figure shows an example of a virtual room mapping screen (part 1) for associating imaging devices with virtual rooms.This figure shows an example of the virtual room map