JP-7855742-B2 - Video display device and video display method

JP7855742B2JP 7855742 B2JP7855742 B2JP 7855742B2JP-7855742-B2

Inventors

中出眞弓
伊藤保
橋本康宣

Assignees

マクセル株式会社

Dates

Publication Date: 20260508
Application Date: 20250109

Claims (10)

In a video display device carried by a user that displays the user's movement trajectory as video, A sensor that detects the position and orientation of the user carrying the video display device, A trajectory information acquisition unit acquires information on the user's movement trajectory from the detection results of the aforementioned sensor, A storage unit that stores the user's trajectory information acquired by the trajectory information acquisition unit and information about an avatar, which is a virtual image representing the user. A display unit that displays the user's movement trajectory using the avatar, The system comprises a trajectory information acquisition unit and a control unit that controls the display unit, The control unit, The system generates the avatar from the avatar information stored in the memory unit, and uses the sensor to determine the current user's field of view. Based on the user's trajectory information stored in the memory unit, the generated avatar is positioned on the user's movement trajectory to match the user's current field of view and displayed on the display unit. A video display device characterized in that, when moving the avatar along the user's movement trajectory, the orientation of the avatar's body when it is moving is displayed in the opposite direction to the avatar's direction of movement, and the avatar is moved from newer trajectory information to older trajectory information in time.
In the video display device according to claim 1, The trajectory information acquisition unit acquires information on the user's movement trajectory at predetermined unit time intervals and stores the position coordinates of each trajectory point in the storage unit. The control unit reads the position coordinates of each trajectory point from the storage unit at intervals of the unit time, or at intervals obtained by multiplying the unit time by an arbitrary coefficient, and places the avatar at those position coordinates.
In the video display device according to claim 2, The trajectory information acquisition unit stores the position coordinates of each trajectory point in the storage unit as the difference between the position coordinates of the trajectory point immediately preceding it in time, and stores the position coordinates of at least the endpoint trajectory point in the storage unit as absolute values.
In the video display device according to claim 1, The image display device is characterized in that, when the control unit cannot place the avatar within the user's current field of view, it performs guidance processing to cause the user to change their field of view or move to a position where the avatar is visible.
In the video display device according to claim 2, Furthermore, it is equipped with an imaging unit that captures the exterior scenery. The imaging data captured by the imaging unit is stored in the storage unit in association with the shooting point, which is the shooting location. The control unit is characterized in that when the position of the avatar is at the shooting point, it displays the image of the shooting data stored in the storage unit on the display unit.
In the video display device according to claim 5, The control unit stores in the storage unit the video footage captured at the point where the direction of the user's movement trajectory changed, from the captured data captured by the imaging unit. The control unit is characterized in that when the position of the avatar is at a shooting point where the direction of the user's movement trajectory changes, it displays the image of the shooting data stored in the storage unit on the display unit.
In the video display device according to claim 2, Furthermore, it is equipped with an imaging unit that captures the exterior scenery. The display unit displays the current video being captured by the imaging unit, The control unit adjusts the image of the exterior view displayed on the display unit accordingly. A video display device characterized by superimposing and displaying the avatar based on the user's movement trajectory.
In a video display system that connects multiple video display devices carried by a user and displays the user's movement trajectory as a video on the video display devices, Each of the aforementioned video display devices is: A sensor that detects the position and orientation of the user carrying the video display device, A trajectory information acquisition unit acquires information on the user's movement trajectory from the detection results of the aforementioned sensor, A storage unit that stores the user's trajectory information acquired by the aforementioned trajectory information acquisition unit and the information of the avatar, which is a virtual image representing the user. A display unit that shows the user's movement trajectory using an avatar, The system comprises a trajectory information acquisition unit and a control unit for controlling the display unit, The control unit, An avatar is generated from the avatar information stored in the memory unit, and the current field of view of the user is determined by the sensor. The user's trajectory information stored in the memory unit and the generated avatar are placed on the user's movement trajectory, aligned with the user's current field of view, and displayed on the display unit. A video display system characterized in that, when moving the avatar along the user's movement trajectory, the orientation of the avatar's body when it is moving is displayed in the opposite direction to the avatar's direction of movement, and the avatar is moved from newer trajectory information to older trajectory information in time.
In a video display system that connects multiple video display devices carried by multiple users and displays the movement trajectories of the multiple users as video on the multiple video display devices, Each of the aforementioned multiple video display devices is: A sensor that detects the position and orientation of the user carrying the video display device, A trajectory information acquisition unit acquires information on the user's movement trajectory from the detection results of the aforementioned sensor, A storage unit that stores the user's trajectory information acquired by the aforementioned trajectory information acquisition unit, and the information of each avatar, which is a virtual image representing the user and other users. A communication processing unit that sends and receives trajectory information of each user with other video display devices carried by other users, A display unit that shows the movement trajectory of each user with their respective avatar, The system comprises the trajectory information acquisition unit, the communication processing unit, and the control unit for controlling the display unit, The control unit, The system generates avatars for each user from the avatar information stored in the memory unit, and uses the sensor to determine the current field of view of the user. Based on the user's trajectory information stored in the memory unit and the trajectory information of other users received by the communication processing unit, the generated avatars are placed on the movement trajectories of the user and other users, aligned with the current field of view of the user, and displayed on the display unit. A video display system characterized in that, when moving the avatar along the user's movement trajectory, the orientation of the avatar's body when it is moving is displayed in the opposite direction to the avatar's direction of movement, and the avatar is moved from newer trajectory information to older trajectory information in time.
In a video display method that shows the user's movement trajectory in video, The steps include detecting the user's position and direction and obtaining information on the user's movement trajectory, The steps include storing the acquired user trajectory information in a storage unit, The steps include generating an avatar, which is a virtual image representing the user, The steps include determining the current field of view of the user, Based on the user's trajectory information stored in the memory unit, the generated avatar is positioned on the user's movement trajectory to match the user's current field of view and displayed on the display unit. A video display method characterized by the step of moving the avatar along the user's movement trajectory, displaying the avatar's body orientation in the opposite direction to the avatar's direction of movement, and moving from newer trajectory information to older trajectory information in time.

Description

This invention relates to a portable video display device and a video display method. In recent years, portable video display devices, such as smartphones, have become widespread. Among these, head-mounted displays (HMDs), worn on the user's head, display images of the real world and augmented reality (AR) images created by a computer or other means superimposed on a glasses-type display screen. Furthermore, by attaching sensors to the HMD, information acquired by the sensors can be displayed on the screen as AR images. For example, Patent Document 1 discloses a configuration for a portable terminal that depicts a user's behavioral history, comprising: terminal information acquisition means for acquiring the terminal's location information; camera means for generating camera images of the surroundings; depiction history calculation means for calculating behavioral history depiction information (avatar) to be displayed based on previously acquired behavioral history and the camera's imaging range; image synthesis means for generating a composite image in which the behavioral history depiction information is depicted on the camera image; and display means for displaying the composite image. International Publication No. 2011/093031 External view of the HMD according to Example 1.A block diagram showing the internal structure of the HMD.A diagram showing an example of a connection configuration between the communication processing unit and external devices.A diagram showing the configuration of the functional blocks of an HMD.A schematic diagram illustrating an example of user trajectory collection.A flowchart illustrating the trajectory collection process.A table for saving location information and trajectory information.A differential trajectory information storage table that stores location information as differential values.A start/end point coordinate table that stores the position information at the beginning and end of the session.A table for storing still image data.A video data storage table for saving videos.A flowchart illustrating the entire avatar display process.A flowchart showing the relief process in Figure 9A.A diagram showing an example of a direction change guide display.A diagram showing an example of a direction of movement guide display.A diagram showing an example of how an avatar is displayed.A diagram showing an example of how the avatar looks when it has changed its orientation.A table showing examples of voice commands used to operate the HMD.A diagram illustrating user finger gestures for operation instructions.A two-dimensional trajectory information storage table for storing position information in Example 2.A table of 2D start and end point coordinates at the beginning and end of the day.A schematic diagram of the avatar display in Example 3.A schematic diagram of the avatar display in Example 4.Trajectory information storage table in Example 7.A diagram showing an example of how an avatar is displayed.A diagram showing the external appearance of the smartphone in Example 8.A diagram showing the configuration of a video display system with multiple HMDs connected according to Example 9.A schematic diagram showing examples of how multiple avatars are displayed. The embodiments of the present invention will be described below, focusing primarily on an example of a head-mounted display (HMD), which is a head-worn image display device.