US-12625521-B2 - Virtual input device, virtual input system, virtual input method, and recording medium

US12625521B2US 12625521 B2US12625521 B2US 12625521B2US-12625521-B2

Abstract

Provided is a virtual input device that includes an input/output unit that acquires image data captured by a camera of a virtual reality display device and output display information to be displayed on a display of the virtual reality display device to the virtual reality display device, a video analysis unit that analyzes the image data to specify a position at which a virtual terminal is to be displayed, and a display information generation unit that generates the display information including the virtual terminal based on user information of a user wearing the virtual reality display device.

Inventors

Yoshiaki Nozawa
Daisuke Shimada
Takuya Yagyu
Shinichi Yamanaka

Assignees

NEC PLATFORMS, LTD.

Dates

Publication Date: 20260512
Application Date: 20240111
Priority Date: 20230208

Claims (10)

1 . A virtual input device comprising: a memory storing instructions; and a processor connected to the memory and configured to execute the instructions to: acquire image data captured by a camera of a virtual reality display device and output display information to be displayed on a display of the virtual reality display device to the virtual reality display device; analyze the image data to detect a physical operation terminal disposed in a real world from the image data; specify a position of the physical operation terminal on the display of the virtual reality display device determine a position of each key of the physical operation terminal based on at least one mark attached to the physical operation terminal; generate the display information in which the virtual terminal having a key arrangement customized according to user information of a user wearing the virtual reality display device is displayed at the specified position of the physical operation terminal such that the virtual terminal is superimposed on the physical operation terminal; display the generated display information including the virtual terminal superimposed on the physical operation terminal on the display of the virtual reality display device.
2 . The virtual input device according to claim 1 , wherein the processor is configured to execute the instructions to generate the display information in which a key to be operated by the user among a plurality of keys included in the virtual terminal is highlighted, according to the user information.
3 . The virtual input device according to claim 2 , wherein the processor is configured to execute the instructions to detect an operation terminal disposed in a real world from the image data, specify a position of the operation terminal on the display of the virtual reality display device, and generate the display information in which the virtual terminal is displayed at the specified position of the operation terminal.
4 . The virtual input device according to claim 1 , wherein the processor is configured to execute the instructions to analyze the image data and recognize an operation of the user, and generate the display information based on the recognized operation of the user.
5 . The virtual input device according to claim 1 , wherein the processor is configured to execute the instructions to acquire associated information based on the user information, and generate the display information in which a key associated to the associated information is highlighted.
6 . The virtual input device according to claim 1 , wherein the processor is configured to execute the instructions to acquire authentication information of the user who uses the virtual reality display device and perform authentication of the user based on the acquired authentication information, and generate the display information for the virtual reality display device used by the authenticated user.
7 . A virtual input system comprising: the virtual input device according to claim 1 ; and a virtual reality display device that includes a camera and a display, wherein the virtual reality display device is configured to output image data captured by the camera to the virtual input device, acquire display information output from the virtual input device, and display an image based on the acquired display information on the display of the virtual reality display device.
8 . The virtual input system according to claim 7 , further comprising an operation terminal that is disposed in a real world and operated by a user wearing the virtual input device, wherein the processor of the virtual input device is configured to execute the instructions to generate the display information in which a virtual terminal is displayed at a position of the operation terminal, and output the generated display information to the virtual reality display device, and the virtual reality display device is configured to display an image based on the display information on the display of the virtual reality display device.
9 . A virtual input method executed by a computer, the method comprising: acquiring image data captured by a camera of a virtual reality display device; analyzing the image data to detect a physical operation terminal disposed in a real world from the image data; specify a position of the physical operation terminal on the display of the virtual reality display device at which a virtual terminal is to be displayed; and generating display information in which the virtual terminal having a key arrangement customized according to user information of a user wearing the virtual reality display device is displayed at the specified position of the physical operation terminal such that the virtual terminal is superimposed on the physical operation terminal; displaying the generated display information including the virtual terminal superimposed on the physical operation terminal on the display of the virtual reality display device.
10 . A non-transitory recording medium recording a program for causing a computer to execute processing of: acquiring image data captured by a camera of a virtual reality display device; analyzing the image data to detect a physical operation terminal disposed in a real world from the image data; specify a position of the physical operation terminal on the display of the virtual reality display device generating display information in which the virtual terminal having a key arrangement customized according to user information of a user wearing the virtual reality display device is displayed at the specified position of the physical operation terminal such that the virtual terminal is superimposed on the physical operation terminal; displaying the generated display information including the virtual terminal superimposed on the physical operation terminal on the display of the virtual reality display device.

Description

This application is based upon and claims the benefit of priority from Japanese Patent Application No. 2023-017951, filed on Feb. 8, 2023, the disclosure of which is incorporated herein in its entirety by reference. TECHNICAL FIELD The present disclosure relates to a virtual input device, a virtual input system, a virtual input method, and a recording medium. BACKGROUND ART A virtual reality display device including a display that can be worn on a human body, such as a head mounted display (HMD), has been developed. By mounting such a device, visual information displayed on the display can be visually recognized. PTL 1 (JP 6826029 B2) discloses a method of displaying a virtual reality environment on an HMD. In the method of PTL 1, it is determined whether an opportunity exists in a virtual reality environment for user interaction with a real-world object. In the method of PTL 1, a real-world object positioned within a real-world viewpoint of a user of the head mounted display is detected in response to the determined opportunity. In the method of PTL 1, an image of the detected real-world physical object is displayed on the head mounted display. It is possible to implement a user interface that receives an operation in a virtual space displayed on the display by using the method of PTL 1. For example, a user interface in which predetermined input processing is executed can be implemented when the user wearing the HMD performs an operation on visual information constructed in the virtual space. In such a method, it is necessary to keep paying attention and concentration in order to keep visually recognizing the visual information displayed on the display of the HMD. Therefore, there is a possibility that a touch panel, a switch, or the like constructed in the virtual space is overlooked due to fatigue caused by continuation of the operation using the virtual reality display device, and an erroneous operation is performed. An object of the present disclosure is to provide a virtual input device, a virtual input system, a virtual input method, and a program capable of implementing a continuous operation on a virtual terminal displayed in a virtual space. SUMMARY A virtual input device according to an aspect of the present disclosure includes an input/output unit that acquires image data captured by a camera of a virtual reality display device and output display information to be displayed on a display of the virtual reality display device to the virtual reality display device, a video analysis unit that analyzes the image data to specify a position at which a virtual terminal is to be displayed, and a display information generation unit that generates the display information including the virtual terminal based on user information of a user who uses the virtual reality display device. A virtual input method according to an aspect of the present disclosure includes acquiring image data captured by a camera of a virtual reality display device, analyzing the image data to specify a position at which a virtual terminal is to be displayed, generating display information including the virtual terminal based on user information of a user who uses the virtual reality display device, and outputting the generated display information to the virtual reality display device to display an image based on the display information on a display of the virtual reality display device. A program according to an aspect of the present disclosure causes a computer to execute processing of acquiring image data captured by a camera of a virtual reality display device, analyzing the image data to specify a position at which a virtual terminal is to be displayed, generating display information including the virtual terminal based on user information of a user who uses the virtual reality display device, and outputting the generated display information to the virtual reality display device to display an image based on the display information on a display of the virtual reality display device. BRIEF DESCRIPTION OF THE DRAWINGS Exemplary features and advantages of the present invention will become apparent from the following detailed description when taken with the accompanying drawings in which: FIG. 1 is a conceptual view illustrating an example of a configuration of a virtual input system according to the present disclosure; FIG. 2 is a conceptual view illustrating an example of an image displayed on a display of a virtual reality display device included in the virtual input system according to the present disclosure; FIG. 3 is a conceptual view illustrating an example of a virtual terminal displayed on the display of the virtual reality display device included in the virtual input system according to the present disclosure; FIG. 4 is a conceptual view illustrating an example of an operation for the virtual terminal displayed on the display of the virtual reality display device included in the virtual input system according to the present