Search

EP-4742006-A1 - HEAD-MOUNTED DISPLAY DEVICE FOR DISPLAYING INTERFACE, AND OPERATION METHOD OF HEAD-MOUNTED DISPLAY DEVICE

EP4742006A1EP 4742006 A1EP4742006 A1EP 4742006A1EP-4742006-A1

Abstract

The present disclosure includes a head-mounted display device and operation method thereof, the head-mounted display device including a camera, a display, memory storing at least one instruction, and at least one processor, wherein the at least one processor is configured to execute the at least one instruction to cause the head-mounted display device to display a first interface through the display, obtain an image of a user captured through the camera, detect a pose of the user based on the obtained image, compare the detected pose with a plurality of poses respectively preset to correspond to a plurality of interfaces, and based on the detected pose corresponding to a second interface that is one of the plurality of interfaces, display the second interface through the display by replacing the first interface.

Inventors

  • Kim, Jiin
  • KWAK, Junho
  • CHO, SUNGWOO

Assignees

  • Samsung Electronics Co., Ltd.

Dates

Publication Date
20260513
Application Date
20240809

Claims (15)

  1. A head-mounted display device (100) comprising: a camera (130); a display (140); memory (180) storing at least one instruction; and at least one processor (190), wherein the at least one processor (190) is configured to execute the at least one instruction to cause the head-mounted display device (100) to display a first interface (500) through the display (140), obtain an image of a user captured through the camera (130), detect a pose of the user based on the obtained image, compare the detected pose with a plurality of poses respectively preset to correspond to a plurality of interfaces, and based on the detected pose corresponding to a second interface (510) that is one of the plurality of interfaces, display the second interface (510) through the display (140) by replacing the first interface (500).
  2. The head-mounted display device (100) of claim 1, wherein the at least one processor (190) is further configured to execute the at least one instruction to cause the head-mounted display device (100) to compare the first interface (500) with the second interface (510), and based on a determination that the first interface (500) and the second interface (510) are different interfaces, display the second interface (510) through the display (140) by replacing the first interface (500).
  3. The head-mounted display device (100) of claim 2, wherein the at least one processor (190) is further configured to execute the at least one instruction to cause the head-mounted display device (100) to, based on a determination that the detected pose is different from the plurality of preset poses or that the first interface (500) and the second interface (510) are the same interface, maintain the first interface (500) displayed through the display (140).
  4. The head-mounted display device (100) of claim 1, wherein the at least one processor (190) is further configured to execute the at least one instruction to cause the head-mounted display device (100) to display first content through the display (140), and display the first interface (500) and the second interface (510) through the display (140) while the first content (300) is displayed.
  5. The head-mounted display device (100) of any one of claims 1 to 4, wherein the at least one processor (190) is further configured to execute the at least one instruction to cause the head-mounted display device (100) to obtain a spatial image by capturing, through the camera (130), an image of a space including the head-mounted display device (100), generate a spatial map representing the space based on the obtained spatial image, determine, based on the generated spatial map and the detected pose, an interface location where the second interface (510) is to be displayed, and display, through the display (140), the second interface (510) at the interface location.
  6. The head-mounted display device (100) of any one of claims 1 to 5, further comprising an eye-tracking sensor configured to track a gaze direction of the user, wherein the at least one processor (190) is further configured to execute the at least one instruction to cause the head-mounted display device (100) to obtain a viewpoint of the user based on the gaze direction.
  7. The head-mounted display device (100) of claim 6, wherein the at least one processor (190) is further configured to execute the at least one instruction to cause the head-mounted display device (100) to detect the pose of the user based on the obtained viewpoint of the user and the obtained image.
  8. The head-mounted display device (100) of claim 6 or 7, wherein the at least one processor (190) is further configured to execute the at least one instruction to cause the head-mounted display device (100) to determine, based on the obtained viewpoint of the user, a content location where second content that shows a user input obtained via at least one of the first interface (500) or the second interface (510) is to be displayed, and display, through the display (140), the second content at the content location.
  9. The head-mounted display device (100) of any one of claims 1 to 8, wherein the first interface (500) is an interface displayed on the display (140) based on the detected pose corresponding to one of the plurality of preset poses that is set to correspond to the first interface (500), and the pose set to correspond to the first interface (500) and a pose set to correspond to the second interface (510) are different poses.
  10. The head-mounted display device (100) of any one of claims 1 to 9, wherein the at least one processor (190) is further configured to execute the at least one instruction to cause the head-mounted display device (100) to compare the detected pose with a preset start pose, and based on the detected pose corresponding to the preset start pose, compare the detected pose with the plurality of preset poses.
  11. An operation method of a head-mounted display device, the operation method comprising: obtaining an image of a user captured through a camera (S100); detecting a pose of the user based on the obtained image (S200); comparing the detected pose with a plurality of poses respectively preset to correspond to a plurality of interfaces (S300); and based on the detected pose corresponding to a second interface that is one of the plurality of interfaces, displaying the second interface through a display by replacing the first interface (S600).
  12. The operation method of claim 11, further comprising comparing the first interface with the second interface (S500), wherein, based on a determination that the first interface and the second interface are different interfaces in the comparing of the first interface with the second interface (S500), the displaying of the second interface through the display (S600) comprises displaying the second interface through the display by replacing the first interface.
  13. The operation method of claim 12, further comprising maintaining the first interface displayed through the display based on the detected pose being different from the plurality of preset poses in the comparing of the detected pose with the plurality of preset poses (S300), or based on a determination that the first interface and the second interface are the same interface in the comparing of the first interface with the second interface (S500) (S700).
  14. The operation method of claim 11, further comprising displaying first content through the display (S50), wherein the first interface and the second interface are displayed through the display while the first content is displayed.
  15. A computer-readable recording medium having recorded thereon a program for performing, on a computer, the operation method of any one of claims 11 to 14.

Description

Technical Field The present disclosure relates to a head-mounted display device and an operation method of the head-mounted display device. Specifically, the present disclosure relates to a head-mounted display device for displaying an interface, and an operation method of the head-mounted display device. Background Art With advancements in technology, head-mounted display devices have been developed that provide images to a user through a display positioned close to the user's eyes while being worn on the user's head. A display included in a head-mounted display device may include an optical see-through display for overlaying a virtual image onto a real-world physical environment space or a real-world object to show them together, or a video see-through display for displaying images of a real-world physical environment space captured by using cameras or the like and presenting the images to the user. Recently, technology has been developed that provides, via an interface, a user input to images provided through a head-mounted display device. Disclosure of Invention Solution to Problem According to an embodiment of the present disclosure, a head-mounted display device is provided. The head-mounted display device may include a camera. The head-mounted display device may include a display. The head-mounted display device may include memory storing at least one instruction and at least one processor. The at least one processor may be configured to execute the at least one instruction to cause the head-mounted display device to display a first interface through the display. The at least one processor may be configured to execute the at least one instruction to cause the head-mounted display device to obtain an image of a user captured through the camera. The at least one processor may be configured to execute the at least one instruction to cause the head-mounted display device to detect a pose of the user based on the obtained image. The at least one processor may be configured to execute the at least one instruction to cause the head-mounted display device to compare the detected pose with a plurality of poses respectively preset to correspond to a plurality of interfaces. The at least one processor may be configured to execute the at least one instruction to cause the head-mounted display device to, based on the detected pose corresponding to a second interface that is one of the plurality of interfaces, display the second interface through the display by replacing the first interface. According to an embodiment of the present disclosure, an operation method of a head-mounted display device is provided. The operation method of the head-mounted display device may include obtaining an image of a user captured through a camera. The operation method of the head-mounted display device may include detecting a pose of the user based on the obtained image. The operation method of the head-mounted display device may include comparing the detected pose with a plurality of poses respectively preset to correspond to a plurality of interfaces. The operation method of the head-mounted display device may include, based on the detected pose corresponding to a second interface that is one of the plurality of interfaces, displaying the second interface through a display by replacing the first interface. According to an embodiment of the present disclosure, there may be provided a computer-readable recording medium having recorded thereon a program for performing, on a computer, at least one of the operation methods according to the disclosed embodiments. Brief Description of Drawings The present disclosure will be readily understood from the following description taken in conjunction with the accompanying drawings in which reference numerals denote structural elements. FIG. 1 is a diagram illustrating a head-mounted display device according to an embodiment of the present disclosure.FIG. 2 is a diagram illustrating a configuration of a head-mounted display device according to an embodiment of the present disclosure.FIG. 3 is a block diagram of a configuration of a head-mounted display device according to an embodiment of the present disclosure.FIG. 4 is a flowchart illustrating an operation method of a head-mounted display device, according to an embodiment of the present disclosure.FIG. 5 is a flowchart illustrating an operation of displaying a first interface and a second interface while first content is displayed, according to an embodiment of the present disclosure.FIG. 6 is a diagram illustrating an operation of displaying an interface corresponding to a detected pose of a user, according to an embodiment of the present disclosure.FIG. 7 is a flowchart illustrating an operation of determining an interface location based on a spatial map generated through a camera and a detected pose, according to an embodiment of the present disclosure.FIG. 8 is a flowchart illustrating an operation of obtaining a viewpoint of a user via