Search

EP-4468122-B1 - SYSTEM, METHOD, AND PROGRAM FOR REALIZING USER INTERFACE BASED ON FINGER IDENTIFICATION

EP4468122B1EP 4468122 B1EP4468122 B1EP 4468122B1EP-4468122-B1

Inventors

  • Inoue, Yohei

Dates

Publication Date
20260506
Application Date
20230119

Claims (14)

  1. A method for providing a user interface to a user (101), comprising steps of: identifying a plurality of fingers of the user; assigning a mode switching function or a mode switching function and an object selection function to a first finger of the identified plurality of fingers; and switching functions assigned to a second finger and a third finger of the identified plurality of fingers to different functions in response to an action by the first finger, the action being (i) a touch action , when the mode switching function is assigned to the first finger; or (ii) a touch action or a hover action, when the mode switching function and the object selection function are assigned to the first finger; wherein at least one of the second finger and the third finger is different from the first finger; and wherein when the object selection function is not assigned to the first finger, returning a function of the second finger or the third finger to its pre-switching function in response to release of the action by the first finger.
  2. The method according to claim 1, wherein when the object selection function is assigned to the first finger, a selected state of a selected object is maintained even after the touch action or the hover action on the object is released.
  3. The method according to claim 1 or 2, wherein one or more functions assigned to the second finger include at least one of a command processing function, a parameter control function, and an object selection function.
  4. The method according to claim 3, wherein one or more functions assigned to the second finger after switching include a parameter control function; and wherein one or more function assigned to the third finger after switching include a function of changing the parameter control function.
  5. The method according to claim 3, wherein one or more functions assigned to the second finger after switching include a parameter control function; the method further includes assigning a function of changing the parameter control function to a finger different from the second finger in response to a touch action by the second finger.
  6. The method according to claim 3, wherein one or more functions assigned to the second finger after switching include a command processing function, wherein the command processing function is one of: an editing function, a search function, a saving function, a copying function, a calculation function, a sending function, and any combination thereof.
  7. The method according to claim 6, wherein the editing function includes at least one of moving, aligning, rotating, image editing, and adding text.
  8. The method according to claim 3, wherein the second finger is the same finger as the first finger.
  9. The method according to claim 1, further comprising a step of returning a function of the second finger or the third finger to a pre-switching function in response to release of the action by the first finger within or below a predetermined time when an object selection function is not assigned to the first finger.
  10. The method according to claim 1, further comprising a step of displaying an icon or a label representing a function assigned to each of the identified fingers on a display (104) used by the user along with a virtual hand representing the identified plurality of fingers.
  11. The method according to claim 10, wherein the label is displayed only when the finger associated with the label is positioned above a predetermined height from a desk used by the user.
  12. The method according to claim 10, wherein when the user's finger is outside a screen area of the display (104), the icon or the label is displayed at the edge of the screen.
  13. A program (200) for causing a computer (103) to perform a method of providing a user interface to a user (101), the method comprising steps of: identifying a plurality of fingers of the user; assigning a mode switching function or a mode switching function and an object selection function to a first finger of the identified plurality of fingers; and switching functions assigned to a second finger and a third finger of the identified plurality of fingers to different functions in response to an action by the first finger, the action being (i) a touch action , when the mode switching function is assigned to the first finger; or (ii) a touch action or a hover action, when the mode switching function and the object selection function are assigned to the first finger; wherein at least one of the second finger and the third finger is different from the first finger; and wherein when the object selection function is not assigned to the first finger, returning a function of the second finger or the third finger to its pre-switching function in response to release of the action by the first finger.
  14. An apparatus for providing a user interface to a user (101), configured to: identify a plurality of fingers of the user; assign a mode switching function or a mode switching function and an object selection function to a first finger of the identified plurality of fingers; and switch functions assigned to a second finger and a third finger of the identified plurality of fingers to different functions in response to an action by the first finger, the action being (i) a touch action, when the mode switching function is assigned to the first finger; or (ii) a touch action or a hover action, when the mode switching function and the object selection function are assigned to the first finger; wherein at least one of the second finger and the third finger is different from the first finger; and wherein when the object selection function is not assigned to the first finger, a function of the second finger or the third finger is returned to its pre-switching function in response to release of the action by the first finger.

Description

TECHNICAL FIELD The present invention relates to a system, method and program for realizing a user interface based on finger identification. BACKGROUND ART Various techniques have been proposed for user interfaces that allow users to interact with computers. One of these techniques is realized by identifying each finger of the user's hand and assigning a function to each. The basic example of such a technique is to assign a different function to each finger. If fixed functions are assigned, only 10 functions can be assigned for 10 fingers. Since 10 functions are not sufficient for many applications, it is necessary to appropriately switch the pairs of functions to be assigned. In order to use the assigned functions smoothly, it is important to make the correspondence between functions and fingers easily understandable. Patent document 1 describes a technology that associates a TV function to a finger. When said finger is in contact with the touchscreen 2, the technology associates a cancel function or a setting function of the TV function to another finger (FIG. 21, etc.). Patent document 2 provides a Touch Enhanced Interface (TEI) that translates data from touch devices into a wide variety of output actions. Patent document 3 provides a multipoint detectable touch panel detecting touches of a plurality of operation fingers onto an operating surface. Patent document 4 provides a user interface system comprising an operation section, a finger position detecting section, a component storing section, an assignment section, and a display controlling section, among others. PRIOR ART PATENT DOCUMENTS Patent document 1: International Publication No. 2006/104132Patent document 2: US 2016/048323Patent document 3: US 2015/253887Patent document 4: US 2008/163131Patent document 1: International Publication No. 2006/104132 SUMMARY OF THE INVENTION TECHNICAL PROBLEM Further improvements can be made in the user interface that identifies the user's fingers and assigns functions to multiple identified fingers. The present invention was made in view of the above, and its problem is to realize a more intuitive and efficient user interface based on the identification of each finger of the user in a system, method or program for providing a user interface to a user. SOLUTION TO THE PROBLEM A first aspect of the present invention is a method for providing a user interface to a user, comprising steps of: identifying a plurality of fingers of the user; assigning a mode switching function or a mode switching function and an object selection function to a first finger of the identified plurality of fingers; and switching functions assigned to a second finger and a third finger of the identified plurality of fingers to different functions in response to an action by the first finger, the action being (i) a touch action , when the mode switching function is assigned to the first finger; or (ii) a touch action or a hover action, when the mode switching function and the object selection function are assigned to the first finger; wherein at least one of the second finger and the third finger is different from the first finger; and wherein when the object selection function is not assigned to the first finger, returning a function of the second finger or the third finger to its pre-switching function in response to release of the action by the first finger. Preferred embodiments are defined in the appended dependent claims. Another aspect of the present invention is a program for causing a computer to perform a method of providing a user interface to a user, the method comprising steps of: identifying a plurality of fingers of the user; assigning a mode switching function or a mode switching function and an object selection function to a first finger of the identified plurality of fingers; and switching functions assigned to a second finger and a third finger of the identified plurality of fingers to different functions in response to an action by the first finger, the action being (i) a touch action , when the mode switching function is assigned to the first finger; or (ii) a touch action or a hover action, when the mode switching function and the object selection function are assigned to the first finger; wherein at least one of the second finger and the third finger is different from the first finger; and wherein when the object selection function is not assigned to the first finger, returning a function of the second finger or the third finger to its pre-switching function in response to release of the action by the first finger. Another aspect of the present invention is an apparatus for providing a user interface to a user, configured to: identify a plurality of fingers of the user; assign a mode switching function or a mode switching function and an object selection function to a first finger of the identified plurality of fingers; and switch functions assigned to a second finger and a third finger of the identified plurality of fi