EP-4740149-A2 - INFERENCE WINDOW FOR GESTURE INPUT

Abstract

A system is configured to determine (520) a window of time during which to execute (540) a particular gesture or voice command to a computer game. The system otherwise does not execute the command outside the window of time, helping to reduce false positives from unintended input.

Inventors

• MATSUI, YASUNORI
• KONDO, YURIKA
• KANG, MIHEE
• TAYLOR, MICHAEL

Assignees

• Sony Interactive Entertainment Inc.

Dates

Publication Date

20260513

Application Date

20240613

Claims (20)

1. 1. An apparatus, comprising: at least one processor assembly programmed with instructions to: execute a computer game; receive game state data related to the execution of the computer game; identify, based on the game state data and during the execution of the computer game, a first window of time during which to execute a command in conformance with a first noncomputer game controller input to the computer game; during the first window of time, identify the first non-computer game controller input; and based on the identification of the first non-computer game controller input during the first window of time, execute the command in relation to the computer game, the command executed in conformance with the first non-computer game controller input.
2. 2. The apparatus of Claim 1, wherein the game state data is first game state data, and wherein the at least one processor assembly is programmed with instructions to: receive second game state data related to the execution of the computer game; identify, based on the second game state data and during the execution of the computer game, a second window of time during which to not execute the command in conformance with the first non-computer game controller input to the computer game; and during the second window of time, not execute the command in relation to the computer game.
3. 3. The apparatus of Claim 2, wherein not executing the command comprises not processing the first non-computer game controller input to the computer game.
4. 4. The apparatus of Claim 2, wherein not executing the command comprises processing the first non-computer game controller input to the computer game but not executing the command in conformance with the first non-computer game controller input to the computer game.
5. 5. The apparatus of Claim 1, wherein the first non-computer game controller input comprises gesture input.
6. 6. The apparatus of Claim 5, comprising a camera, wherein the gesture input is identified using one or more images from the camera.
7. 7. The apparatus of Claim 1 , wherein the first non-computer game controller input comprises voice input.
8. 8. The apparatus of Claim 7, comprising a microphone, wherein the voice input is identified using voice data from the microphone.
9. 9. The apparatus of Claim 1, wherein the game state data indicates a first context associated with execution of the computer game, and wherein the at least one processor assembly is programmed with instructions to: identify the first window of time based on identification of the first context from the game state data.
10. 10. The apparatus of Claim 9, wherein the at least one processor assembly is programmed with instructions to: provide the game state data as input to a machine learning (ML) model; receive, as an output from an activation layer of the ML model, an inference indicating the first context; and identify the first context from the inference.
11. 11. The apparatus of Claim 9, wherein the first window of time is a dynamic window of time.
12. 12. The apparatus of Claim 11, wherein the dynamic first window of time lasts as long as the first context continues during execution of the computer game.
13. 13. The apparatus of Claim 1, wherein the at least one processor assembly is programmed with instructions to: provide the game state data as input to a machine learning (ML) model; and receive, as an output from an activation layer of the ML model, an inference indicating the first window of time.
14. 14. The apparatus of Claim 13, wherein the ML model is trained using at least one dataset of game state data and ground truth windows of time during which to monitor for noncomputer game controller input.
15. 15. A method, comprising: executing a computer game; receiving game state data related to the execution of the computer game; identifying, based on the game state data and during the execution of the computer game, a first window of time during which to execute a command in conformance with a first non-computer game controller input to the computer game; during the first window of time, identifying the first non-computer game controller input; and based on the identification of the first non-computer game controller input during the first window of time, executing the command in relation to the computer game, the command executed in conformance with the first non-computer game controller input.
16. 16. The method of Claim 15, wherein the first non-computer game controller input comprises one or more of: gesture input of a predetermined gesture, voice input of a predetermined voice command.
17. 17. A sy stem compri si ng : at least one computer medium that is not a transitory signal and that comprises instructions executable by at least one processor assembly to: infer a first window of time during which to execute, for a first execution instance of a computer game, a non-computer game controller command to the first execution instance of the computer game, the first window of time not spanning all of the first execution instance of the computer game; and based on the inference, execute the non-computer game controller command during the first window of time.
18. 18. The system of Claim 17, wherein the first window of time does not span all of a particular gameplay sequence of a segment or level of the computer game but spans some of the particular gameplay sequence of the segment or level of the computer game.
19. 19. The system of Claim 17, wherein the instructions are executable to: infer the first window of time using game state data and a machine learning (ML) model.
20. 20. The system of Claim 19, wherein the ML model is trained using at least one dataset of game state data, player outcomes from previous gameplay instances, and ground truth windows of time during which to execute respective non-computer game controller commands.

Description

INFERENCE WINDOW FOR GESTURE INPUT FIELD The disclosure below relates generally to inferred windows of time during which to monitor for gesture input to a computer game. BACKGROUND As recognized herein, one of the technical challenges facing computer game developers and console makers alike is the need to correctly identify gesture and speech input to a computer game to distinguish that input from other natural gestures and speech that might be unrelated to the game. For instance, a player might make an unrelated gesture or speak to a non-player while the game is executing, but the game system still picks up on the gesture or speech and executes a corresponding game command even though the player did not intend to actually control the game. This in turn adversely affects gameplay, sometimes to a significant degree. There are currently no adequate solutions to the foregoing computer-related, technological problems. SUMMARY Accordingly, in one aspect an apparatus includes at least one processor assembly programmed with instructions to execute a computer game and to receive game state data related to the execution of the computer game. The at least one processor assembly is also programmed with instructions to identify, based on the game state data and during the execution of the computer game, a first window of time during which to execute a command in conformance with a first non-computer game controller input to the computer game. The at least one processor assembly is further programmed with instructions to identify the first noncomputer game controller input during the first window of time and to execute the command in relation to the computer game based on the identification of the first non-computer game controller input during the first window of time. The command is executed in conformance with the first non-computer game controller input. Thus, in one example implementation the game state data may be first game state data and the at least one processor assembly may be programmed with instructions to receive second game state data related to the execution of the computer game. Here the at least one processor assembly may be programmed with instructions to then identify, based on the second game state data and during the execution of the computer game, a second window of time during which to not execute the command in conformance with the first non-computer game controller input to the computer game. The at least one processor assembly may therefore also be programmed with instructions to, during the second window of time, not execute the command in relation to the computer game. Not executing the command may include not processing the first non-computer game controller input to the computer game, and/or may include processing the first non-computer game controller input to the computer game but not executing the command in conformance with the first non-computer game controller input. In various examples, the first non-computer game controller input may include gesture input. So the apparatus might include a camera and the gesture input may be identified using one or more images from the camera. Also in various examples, the first non-computer game controller input may include voice input. So the apparatus might also include a microphone and the voice input may be identified using voice data from the microphone. Additionally, if desired the game state data may indicate a first context associated with execution of the computer game and the at least one processor assembly may be programmed with instructions to identify the first window of time based on identification of the first context from the game state data. So, for example, the at least one processor assembly may be programmed with instructions to provide the game state data as input to a machine learning (ML) model, receive an inference indicating the first context as an output from an activation layer of the ML model, and identify the first context from the inference. The first window of time may be a dynamic window of time, such as one that lasts as long as the first context continues during execution of the computer game. Also if desired, the at least one processor assembly may be programmed with instructions to provide the game state data as input to a machine learning (ML) model and to receive, as an output from an activation layer of the ML model, an inference indicating the first window of time itself. The ML model may be trained using at least one dataset of game state data and ground truth windows of time during which to monitor for non-computer game controller input. In another aspect, a method includes executing a computer game and receiving game state data related to the execution of the computer game. The method also includes identifying, based on the game state data and during the execution of the computer game, a first window of time during which to execute a command in conformance with a first non-computer game controller i