US-12623355-B2 - Method and apparatus for estimating touch locations and touch pressures

US12623355B2US 12623355 B2US12623355 B2US 12623355B2US-12623355-B2

Abstract

A tactile sensing system of a robot may include: a plurality of piezoelectric elements disposed at an object, and including a transmission (TX) piezoelectric element and a reception (RX) piezoelectric element; and at least one processor configured to: control the TX piezoelectric element to generate an acoustic wave having a chirp spread spectrum (CSS) at every preset time interval, along a surface of the object; receive, via the RX piezoelectric element, an acoustic wave signal corresponding to the generated acoustic wave; select frequency bands from a plurality of frequency bands of the acoustic wave signal; and estimate a location of a touch input on the surface of the object by inputting the acoustic wave signal of the selected frequency bands into a neural network configured to provide a touch prediction score for each of a plurality of predetermined locations on the surface of the object.

Inventors

Xiaoran FAN
Daewon Lee
Lawrence Jackel
Richard Howard
Daniel Dongyuel Lee
Ibrahim Volkan Isler

Assignees

SAMSUNG ELECTRONICS CO., LTD.

Dates

Publication Date: 20260512
Application Date: 20241202

Claims (20)

1 . An apparatus for identifying a location of a touch input, the apparatus comprising: a plurality of piezoelectric elements disposed at an object, and comprising a first piezoelectric element and a second piezoelectric element; at least one processor including processing circuitry; and a memory storing instructions that, when executed by the at least one processor individually or collectively, cause the apparatus to: control the first piezoelectric element to generate an acoustic wave, along a surface of the object; receive, by using the second piezoelectric element, a signal corresponding to the generated acoustic wave; identify a first variance of the signal that is received in case that the surface of the object is touched, and a second variance of the signal that is received in case that there is no touch on the surface of the object; select frequency bands from a plurality of frequency bands of the signal, based on the first variance of the signal, and the second variance of the signal; execute a neural network based on the received signal corresponding to the selected frequency bands, the neural network trained to provide touch prediction scores corresponding to a plurality of defined locations on the surface of the object; and identify a location of a touch input based on the touch predication scores provided from the neural network.
2 . The apparatus of claim 1 , wherein the generated acoustic wave has a chirp spread spectrum (CSS) at a defined time interval, and wherein the CSS has a linearly increasing frequency over time and has a constant amplitude within a predetermined frequency range.
3 . The apparatus of claim 1 , wherein the instructions that, when executed by the at least one processor individually or collectively, further cause the apparatus to: reduce a noise of the received signal by filtering the received signal, the noise of the received signal comprising an electrical noise of the object and a mechanical noise of the object.
4 . The apparatus of claim 3 , wherein the object is a robot, and the electrical noise is a pulse-width modulation (PWM) noise of a motor of the robot, and wherein the instructions that, when executed by the at least one processor individually or collectively, cause the apparatus to: filter out the PWM noise at 30 kHz, 60 kHz, and 90 kHz of the received signal, and filter out the mechanical noise that resides below a 20 kHz range of the received signal.
5 . The apparatus of claim 1 , wherein the least one processor is configured to: select a defined number of the frequency bands in an ascending order of a ratio of the second variance to the first variance.
6 . The apparatus of claim 1 , wherein y is a defined number of the frequency bands that are selected from the plurality of frequency bands, and wherein the instructions that, when executed by the at least one processor individually or collectively, cause the apparatus to: determine γ such that a weighted sum of a touch prediction accuracy of the neural network and an inverse of γ is maximized.
7 . The apparatus of claim 1 , wherein the instructions that, when executed by the at least one processor individually or collectively, cause the apparatus to: at a calibration duration, obtain a baseline touch prediction score for the plurality of defined locations on the surface of the object, in case that the surface of the object is not touched; and when the surface of the object is touched after calibration, adjust a touch prediction score obtained from the neural network based on the baseline touch prediction score.
8 . The apparatus of claim 2 , wherein the predetermined frequency range is a range from 20 kHz to 80 Hz, and the defined time interval is in a range from 90 ms to 110 ms.
9 . The apparatus of claim 1 , wherein the least one processor is further configured to: estimate a pressure of the touch input by applying a first order Fourier model to a total energy of the received signal across the selected frequency bands.
10 . A method for acquiring tactile sensing data, the method comprising: controlling a first piezoelectric element to generate an acoustic wave, along a surface of an object; receiving, by using a second piezoelectric element, a signal corresponding to the generated acoustic wave; identifying a first variance of the received signal in case that the surface of the object is touched, and a second variance of the received signal in case that there is no touch on the surface of the object; selecting frequency bands from a plurality of frequency bands of the received signal, based on the first variance of the signal, and the second variance of the signal; executing, based on the received signal corresponding to the selected frequency bands, a neural network that is trained to provide touch prediction scores corresponding to a plurality of defined locations on the surface of the object; and identify a location of a touch input based on the touch prediction scores provided from the neural network.
11 . The method of claim 10 , wherein the generated acoustic wave has a chirp spread spectrum (CSS) at a defined time interval, and wherein the CSS has a linearly increasing frequency over time and has a constant amplitude within a predetermined frequency range.
12 . The method of claim 10 , further comprising: reducing a noise of the received signal by filtering the received signal, wherein the noise of received signal comprising an electrical noise of the object and a mechanical noise of the object.
13 . The method of claim 12 , wherein the object is a robot, and the electrical noise is a pulse-width modulation (PWM) noise of a motor of the robot, and wherein the reducing comprises: filtering out the PWM noise at 30 kHz, 60 kHz, and 90 kHz of the received signal; and filtering out the mechanical noise that resides below a 20 kHz range of the received signal.
14 . The method of claim 10 , wherein the selecting frequency bands comprises: selecting a defined number of the frequency bands in an ascending order of a ratio of the second variance to the first variance of the received signal.
15 . The method of claim 10 , wherein y is a defined number of the frequency bands that are selected from the plurality of frequency bands, and wherein the selecting frequency bands comprises: determining γ such that a weighted sum of a touch prediction accuracy of the neural network and an inverse of γ is maximized.
16 . The method of claim 10 , further comprising: at a calibration interval, obtaining a baseline touch prediction score for the plurality of defined locations on the surface of the object, in case that the surface of the object is not touched; and based on the surface of the object being touched after calibration, adjusting a touch prediction score obtained from the neural network based on the baseline touch prediction score.
17 . The method of claim 11 , wherein the predetermined frequency range is a range from 20 kHz to 80 Hz, and the defined time interval is in a range from 90 ms to 110 ms.
18 . The method of claim 10 , further comprising: estimating a pressure of the touch input by applying a first order Fourier model to a total energy of the received signal across the selected frequency bands.
19 . A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor individually or collectively, cause the at least one processor to: control a first piezoelectric element to generate an acoustic wave, along a surface of an object; receive, by using a second piezoelectric element, a signal corresponding to the generated acoustic wave; identify a first variance of the signal that is received in case that the surface of the object is touched, and a second variance of the signal that is received in case that there is no touch on the surface of the object; select frequency bands from a plurality of frequency bands of the signal, based on the first variance of the signal and the second variance of the signal; execute a neural network based on the received signal corresponding to the selected frequency bands, the neural network trained to provide touch prediction scores corresponding to a plurality of defined locations on the surface of the object; and identify a location of a touch input based on the touch prediction scores provided from the neural network.
20 . The non-transitory computer-readable storage medium of claim 19 , wherein the instructions cause the at least one processor to: reduce a noise of the received signal by filtering the received signal, the noise of the received signal comprising an electrical noise of the object and a mechanical noise of the object.

Description

CROSS-REFERENCE TO RELATED APPLICATION This application is a continuation application of U.S. patent application Ser. No. 17/553,321, filed Dec. 16, 2021, which is based on and claims priority under 35 U.S.C. § 119 from U.S. Provisional Application No. 63/241,909 filed on Sep. 8, 2021, in the U.S. Patent & Trademark Office, the disclosures of which are incorporated by reference herein in their entireties. BACKGROUND 1. Field The disclosure relates to a method and an apparatus for controlling a tactile sensor to inject an acoustic wave signal to be distributed over a surface of an object, and when a touch is made at an arbitrary location of the surface, estimating either or both of a location and a pressure of the touch. 2. Description of Related Art Tactile sensing is important for many robotic applications. For example, as robots work in dynamic environments in collaboration with humans, tactile sensing plays a key role for human-robot interactions (HRI), such as a safe robot operation around humans, providing emotional support to humans, and controlling robot behaviors according to human guidance. Existing approaches for realizing tactile skins on robots may be divided into two methods: a first method of using tens to thousands of exteroceptive sensors such as capacitive, magnetic and infrared (IR) types of sensors, that are deployed on a robot; and a second method of using proprioception, such as motor torque, position, velocity and momentum readings coupled with inverse kinematics and dynamics, to infer a contact location and a contact force. The first method may require a bulky structure and a high manufacturing cost, and the second method may provide poor estimation results of the contact location and the contact force due to noisy and time varying properties of motors. This complicates robotic system design and adds a significant amount of extra cost and sensor management overhead. SUMMARY In accordance with an aspect of the disclosure, there is provided an apparatus for acquiring tactile sensing data, including: a plurality of piezoelectric elements disposed at an object, and including a transmission (TX) piezoelectric element and a reception (RX) piezoelectric element; a memory storing instructions; and at least one processor configured to execute the instructions to: control the TX piezoelectric element to generate an acoustic wave having a chirp spread spectrum (CSS) at every preset time interval, along a surface of the object, wherein the CSS has a linearly increasing frequency over time and has a constant amplitude within a predetermined frequency range; receive, via the RX piezoelectric element, an acoustic wave signal corresponding to the generated acoustic wave; select frequency bands from a plurality of frequency bands of the acoustic wave signal, based on a first variance of the acoustic wave signal that is received when the surface of the object is touched during a movement of the object, and a second variance of the acoustic wave signal that is received when there is no touch on the surface of the object during the movement of the object; and estimate a location of a touch input on the surface of the object by inputting the acoustic wave signal of the selected frequency bands into a neural network configured to provide a touch prediction score for each of a plurality of predetermined locations on the surface of the object. The least one processor may be further configured to: filter the received acoustic wave signal, using at least one filter configured to reduce an electrical noise of the object and a mechanical noise of the object, from the received acoustic wave signal. The object may be a robot, and the electrical noise may be a pulse-width modulation (PWM) noise of a motor of the robot, and the at least one filter may be further configured to filter the PWM noise at 30 kHz, 60 kHz, and 90 kHz of the received acoustic wave signal, and filter the mechanical noise that resides below a 20 kHz range of the received acoustic wave signal. The least one processor may be further configured to: select a predetermined number of the frequency bands in an ascending order of a ratio of the second variance to the first variance of the received acoustic wave signal. When γ is a predetermined number of the frequency bands that are selected from the plurality of frequency bands, the least one processor may be further configured to: predetermine γ such that a weighted sum of a touch prediction accuracy of the neural network and an inverse of γ is maximized. The least one processor may be further configured to: at a preset calibration interval, obtain a baseline touch prediction score for the plurality of predetermined locations on the surface of the object, when the surface of the object is not touched; and when the surface of the object is touched after calibration, adjust the touch prediction score obtained from the neural network based on the baseline touch prediction score. The predetermined frequ