Search

EP-4198559-B1 - OBJECT RECOGNITION METHOD AND APPARATUS BASED ON ULTRASONIC ECHO, AND STORAGE MEDIUM

EP4198559B1EP 4198559 B1EP4198559 B1EP 4198559B1EP-4198559-B1

Inventors

  • GUO, Ziyi

Dates

Publication Date
20260513
Application Date
20220111

Claims (14)

  1. An object recognition method based on ultrasonic echoes, performed by a computer device, the method comprising: controlling (301) an acoustic wave transmission apparatus of a terminal to transmit an ultrasonic signal to an object to be recognized; controlling (302) an acoustic wave receiving apparatus of the terminal to receive an ultrasonic echo signal reflected by the object to be recognized, the ultrasonic echo signal corresponding to the ultrasonic signal; extracting (303) an ultrasonic echo feature of the ultrasonic echo signal; processing (304) the ultrasonic echo feature using a target model, to obtain an ultrasonic wave feature for characterizing the object to be recognized, the ultrasonic wave feature having a target dimension; and performing (305) image translation processing on the ultrasonic wave feature of the object to be recognized by using a pre-trained image translation network, to obtain object image information corresponding to the object to be recognized, characterized in that , the method further comprises: acquiring image training data and ultrasonic echo training data, which are collected from a sample object, the sample object and the object to be recognized belonging to a same object class; inputting the image training data into an image feature extraction network in a preset model to determine an image training feature having the target dimension; obtaining frequency spectrum information of the ultrasonic echo training data to determine ar ultrasonic echo training feature; inputting the ultrasonic echo training feature into an ultrasonic wave conversion network in the preset model for conversion to obtain an ultrasonic wave training feature having the target dimension; obtaining similarity loss between the ultrasonic wave training feature and the image training feature; and adjusting parameters of the ultrasonic wave conversion network in the preset model based on the similarity loss, to obtain the target model.
  2. The method according to claim 1, wherein the adjusting parameters of the ultrasonic wave conversion network in the preset model based on the similarity loss to obtain the target model comprises: setting parameters of the image feature extraction network in the preset model to have fixed values; and in response to setting parameters of the image feature extraction network in the preset model to have fixed values, adjusting the parameters of the ultrasonic wave conversion network in the preset model based on the similarity loss, to obtain the target model.
  3. The method according to claim 2, further comprising: acquiring pre-training data indicating a correspondence between pre-training images and pre-training features; and training the image feature extraction network in the preset model based on the pre-training data to adjust the parameters of the image feature extraction network in the preset model.
  4. The method according to claim 1, further comprising: controlling, based on a preset frequency feature, the acoustic wave transmission apparatus to transmit single-channel ultrasonic waves to the sample object; receiving reflected ultrasonic wave data by using the acoustic wave receiving apparatus to determine the ultrasonic echo training data: in response to receiving the ultrasonic echo training data, collecting image training data corresponding to the sample object by using an image collection apparatus in the terminal, wherein the obtaining similarity loss between the ultrasonic wave training feature and the image training feature comprises: aligning the ultrasonic echo training data with the image training data based on time information of the ultrasonic echo training data and the image training data to generate a training sample pair; and determining the similarity loss based on the training sample pair.
  5. The method according to claim 4, wherein the receiving reflected ultrasonic wave data by using the acoustic wave receiving apparatus to determine the ultrasonic echo training data comprises: receiving to-be-determined ultrasonic wave data by using the acoustic wave receiving apparatus; filtering the to-be-determined ultrasonic wave data based on the preset frequency feature to determine the ultrasonic wave data reflected by the object to be recognized from the to-be-determined ultrasonic wave data; and determining the ultrasonic echo training data based on ultrasonic wave data reflected by the object to be recognized.
  6. The method according to claim 5, wherein the filtering the to-be-determined ultrasonic wave data based on the preset frequency feature to determine the ultrasonic wave data from the to-be-determined ultrasonic wave data comprises: dividing the to-be-determined ultrasonic wave data into a plurality of fixed-length ultrasonic sequences according to the preset frequency feature; filtering the ultrasonic sequences by using a preset filter to obtain filtered sequences; and performing normalization processing on the filtered sequences by using a sliding window to determine the ultrasonic wave data.
  7. The method according to claim 4, wherein the aligning the ultrasonic echo training data with the image training data based on time information of the ultrasonic echo training data and the image training data to generate a training sample pair comprises: in response to receiving the ultrasonic echo training data, collecting a video of the sample object by using the image collection apparatus; acquiring a time stamp corresponding to each video frame in the video; and aligning the ultrasonic echo training data with the image training data according to the time stamp to generate the training sample pair.
  8. The method according to claim 1, wherein the controlling (301) an acoustic wave transmission apparatus of a terminal to transmit an ultrasonic signal to an object to be recognized comprises: determining a recognition instruction in response to a private operation; and controlling, in response to the recognition instruction, the acoustic wave transmission apparatus to transmit single-channel ultrasonic waves.
  9. The method according to any one of claims 1 to 8, further comprising: acquiring a preset image stored in a target application, the target application corresponding to the target model; comparing the object image information with the preset image to obtain comparison information; and indicating whether to execute the target application based on the comparison information.
  10. The method according to any one of claims 1 to 8, the object image information being face image information, and the method further comprising: acquiring location information and depth-of-field information corresponding to the face image information; verifying the object to be recognized based on the location information and the depth-of-field information to determine object recognition information; and indicating whether to execute the target application based on the object recognition information.
  11. The method according to claim 1, wherein the ultrasonic signal is a single-channel ultrasonic wave, the terminal is a mobile phone, the acoustic wave transmission apparatus is a loudspeaker, and the acoustic wave receiving apparatus is a microphone.
  12. An object recognition apparatus based on ultrasonic echoes, comprising: a transmission unit (1201) configured to control an acoustic wave transmission apparatus of a terminal to transmit an ultrasonic signal to an object to be recognized; a receiving unit (1202) configured to control an acoustic wave receiving apparatus of the terminal to receive an ultrasonic echo signal reflected by the object to be recognized, the ultrasonic echo signal corresponding to the ultrasonic signal; an extraction unit (1203) configured to extract an ultrasonic echo feature of the ultrasonic echo signal; a conversion unit (1204) configured to process the ultrasonic echo feature using a target model, to obtain an ultrasonic wave feature for characterizing the object to be recognized, the ultrasonic wave feature having a target dimension; and a recognition unit (1205) configured to perform image translation processing on the ultrasonic wave feature of the object to be recognized by using a pre-trained image translation network, to obtain object image information corresponding to the object to be recognized, characterized in that , the recognition unit is further configured to: acquire image training data and ultrasonic echo training data, which are collected from a sample object, the sample object and the object to be recognized belonging to a same object class; input the image training data into an image feature extraction network in a preset model to determine an image training feature having the target dimension; obtain frequency spectrum information of the ultrasonic echo training data to determine ar ultrasonic echo training feature; input the ultrasonic echo training feature into an ultrasonic wave conversion network in the preset model for conversion to obtain an ultrasonic wave training feature having the target dimension; obtain similarity loss between the ultrasonic wave training feature and the image training feature; and adjust parameters of the ultrasonic wave conversion network in the preset model based on the similarity loss, to obtain the target model.
  13. A computer device, comprising an acoustic wave transmission apparatus, an acoustic wave receiving apparatus, a processor and a memory; the memory being configured to store program code; and the processor being configured to perform, with the acoustic wave transmission apparatus and the acoustic wave receiving apparatus, the object recognition method based on ultrasonic echoes according to any one of claims 1 to 11 according to instructions in the program code.
  14. A computer-readable storage medium, storing instructions, the instructions, when run on the computer device according to claim 13, causing the computer device to perform the object recognition method based on ultrasonic echoes according to any one of claims 1 to 11.

Description

FIELD The present disclosure relates to the technical field of artificial intelligence, in particular to object recognition based on ultrasonic echoes. BACKGROUND With the development of artificial intelligence technologies, object recognition has been widely applied to the fields such as security and finance. During the object recognition, using objects are recognized, such that the accuracy of permission management for application scenarios and the safety performance of terminals are improved. In related technologies, the object recognition is performed based on image recognition, that is, features of images are extracted by collecting image information of using objects or objects to be recognized, so as to take obtained results as recognition results of the objects. For example, CN 110688957 A discloses a living body detection method and device applied to face recognition and a storage medium, and belongs to the technical field of face recognition. The method comprises the steps of obtaining a first sound wave signal and a second sound wave signal, wherein the second sound wave signal is used for representing a sound wave signal obtained by reflecting a target face after the first sound wave signal is transmitted; obtaining a time domain feature and a frequency domain feature according to the first sound wave signal and the second sound wave signal; fusing the time domain features and the frequency domain features to obtain fused features; and based on the living body detection model, obtaining a detection label corresponding to the fusion feature. CN110074813A discloses a method and a system for reconstruction of ultrasonic images. A construction method of an image reconstruction network model comprises the following steps of obtaining a training sample set, wherein the training sample set comprises a plurality of sample pairs, each sample pair comprises a group of ultrasonic radio frequency training data and a corresponding training grayscale image, and the ultrasonic radio frequency training data is obtained according to natural images and/or medical images; constructing a generative adversarial network model, wherein the generative adversarial network model comprises a generator and a judger; utilizing the training sample set to train the generative adversarial network model, enabling the generative adversarial network model to reach Nash equilibrium, and using the generator under the Nash equilibrium state as the image reconstruction network model for the reconstruction of the ultrasonic images. CN111178340A discloses an image recognition method and a training method of an image recognition model. In one embodiment, the image recognition method comprises the following steps: firstly, acquiring a target face image; then, carrying out wavelet analysis processing of a preset number of layers on the target face image to obtain a target wavelet face image of a preset number of layers; then, inputting the target wavelet face image with the preset number of layers into a preset image recognition model to obtain a prediction value corresponding to the target face image, wherein the image recognition model is generated by training wavelet face training images of a preset number of layers corresponding to a plurality of face training images and a mark value corresponding to each face training image, and the plurality of face training images comprise an attack image and a non-attack image; and finally, determining an attack image recognition result corresponding to the target face image according to the prediction value. SUMMARY In view of this, the present disclosure provides an object recognition method based on ultrasonic echoes, which can effectively improve the accuracy of object recognition. The invention is set out in the appended set of claims. As can be seen from the foregoing technical solution, the embodiments of the present disclosure have the following advantages. An ultrasonic signal is transmitted, by using an acoustic wave transmission apparatus of a terminal to an object to be recognized; then, an ultrasonic echo signal reflected by the object to be recognized is received based on the ultrasonic signal through an acoustic wave receiving apparatus of the terminal, a target model is called to perform feature dimension conversion on the ultrasonic echo feature corresponding to the ultrasonic echo signal, to obtain an ultrasonic wave feature having a target dimension for characterizing the object to be recognized; and image translation processing is further performed on the ultrasonic wave feature having the target dimension of the object to be recognized, to obtain object image information corresponding to the object to be recognized. Compared with image collection in which accurate imaging may be implemented with sufficient ambient light, the transmission of ultrasonic signals and the reception of reflected ultrasonic echo signals are less sensitive to the changes of scenarios. In various scenar