Search

CN-122002209-A - Multi-mode cooperative processing method, control device and terminal equipment

CN122002209ACN 122002209 ACN122002209 ACN 122002209ACN-122002209-A

Abstract

The application discloses a multi-mode cooperative processing method, a control device and terminal equipment, which relate to the technical field of audio processing, wherein the multi-mode cooperative processing method comprises the following steps of obtaining an audio signal; the method comprises the steps of extracting a target vibration signal in an audio signal, carrying out space information analysis on the target vibration signal, determining the frequency spectrum characteristic of the target vibration signal, determining target distance information in space information of a sound source based on the frequency spectrum characteristic of the target vibration signal, determining a tactile feedback parameter corresponding to the space information of the sound source, and generating a driving signal for driving a vibration unit to work based on the mapping relation between the space information of the sound source and the tactile feedback parameter so as to generate tactile feedback. The method converts the sound source space position into perceivable directional touch experience, and solves the technical problem that the haptic feedback effect on different distance positions is poor due to the fact that the audio space information is not combined with the haptic feedback.

Inventors

  • XIE PING
  • YANG XINFENG

Assignees

  • 歌尔股份有限公司

Dates

Publication Date
20260508
Application Date
20251230

Claims (10)

  1. 1. The multi-mode cooperative processing method is characterized by comprising the following steps of: Acquiring an audio signal; Extracting a target vibration signal in the audio signal, carrying out spatial information analysis on the target vibration signal, and determining the frequency spectrum characteristics of the target vibration signal; determining target distance information in the spatial information of the sound source based on the spectral characteristics of the target vibration signal; Determining a haptic feedback parameter corresponding to spatial information of the sound source; Based on the mapping relation between the space information of the sound source and the tactile feedback parameters, a driving signal for driving the vibration unit to work is generated so as to generate tactile feedback.
  2. 2. The multi-modal collaborative processing method as set forth in claim 1, wherein determining target distance information in spatial information of a sound source based on spectral characteristics of a target vibration signal includes: calculating the energy ratio of a high-frequency signal and a low-frequency signal in the target vibration signal based on the frequency spectrum characteristics of the target vibration signal, and determining target distance information in the spatial information of the sound source; the generating a driving signal for driving the vibration unit to work so as to generate haptic feedback based on the mapping relation between the spatial information of the sound source and the haptic feedback parameter comprises: Determining the working strength of the motor according to the target distance information of the sound source based on the mapping relation between the space information of the sound source and the tactile feedback parameters; A driving signal for driving the vibration unit to operate at a corresponding operating intensity to generate haptic feedback is generated.
  3. 3. The multi-modal collaborative processing method as set forth in claim 1, wherein determining target distance information in spatial information of a sound source based on spectral characteristics of a target vibration signal includes: calculating the change trend of the target vibration signal along with time based on the frequency spectrum characteristics of the target vibration signal, and determining target distance information in the spatial information of the sound source; the generating a driving signal for driving the vibration unit to work so as to generate haptic feedback based on the mapping relation between the spatial information of the sound source and the haptic feedback parameter comprises: Determining the working strength of the motor according to the target distance information of the sound source based on the mapping relation between the space information of the sound source and the tactile feedback parameters; A driving signal for driving the vibration unit to operate at a corresponding operating intensity to generate haptic feedback is generated.
  4. 4. The multi-modal collaborative processing method as set forth in claim 1 wherein said vibratory unit includes a plurality of motors and said haptic feedback parameters include motor position, motor operating intensity and motor operating frequency corresponding to each of said motors; the generating a driving signal for driving the vibration unit to work so as to generate haptic feedback based on the mapping relation between the spatial information of the sound source and the haptic feedback parameter comprises: acquiring a motor position, and determining a target motor according to the acquired motor position based on the mapping relation between the spatial information of the sound source and the tactile feedback parameters; and generating a driving signal for controlling the target motor to work according to the motor working intensity and the motor working frequency so as to generate haptic feedback.
  5. 5. The multi-modal collaborative processing method as set forth in claim 1, wherein the spatially resolving the target vibration signal further includes: Carrying out framing treatment on the target vibration signal, acquiring time-frequency information from the target vibration signal, and determining the target frequency of the sound source; the generating a driving signal for driving the vibration unit to work so as to generate haptic feedback based on the mapping relation between the spatial information of the sound source and the haptic feedback parameter comprises: determining the working frequency of the motor according to the target frequency of the sound source based on the mapping relation between the space information of the sound source and the tactile feedback parameters; a drive signal is generated for driving the target motor to operate at a corresponding operating frequency to produce haptic feedback.
  6. 6. The multi-modal collaborative processing method of any one of claims 1-5, wherein prior to performing the step of generating a drive signal for driving operation of a vibratory unit to produce haptic feedback, the multi-modal collaborative processing method includes: Acquiring configuration information, and updating target tactile feedback parameters according to the configuration information, wherein the configuration information acquisition comprises any one or a combination of more of the following: Acquiring configuration information triggered based on a user setting interface; Acquiring configuration information updated based on machine learning; Configuration information determined based on the scene mode instructions is obtained.
  7. 7. The multi-modal collaborative processing method of any one of claims 1-5, wherein prior to performing the step of generating a drive signal for driving operation of a vibratory unit to produce haptic feedback, the multi-modal collaborative processing method includes: extracting envelope features in the audio signal to determine amplitude features of the sound source; And updating the target tactile feedback parameters based on the amplitude characteristics of the sound source.
  8. 8. The multi-modal collaborative processing method according to any one of claims 1-5, further comprising the steps of: Extracting a target acoustic signal from the audio signal; An acoustic drive signal for controlling the operation of a speaker unit comprising at least one speaker assembly is generated and output based on the target acoustic signal.
  9. 9. A control device, characterized in that the device comprises a memory, a processor and a multi-modal co-processing program stored on the memory and executable on the processor, the multi-modal co-processing program being configured to implement the steps of the multi-modal co-processing method according to any one of claims 1 to 8.
  10. 10. A terminal device, comprising: A vibration unit including at least one motor; A speaker unit including at least one speaker assembly; a processor for controlling the operation of the vibration unit, the speaker unit, and for implementing the steps of the multi-modal collaborative processing method according to any one of claims 1-8.

Description

Multi-mode cooperative processing method, control device and terminal equipment Technical Field The present application relates to the field of audio processing technologies, and in particular, to a multi-mode collaborative processing method, a control device, and a terminal device. Background The three-dimensional space sound effect (3D Spatial Audio) simulates the sound transmission effect in a real three-dimensional space through a technical means, so that a listener can perceive the direction, the distance and the motion track of the sound to create an immersive hearing experience, and the core is a mechanism for restoring the positioning of the human ear to the sound through a Head Related Transfer Function (HRTF). In the related art, a mobile phone, a game machine, smart glasses, VR (Virtual Reality) devices, etc., implement haptic feedback using a motor. However, such haptic feedback can generally only achieve single vibration feedback, but is not combined with spatial information of audio, resulting in user hearing and haptic rupture, and the actual haptic feedback effect is not good, especially, sounds at different distance positions cannot be distinguished, and user experience is affected. Based on this, a solution is needed that enables haptic feedback in combination with audio spatial information, optimizing the user experience. Disclosure of Invention The application mainly aims to provide a multi-mode cooperative processing method, a control device and terminal equipment, and aims to solve the technical problem that haptic feedback effects at different distance positions are poor due to the fact that audio space information and haptic feedback are not combined. In one aspect, a multi-mode collaborative processing method is provided, including the following steps: Acquiring an audio signal; Extracting a target vibration signal in the audio signal, carrying out spatial information analysis on the target vibration signal, and determining the frequency spectrum characteristics of the target vibration signal; determining target distance information in the spatial information of the sound source based on the spectral characteristics of the target vibration signal; Determining a haptic feedback parameter corresponding to spatial information of the sound source; Generating a driving signal for driving the vibration unit to work based on the mapping relation between the spatial information of the sound source and the haptic feedback parameter to generate haptic feedback In an embodiment, the determining the target distance information in the spatial information of the sound source based on the spectral characteristics of the target vibration signal includes: calculating the energy ratio of a high-frequency signal and a low-frequency signal in the target vibration signal based on the frequency spectrum characteristics of the target vibration signal, and determining target distance information in the spatial information of the sound source; the generating a driving signal for driving the vibration unit to work so as to generate haptic feedback based on the mapping relation between the spatial information of the sound source and the haptic feedback parameter comprises: Determining the working strength of the motor according to the target distance information of the sound source based on the mapping relation between the space information of the sound source and the tactile feedback parameters; A driving signal for driving the vibration unit to operate at a corresponding operating intensity to generate haptic feedback is generated. In an embodiment, the determining the target distance information in the spatial information of the sound source based on the spectral characteristics of the target vibration signal includes: calculating the change trend of the target vibration signal along with time based on the frequency spectrum characteristics of the target vibration signal, and determining target distance information in the spatial information of the sound source; the generating a driving signal for driving the vibration unit to work so as to generate haptic feedback based on the mapping relation between the spatial information of the sound source and the haptic feedback parameter comprises: Determining the working strength of the motor according to the target distance information of the sound source based on the mapping relation between the space information of the sound source and the tactile feedback parameters; A driving signal for driving the vibration unit to operate at a corresponding operating intensity to generate haptic feedback is generated. In an embodiment, the vibration unit includes a plurality of motors, and the haptic feedback parameters include motor positions, motor operation intensities, and motor operation frequencies corresponding to the respective motors; the generating a driving signal for driving the vibration unit to work so as to generate haptic feedback based on the mapping relati