CN-122002208-A - Acoustic ray tracing method, acoustic ray tracing device, electronic equipment, wearable equipment and storage medium

CN122002208ACN 122002208 ACN122002208 ACN 122002208ACN-122002208-A

Abstract

The acoustic ray tracing method, the device, the electronic equipment, the wearable equipment and the storage medium provided by the invention are used for constructing a reflection tree according to a reporting path by determining collision information and the reporting path corresponding to rays sent by a listener position, determining a plurality of first propagation paths from the sound source position to the listener position according to the collision information and the sound source position for each sound source, determining a plurality of second propagation paths from the sound source position to the listener position according to the reflection tree, determining an echo map corresponding to the first propagation path for each sound source, determining an echo map corresponding to the second propagation path, determining a filter corresponding to the sound source according to the echo map, and processing the non-spatialization audio stream of the corresponding sound source according to the filter to determine a reverberation signal.

Inventors

LIU XUESONG
YUE CHENGRUI

Assignees

万有引力（宁波）电子科技有限公司

Dates

Publication Date: 20260508
Application Date: 20241107

Claims (15)

1. An acoustic ray tracing method, comprising: Determining collision information and a reporting path corresponding to light rays sent by a listener position, and constructing a reflecting tree according to the reporting path, wherein the collision information is information of each collision point obtained by multiple collisions between the light rays and a scene model; Determining, for each sound source, a plurality of first propagation paths from the sound source position to the listener position according to the collision information and the sound source position, and a plurality of second propagation paths from the sound source position to the listener position according to the reflection tree; for each sound source, determining an echo map corresponding to the first propagation path, and determining an echo map corresponding to the second propagation path; and determining a filter corresponding to the sound source according to the echo diagram, and processing the non-spatialization audio stream of the corresponding sound source according to the filter corresponding to each sound source to determine a reverberation signal.
2. The method of claim 1, wherein determining collision information and a reporting path corresponding to light rays emitted from a listener location, and constructing a reflection tree according to the reporting path, comprises: When at least one of the following is satisfied, executing the collision information and the reporting path corresponding to the light rays sent by the listener position, and constructing a reflection tree according to the reporting path: the listener position changes, the scene model changes, and the reflectivity of any reflecting surface in the scene model changes.
3. The method of claim 1, wherein the scene model is represented by triangulation, the method further comprising: Determining diffraction edges in the scene model, wherein the diffraction edges are coincident edges of two non-coplanar triangles; Determining and storing diffraction edge information, wherein the diffraction edge information corresponds to the diffraction edge, and when dynamic geometry in the scene model changes, the corresponding diffraction edge information is updated; correspondingly, determining collision information and reporting paths corresponding to the light rays sent by the listener position comprises the following steps: and carrying out ray tracing on rays emitted from the listener position, determining collision information, and determining the reporting path according to the diffraction edge information.
4. The method of claim 3, wherein the reporting path includes a reflection path and a diffraction path, wherein performing ray tracing on rays emanating from a listener location, determining collision information, and determining the reporting path based on the diffraction edge information, comprises: For each ray, repeating the following steps to determine collision information of each collision of the ray until reaching the condition of ending tracking the ray: When the light collides with the scene model, determining collision information of the collision; When the light ray is determined not to be finished being traced, determining the reflection path, and calculating the direction of the reflected light ray so as to determine whether collision with another reflection surface in the scene model can occur according to the direction of the reflected light ray; And determining whether diffraction occurs according to the diffraction edge information when the light is determined not to be finished being traced, determining the diffraction path when diffraction occurs, and calculating the direction of the diffracted light so as to determine whether collision with another reflecting surface in the scene model occurs according to the direction of the diffracted light.
5. A method according to claim 3, wherein determining whether diffraction will occur based on the diffraction edge information comprises: determining the collision point of the light ray and the reflecting surface; Determining whether the collision point is close to the edge of the reflector corresponding triangle by determining whether the edge is the saved diffraction edge according to the barycentric coordinates of the collision point after conversion; If the edge is one of a plurality of saved diffraction edges, two shadow areas are determined according to two reflection surfaces corresponding to the diffraction edges, and diffraction is determined to occur when the light rays are in any shadow area.
6. A method according to claim 3, wherein constructing a reflection tree from the reporting path comprises: The reporting path is subjected to de-duplication processing to obtain a path after de-duplication processing, wherein the reporting path comprises a starting node and a terminating node, and the starting node or the terminating node is a reflecting surface or a diffraction edge; determining a listener as a root node of the reflection tree, and repeatedly executing the following steps until the maximum depth of the reflection tree reaches a preset depth: For each node in the reflection tree, a target node to which the node can be connected is searched from all paths after the de-duplication processing, and the target node is determined as a child node of the node.
7. The method of claim 6, wherein the method further comprises: When the reflection tree is determined, determining a spatial position corresponding to a root node as a listener position when the node is the root node; when the node is a reflecting surface, determining a mirror image position of a space position corresponding to a father node of the node relative to the reflecting surface, and determining the mirror image position as the space position corresponding to the node; And when the node is a diffraction edge, determining the central position of the diffraction edge as a diffraction point or a space position corresponding to the node.
8. The method of claim 7, wherein the reflection paths in the second propagation paths are specular reflection paths, wherein determining a plurality of second propagation paths from the sound source location to the listener location based on the reflection tree comprises: determining, for each sound source, a reflecting surface and a diffraction edge of a first collision between a ray emitted from the sound source position and the scene model, so as to obtain a sound source visible node list; for each leaf node in the reflection tree, determining whether a propagation path corresponding to the leaf node is a legal path when the leaf node is in the sound source visible list; when the leaf nodes are not in the sound source visible list, determining that propagation paths corresponding to the leaf nodes are illegal paths; And determining the second propagation path according to the determined legal path, wherein the second propagation path is a path obtained by sequentially connecting the sound source position, reflection points or diffraction points corresponding to all nodes in the legal path and the listener position.
9. The method of claim 8, wherein determining whether the propagation path corresponding to the leaf node is a legal path comprises: When each node in the child nodes from the leaf node to the root node meets a target condition, determining that the path from the leaf node to the root node is a legal path, otherwise, determining that the path is an illegal path; when the node is a reflecting surface, the target condition is that a reflecting point corresponding to the reflecting surface is in a triangle corresponding to the reflecting surface and a corresponding path is not blocked, wherein the position of the reflecting point is related to the space position corresponding to the reflecting surface and the space position of a next-stage node; When the node is a diffraction edge, the target condition is that the corresponding path is not blocked, and the node at the previous stage and the node at the next stage of the node are respectively positioned in two different shadow areas corresponding to the node.
10. The method of claim 9, wherein determining an echo map corresponding to the second propagation path comprises: determining initial energy corresponding to the sound source aiming at the sound source; Determining a distance attenuation coefficient and an air absorption coefficient according to the total length of each second propagation path, determining a reflection attenuation coefficient of each reflecting surface in the second propagation path, multiplying the reflection attenuation coefficients of the reflecting surfaces to obtain an overall reflection attenuation coefficient, and determining a diffraction coefficient according to diffraction edge information of each diffraction edge in the second propagation path, wherein the reflection attenuation coefficient of the reflecting surface is related to the reflectivity and the scattering rate of the reflecting surface; determining the multiplication result of the initial energy, the distance attenuation coefficient, the air absorption coefficient, the integral reflection attenuation coefficient and the diffraction coefficient as listener received energy; determining a direction of the last node pointing to the listener position as a receiving direction; And determining the received energy of the listener, the receiving direction and the propagation time corresponding to the second propagation path as an echo diagram corresponding to the second propagation path.
11. An acoustic ray tracing apparatus, comprising: The listener tracking module is used for determining collision information and a reporting path corresponding to light rays sent by the listener position, wherein the collision information is information of each collision point obtained by multiple collisions between the light rays and the scene model; the reflection tree construction module is used for constructing a reflection tree according to the reporting path, wherein the reporting path is a path formed by a reflection surface or a diffraction edge corresponding to the collision point; a sound source tracking module for determining, for each sound source, a plurality of first propagation paths from the sound source position to the listener position based on the collision information and the sound source position; A path search module for determining a plurality of second propagation paths from the sound source position to the listener position according to the reflection tree, wherein the second propagation paths are combined paths of specular reflection paths and diffraction paths; The sound source tracking module is further configured to determine, for each sound source, an echo map corresponding to the first propagation path; the path modeling module is used for determining an echo diagram corresponding to the second propagation path; a filter synthesis module, configured to determine a filter corresponding to the sound source according to the echo map; And the processing module is used for processing the non-spatialization audio streams of the corresponding sound sources according to the filters corresponding to the sound sources so as to determine reverberation signals.
12. An electronic device comprising at least one processor and a memory; The memory stores computer-executable instructions; The at least one processor executing computer-executable instructions stored in the memory causes the at least one processor to perform the method of any one of claims 1 to 10.
13. A wearable device, characterized by comprising a processing unit for performing the method of any of claims 1 to 10.
14. A computer readable storage medium having stored therein computer executable instructions which when executed by a processor implement the method of any one of claims 1 to 10.
15. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1 to 10.

Description

Acoustic ray tracing method, acoustic ray tracing device, electronic equipment, wearable equipment and storage medium Technical Field The present invention relates to the field of audio signal processing technologies, and in particular, to an acoustic ray tracing method, an acoustic ray tracing device, an electronic device, a wearable device, and a storage medium. Background Spatial audio technology is a technology that uses a user as a center and processes sounds that do not contain any spatial features so that they appear to have a certain spatial feature, making the content heard by the user more realistic. In a virtual reality application or a mixed reality application, spatial audio technology is used, so that the spatial characteristics of sound can be matched with visual content, and better immersion can be obtained. Through real-time acoustic modeling in a virtual reality application or a mixed reality application, the space audio can be matched with the acoustic characteristics of a set space (virtual space or real space where a user is located), so that the content heard by the user can be matched with visual content, and a stronger sense of reality is provided. For example, when the sound source changes, the content heard by the listener will also change. In general, in real-time acoustic modeling, an acoustic path from a sound source to a listener is modeled based on geometric characteristics of a current space, acoustic parameters, a sound source, and a listener position using an acoustic ray tracing method, and an undespatial audio stream is processed according to a modeling result to obtain an audio stream matching an expected acoustic characteristic. However, the ray tracing method can equivalent the propagation of sound to the movement of particles, neglecting the diffraction effect of sound waves, and when the shielding exists between a sound source and a listener, the diffraction of sound waves is an important path of propagation between the sound source and the listener, and if modeling of diffraction is absent, the determined spatial audio is deficient in reality. For example, in the presence of an occlusion between a sound source and a listener, sound from the sound source may not be fully heard in the simulated audio, and when the listener moves to a location where the sound source is visible, the sound may suddenly appear, which can severely impact the continuity and immersion of the spatial audio. Therefore, how to model the diffraction of the sound propagation process to determine the accurate reverberation signal is a technical problem to be solved urgently. Disclosure of Invention The invention provides an acoustic ray tracing method, an acoustic ray tracing device, electronic equipment, wearable equipment and a storage medium, which are used for modeling reflection, scattering and diffraction in a sound propagation process so as to accurately determine a reverberation signal. In a first aspect, the present invention provides an acoustic ray tracing method, comprising: Determining collision information and a reporting path corresponding to light rays sent by a listener position, and constructing a reflecting tree according to the reporting path, wherein the collision information is information of each collision point obtained by multiple collisions between the light rays and a scene model; Determining, for each sound source, a plurality of first propagation paths from the sound source position to the listener position according to the collision information and the sound source position, and a plurality of second propagation paths from the sound source position to the listener position according to the reflection tree; for each sound source, determining an echo map corresponding to the first propagation path, and determining an echo map corresponding to the second propagation path; and determining a filter corresponding to the sound source according to the echo diagram, and processing the non-spatialization audio stream of the corresponding sound source according to the filter corresponding to each sound source to determine a reverberation signal. Optionally, determining collision information and a reporting path corresponding to the light rays sent by the listener position, and constructing a reflection tree according to the reporting path includes executing the determination of the collision information and the reporting path corresponding to the light rays sent by the listener position when at least one of the following is satisfied, and constructing the reflection tree according to the reporting path, wherein the listener position is changed, the scene model is changed, and the reflectivity of any reflecting surface in the scene model is changed. Optionally, the scene model is represented by triangulation, and the method further comprises: Determining diffraction edges in the scene model, wherein the diffraction edges are coincident edges of two non-coplanar triangles; Determi