CN-122002207-A - Acoustic ray tracing method, acoustic ray tracing device, electronic equipment, wearable equipment and storage medium

Abstract

According to the acoustic ray tracing method, the device, the electronic equipment, the wearable equipment and the storage medium, collision information corresponding to rays sent by the listener position is determined, each propagation path from the sound source position to the listener position is determined according to the collision information and the sound source position for each sound source, an echo diagram corresponding to the sound source is determined according to each propagation path, a filter corresponding to the sound source is determined according to the echo diagram, the non-spatialization audio streams of the corresponding sound source are processed according to the filters corresponding to each sound source to determine reverberation signals, the tracing path is divided into paths from the listener to the last collision point during inverse tracing, and the paths from the last collision point to the sound source are respectively modeled, so that when the listener position is not changed, the step of determining the collision information is not needed, and when the number of the sound sources is multiple, the step of determining the collision information is calculated only once, and therefore the calculated amount is effectively reduced.

Inventors

• LIU XUESONG
• YUE CHENGRUI

Assignees

• 万有引力（宁波）电子科技有限公司

Dates

Publication Date

20260508

Application Date

20241107

Claims (20)

1. 1. An acoustic ray tracing method, comprising: determining collision information corresponding to light rays emitted by the listener, wherein the collision information is information of each collision point obtained by multiple collisions between the light rays and the scene model; For each sound source, determining respective propagation paths from the sound source position to the listener position according to the collision information and the sound source position, and determining an echo map corresponding to the sound source according to the respective propagation paths; Determining a filter corresponding to the sound source according to the echo diagram; And processing the non-spatialization audio streams of the corresponding sound sources according to the filters corresponding to the sound sources so as to determine reverberation signals.
2. 2. The method of claim 1, wherein determining collision information corresponding to light rays emitted from the listener's location comprises: performing a determination of collision information corresponding to the rays of light emitted by the listener's location when at least one of the following is satisfied: the listener position changes, the scene model changes, and the reflectivity of any reflecting surface in the scene model changes.
3. 3. The method of claim 1, wherein determining collision information corresponding to light rays emitted from the listener's location comprises: determining collision information of a target number of rays, wherein the number of rays emitted by a listener position is N; wherein the target number is less than N when the listener position is not significantly changed, and/or is equal to N when the listener position is significantly changed.
4. 4. A method according to claim 3, characterized in that the method further comprises: Determining first collision information of N rays emitted from the listener position, wherein the first collision information comprises reflection surfaces of the N rays subjected to first collision, and the reflection surfaces subjected to the first collision are reflection surfaces subjected to de-duplication treatment; Determining the change rate of the reflecting surface according to the reflecting surface of the first collision simulated at this time and the reflecting surface of the first collision simulated at history; Determining whether the listener position is subject to insignificant change based on the rate of change of the reflecting surface and an external reset condition.
5. 5. The method of claim 4, wherein the rate of change of the reflecting surface comprises a new rate and a deleted rate, wherein determining whether the listener position has changed non-significantly based on the rate of change of the reflecting surface and an external reset condition comprises: When the new rate is less than a first threshold and the deletion rate is less than a second threshold and the external reset condition is not triggered, determining that the listener position is subject to insignificant change; and/or determining that the listener position has changed significantly when at least one of: the new rate is greater than or equal to a first threshold, the deletion rate is greater than or equal to a second threshold, and the external reset condition is triggered; The new increasing rate is related to the number of the reflecting surfaces of the first collision, which are newly increased compared with the current simulation and the historical simulation, and the deleting rate is related to the number of the reflecting surfaces of the first collision, which are deleted compared with the current simulation and the historical simulation.
6. 6. The method according to claim 4, wherein the method further comprises: determining that the external reset condition is triggered when at least one of: the scene model is changed or the reflectivity of any reflecting surface in the scene model is changed, the distance moved by the listener position is more than a preset distance compared with the historical simulation in the simulation, and the continuous multiple simulations do not trigger significant changes.
7. 7. The method of claim 5, wherein determining collision information for a target number of rays when the listener position is not significantly changed comprises: Determining the target quantity according to the change rate of the reflecting surface, wherein the target quantity is positively correlated with the change rate of the reflecting surface; and selecting the target quantity of light rays from N light rays emitted from the listener position, and determining collision information of the target quantity of light rays.
8. 8. The method of claim 7, wherein selecting the target number of rays from the N rays emitted from the listener location comprises: When the simulation is compared with the history simulation, all the light rays passing through the newly added reflecting surface of the first collision are reserved; And when the number of the reserved light rays is smaller than the target number, selecting a plurality of light rays from the rest light rays in the N light rays to obtain the target number of light rays.
9. 9. The method of claim 7, wherein determining collision information for the target number of rays comprises: For each ray, repeating the following steps to determine collision information of each collision of the ray until reaching the condition of ending tracking the ray: when the light collides with a reflecting surface in the scene model, determining collision information of the collision; Determining the energy of the residual light according to the reflectivity of the reflecting surface, determining the distance between the current collision point and the last collision point to obtain the propagation distance of the light, and determining whether to finish tracking the light according to the energy of the residual light, the propagation distance of the light and the collision times of the light; And when the light ray is not finished to be traced, determining the direction of the reflected light ray according to the scattering rate of the reflecting surface, and determining whether the reflected light ray collides with another reflecting surface in the scene model according to the direction of the reflected light ray.
10. 10. The method of claim 9, wherein determining the direction of the reflected light based on the scattering rate of the reflective surface comprises: determining a specular reflection direction according to the incidence direction of the light and the normal vector of the reflection surface; and determining a random scattering direction, and determining the direction of the reflected light according to the random scattering direction, the scattering rate of the reflecting surface and the specular reflection direction.
11. 11. The method according to any one of claims 1 to 10, wherein the collision information includes a position of a collision point, a reflection surface where the collision point is located, determining respective propagation paths from the sound source position to the listener position based on the collision information and the sound source position, and determining an echo map corresponding to the sound source based on the respective propagation paths, comprising: For rays corresponding to each collision point, when it is determined that no obstruction exists between the position of the collision point and the sound source position according to the scene model, determining a path between the position of the collision point and the sound source position to determine respective propagation paths of the sound source position to the listener position; Determining an echo diagram corresponding to the propagation path according to the propagation path and the reflectivity and scattering rate of a reflecting surface where an collision point is located, wherein the echo diagram comprises sound source receiving energy, arrival time of light reaching the sound source and arrival direction, and the arrival direction is the opposite direction of the light emitted from the listener position corresponding to the collision point; and determining an echo diagram corresponding to any sound source according to the echo diagrams corresponding to the propagation paths.
12. 12. The method of claim 11, wherein the collision information further comprises a specular reflection angle and a cumulative propagation distance of the light, wherein determining the sound source received energy corresponding to the propagation path based on the propagation path, the reflectivity and the scattering rate of the reflecting surface on which the collision point is located, comprises: Determining the type of propagation path between the position of the collision point and the position of the sound source according to the specular reflection angle of the light ray; Adding the accumulated propagation distance and the distance between the collision point and the sound source position to determine the total propagation distance between the sound source position and the listener position of the light ray, and determining an air absorption coefficient according to the total propagation distance; When the type of the propagation path is a specular reflection path, determining the sound source receiving energy according to the air absorption coefficient, the reflectivity and the scattering rate of a reflection surface where an impact point is located and the energy currently carried by light rays; When the type of the propagation path is a scattering path, the received energy of the sound source is determined according to the air absorption coefficient, the reflectivity and scattering rate of the reflecting surface where the collision point is located, the scattering proportion and the current carried energy of the light, wherein the scattering proportion is related to the included angle between the light from the collision point to the sound source and the normal vector of the reflecting surface, the distance from the position of the collision point to the center of the sound source and the radius of the sound source.
13. 13. The method of claim 11, wherein after determining an echo map corresponding to the propagation path based on the propagation path, the reflectivity and the scattering rate of the reflecting surface on which the collision point is located, the method further comprises: Updating the light energy according to the residual light energy after the light is reflected and the sound source receiving energy; When the updated light energy does not reach the lower limit of the light energy, if a next collision point exists behind the collision point, determining a path between the position of the next collision point and the position of the sound source, and calculating an echo diagram; And stopping calculating the subsequent collision points of the collision points when the updated light energy reaches the lower limit of the light energy.
14. 14. A method according to claim 3, wherein determining a filter corresponding to the sound source from the echo map comprises: determining a smoothing coefficient according to the target quantity, wherein the smoothing coefficient represents the reserved proportion of a first filter in the simulation; determining a second filter corresponding to the sound source according to the echo diagram; And determining a final filter generated by the simulation according to the smoothing coefficient, the first filter and the second filter for each sound source, wherein the second filter represents a filter determined by the simulation based on tracking the light rays of the target quantity, and the smoothing coefficient is inversely related to the target quantity.
15. 15. The method of claim 14, wherein the smoothing factor is a value between 0 and 1, wherein determining a final filter generated by the simulation based on the smoothing factor, the first filter, and the second filter comprises: Determining a compensation parameter according to the smoothing coefficient, wherein the compensation parameter is a numerical value larger than 1 and is related to the square value of the smoothing coefficient; calculating a first product of the smoothing coefficient and the first filter, calculating a difference value between a numerical value 1 and the smoothing coefficient, and calculating a second product of the difference value and the second filter; and calculating a summation result of the first product and the second product, and determining a multiplication result of the compensation parameter and the summation result as a final filter generated by the simulation.
16. 16. The method of claim 1, wherein determining collision information corresponding to light rays emitted from the listener's location comprises: Determining collision information and a reporting path corresponding to light rays sent by a listener position, and constructing a reflecting tree according to the reporting path, wherein the reporting path is a path formed by reflecting surfaces or diffraction edges corresponding to the collision points; Accordingly, for each sound source, determining respective propagation paths from the sound source position to the listener position according to the collision information and the sound source position, and determining an echo map corresponding to the sound source according to the respective propagation paths, including: Determining, for each sound source, a plurality of first propagation paths from the sound source position to the listener position according to the collision information and the sound source position, and a plurality of second propagation paths from the sound source position to the listener position according to the reflection tree; for each sound source, an echo map corresponding to the first propagation path is determined, and an echo map corresponding to the second propagation path is determined.
17. 17. An acoustic ray tracing apparatus, comprising: the listener tracking module is used for determining collision information corresponding to light rays sent by the listener position, wherein the collision information is information of each collision point obtained by multiple collisions between the light rays and the scene model; A sound source tracking module, configured to determine, for each sound source, respective propagation paths from the sound source position to the listener position according to the collision information and the sound source position, and determine echo maps corresponding to the sound sources according to the respective propagation paths; a filter synthesis module, configured to determine a filter corresponding to the sound source according to the echo map; And the processing module is used for processing the non-spatialization audio streams of the corresponding sound sources according to the filters corresponding to the sound sources so as to determine reverberation signals.
18. 18. An electronic device comprising at least one processor and a memory; The memory stores computer-executable instructions; the at least one processor executing computer-executable instructions stored in the memory causes the at least one processor to perform the method of any one of claims 1 to 16.
19. 19. A wearable device, characterized by comprising a processing unit for performing the method of any of claims 1 to 16.
20. 20. A computer readable storage medium having stored therein computer executable instructions which when executed by a processor implement the method of any one of claims 1 to 16.

Description

Acoustic ray tracing method, acoustic ray tracing device, electronic equipment, wearable equipment and storage medium Technical Field The present invention relates to the field of audio signal processing technologies, and in particular, to an acoustic ray tracing method, an acoustic ray tracing device, an electronic device, a wearable device, and a storage medium. Background Spatial audio technology is a technology that uses a user as a center and processes sounds that do not contain any spatial features so that they appear to have a certain spatial feature, making the content heard by the user more realistic. In a virtual reality application or a mixed reality application, spatial audio technology is used, so that the spatial characteristics of sound can be matched with visual content, and better immersion can be obtained. Through real-time acoustic modeling in a virtual reality application or a mixed reality application, the space audio can be matched with the acoustic characteristics of a space (virtual space or a real space where a user is located) set by a lock, so that the content heard by the user can be matched with visual content, and stronger sense of reality is provided. For example, when the sound source changes, the content heard by the listener will also change. In general, in real-time acoustic modeling, an acoustic path from a sound source to a listener is modeled based on geometric characteristics of a current space, acoustic parameters, a sound source, and a listener position using an acoustic ray tracing method, and an undespatial audio stream is processed according to a modeling result to obtain an audio stream matching an expected acoustic characteristic. In the case of performing ray tracing, since each ray emitted from the sound source position needs to be traced and each ray needs to be reflected multiple times in the room, the calculation amount of the ray tracing algorithm is large, and how to reduce the calculation amount of the ray tracing algorithm is a technical problem to be solved. Disclosure of Invention The invention provides an acoustic ray tracing method, an acoustic ray tracing device, electronic equipment, wearable equipment and a storage medium, which are used for effectively reducing the calculated amount of a ray tracing algorithm. In a first aspect, the present invention provides an acoustic ray tracing method, comprising: determining collision information corresponding to light rays emitted by the listener, wherein the collision information is information of each collision point obtained by multiple collisions between the light rays and the scene model; For each sound source, determining respective propagation paths from the sound source position to the listener position according to the collision information and the sound source position, and determining an echo map corresponding to the sound source according to the respective propagation paths; Determining a filter corresponding to the sound source according to the echo diagram; And processing the non-spatialization audio streams of the corresponding sound sources according to the filters corresponding to the sound sources so as to determine reverberation signals. Optionally, determining collision information corresponding to the light emitted by the listener position includes: performing a determination of collision information corresponding to the rays of light emitted by the listener's location when at least one of the following is satisfied: the listener position changes, the scene model changes, and the reflectivity of any reflecting surface in the scene model changes. Optionally, determining collision information corresponding to the light emitted by the listener position includes: determining collision information of a target number of rays, wherein the number of rays emitted by a listener position is N; wherein the target number is less than N when the listener position is not significantly changed, and/or is equal to N when the listener position is significantly changed. Optionally, the method further comprises: Determining first collision information of N rays emitted from the listener position, wherein the first collision information comprises reflection surfaces of the N rays subjected to first collision, and the reflection surfaces subjected to the first collision are reflection surfaces subjected to de-duplication treatment; Determining the change rate of the reflecting surface according to the reflecting surface of the first collision simulated at this time and the reflecting surface of the first collision simulated at history; Determining whether the listener position is subject to insignificant change based on the rate of change of the reflecting surface and an external reset condition. Optionally, the rate of change of the reflecting surface includes a new rate and a deleted rate, and determining whether the listener position is subject to an insignificant change based on the rate of change of t