US-20260129392-A1 - SPATIAL AUDIO RENDERING

US20260129392A1US 20260129392 A1US20260129392 A1US 20260129392A1US-20260129392-A1

Abstract

A method, including: generating a bitstream configured to define a six-degrees of freedom rendering, the bitstream including: a six degrees of freedom audio scene; and information configured to define at least one rendering mode, the information including: an identifier configured to identify the at least one rendering mode; and at least one rendering modification associated with the at least one rendering mode to be applied by a renderer when rendering the six degrees of freedom audio scene when the at least one rendering mode is selected at the renderer.

Inventors

Jussi Artturi Leppänen
Sujeet Shyamsundar Mate
Arto Juhani Lehtiniemi

Assignees

NOKIA TECHNOLOGIES OY

Dates

Publication Date: 20260507
Application Date: 20230913
Priority Date: 20221010

Claims (20)

1 . A method, comprising: obtaining a bitstream configured to define a six-degrees of freedom rendering, the bitstream comprising a six degrees of freedom audio scene; obtaining information configured to define at least one rendering modification parameter; rendering the bitstream to generate at least two output audio signals, wherein the rendering is configured to be modified based on the at least one rendering modification parameter; and controlling the outputting based on the at least two output audio signals using the at least one rendering modification parameter.
2 . The method as claimed in claim 1 , wherein the information comprising at least one rendering modification to be applied with a renderer when rendering an output audio signal from the six degrees of freedom audio stream when at least one rendering mode is selected at the renderer comprises the at least one rendering modification parameter, the at least one rendering modification parameter being configured to control a modification of at least one rendering process at the renderer.
3 . The method as claimed in claim 2 , wherein the at least one rendering modification parameter is further configured to control a modification of at least one default rendering process at the renderer, wherein the default rendering process is applied with the renderer when rendering of the six degrees of freedom audio scene when the at least one rendering mode is not selected.
4 . The method as claimed in claim 2 , wherein the at least one rendering modification parameter comprises at least one of: a reverberation modification configured to selectively enable reverberation for at least one audio source within the six degrees of freedom audio scene; a reflections modification configured to selectively enable reflections for at least one audio source within the six degrees of freedom audio scene; an occlusion modification configured to selectively enable occlusions for at least one audio source within the six degrees of freedom audio scene; a diffraction modification configured to selectively enable diffraction for at least one audio source within the six degrees of freedom audio scene; a heterogenous extent modification configured to selectively enable heterogenous propagation for at least one audio source within the six degrees of freedom audio scene; a homogenous extent modification configured to selectively enable homogenous propagation for at least one audio source within the six degrees of freedom audio scene; a portals modification configured to selectively enable portals for at least one audio source within the six degrees of freedom audio scene; a distance gain modification configured to selectively enable distance gains for at least one audio source within the six degrees of freedom audio scene; or a doppler modification configured to selectively enable doppler effects for at least one audio source within the six degrees of freedom audio scene.
5 . The method as claimed in claim 4 , wherein the at least one rendering modification parameter comprises at least one of: a disable effect modification configured to disable at least one rendering process; an attenuate effect modification configured to attenuate at least one rendering process; or an enhance effect modification configured to enhance at least one rendering process.
6 . The method as claimed in claim 1 , wherein the bitstream configured to define the six-degrees of freedom rendering comprises at least one of: receiving the information in an encoder input file format and generating an encoded MPEG-I format bitstream to be combined with an encoded six degrees of freedom audio scene bitstream; or obtaining the information in an MPEG-I format and combining the information with an encoded six degrees of freedom audio scene bitstream.
7 . A method, comprising: obtaining a bitstream configured to define a six-degrees of freedom rendering, the bitstream comprising a six degrees of freedom audio scene; obtaining information configured to define at least one rendering mode, the information comprising: an identifier configured to identify the at least one rendering mode; and at least one rendering modification associated with the at least one rendering mode; obtaining information identifying a desired rendering mode; rendering the bitstream to generate at least two output audio signals from the bitstream configured to define a six-degrees of freedom audio rendering, wherein the rendering is modified based on the at least one rendering modification associated with a selected one of the at least one rendering mode, the selected one of the at least one rendering mode being selected based on the information identifying the desired rendering mode; and controlling the outputting of the at least two output audio signals.
8 . The method as claimed in claim 7 , wherein the information comprising the at least one rendering modification associated with the at least one rendering mode comprises at least one modification parameter, wherein rendering the bitstream to generate at least two output audio signals from the bitstream comprises rendering the bitstream based on the at least one modification parameter controlling a modification of at least one rendering process.
9 . The method as claimed in claim 8 , wherein the at least one modification parameter is configured to control a modification of at least one default rendering process, wherein rendering the bitstream to generate at least two output audio signals from the bitstream configured to define a six-degrees of freedom audio rendering comprises rendering the bitstream with applying the default rendering process when the at least one rendering mode is not selected.
10 . The method as claimed in claim 9 , further comprising: determining the default rendering process based on: the bitstream configured to define a six-degrees of freedom rendering; and at least one renderer defined value.
11 . The method as claimed in claim 8 , wherein the at least one modification parameter comprises at least one of: a reverberation modification configured to selectively enable reverberation for at least one audio source within the six degrees of freedom audio scene; a reflections modification configured to selectively enable reflections for at least one audio source within the six degrees of freedom audio scene; an occlusion modification configured to selectively enable occlusions for at least one audio source within the six degrees of freedom audio scene; a diffraction modification configured to selectively enable diffraction for at least one audio source within the six degrees of freedom audio scene; a heterogenous extent modification configured to selectively enable heterogenous propagation for at least one audio source within the six degrees of freedom audio scene; a homogenous extent modification configured to selectively enable homogenous propagation for at least one audio source within the six degrees of freedom audio scene; a portals modification configured to selectively enable portals for at least one audio source within the six degrees of freedom audio scene; a distance gain modification configured to selectively enable distance gains for at least one audio source within the six degrees of freedom audio scene; or a doppler modification configured to selectively enable doppler effects for at least one audio source within the six degrees of freedom audio scene.
12 . The method as claimed in claim 11 , wherein the at least one modification parameter comprises at least one of: a disable effect modification configured to disable at least one rendering process; an attenuate effect modification configured to attenuate at least one rendering process; or an enhance effect modification configured to enhance at least one rendering process.
13 . The method as claimed in claim 7 , wherein obtaining information configured to define at least one rendering mode comprises obtaining at least one predetermined information prior to the obtaining of the bitstream.
14 . The method as claimed in claim 13 , wherein obtaining information configured to define at least one rendering mode comprises receiving at least one further at least one information configured to define at least one rendering mode, wherein the received at least one further at least one information configured to define at least one rendering mode supersedes the at least one predetermined information configured to define at least one rendering mode.
15 . The method as claimed in claim 7 , wherein the bitstream further comprises the information configured to define the at least one rendering mode wherein obtaining information configured to define at least one rendering mode comprises obtaining the information from the bitstream.
16 . The method as claimed in claim 7 , wherein the information configured to define the at least one rendering mode is in an encoder input format.
17 . The method as claimed in claim 7 , wherein obtaining information identifying a desired rendering mode comprises obtaining an input from a user interface identifying the desired rendering mode.
18 - 19 . (canceled)
20 . An apparatus, comprising: at least one processor; and at least one memory storing instructions that, when executed with the at least one processor, cause the apparatus at least to: generate a bitstream configured to define a six-degrees of freedom rendering, the bitstream comprising: a six degrees of freedom audio scene; and information configured to define at least one rendering mode, the information comprising: an identifier configured to identify the at least one rendering mode; and at least one rendering modification associated with the at least one rendering mode to be applied with a renderer when rendering the six degrees of freedom audio scene when the at least one rendering mode is selected at the renderer.
21 . An apparatus, comprising: at least one processor; and at least one memory storing instructions that, when executed with the at least one processor, cause the apparatus at least to: obtain a bitstream configured to define a six-degrees of freedom rendering, the bitstream comprising a six degrees of freedom audio scene; obtain information configured to define at least one rendering mode, the information comprising: an identifier configured to identify the at least one rendering mode; and at least one rendering modification associated with the at least one rendering mode; obtain information identifying a desired rendering mode; render the bitstream to generate at least two output audio signals from the bitstream configured to define a six-degrees of freedom audio rendering, wherein the rendering is modified based on the at least one rendering modification associated with a selected one of the at least one rendering mode, the selected one of the at least one rendering mode being selected based on the information identifying the desired rendering mode; and control the outputting of the at least two output audio signals.

Description

FIELD The present application relates to apparatus and methods for spatial audio rendering which employ selectable rendering modes, but not exclusively for spatial audio rendering which employ selectable rendering modes in augmented reality and/or virtual reality apparatus. BACKGROUND Spatial audio capture approaches attempt to capture an audio environment such that the audio environment can be perceptually recreated to a listener in an effective manner and furthermore may permit a listener to move and/or rotate within the recreated audio environment. For example in some systems (3 degrees of freedom—3DoF) the listener may rotate their head and the rendered audio signals reflect this rotation motion. In some systems (3 degrees of freedom plus—3DoF+) the listener may ‘move’ slightly within the environment as well as rotate their head and in others (6 degrees of freedom—6DoF) the listener may freely move within the environment and rotate their head. Rendering is a process wherein the captured audio signals (or transport audio signals derived from the captured audio signals) and parameters are processed to produce a suitable output for outputting to a listener, for example via headphones or loudspeakers or any suitable audio transducer. SUMMARY There is provided according to a first aspect a method comprising: generating a bitstream configured to define a six-degrees of freedom rendering, the bitstream comprising: a six degrees of freedom audio scene; and information configured to define at least one rendering mode, the information comprising: an identifier configured to identify the at least one rendering mode; and at least one rendering modification associated with the at least one rendering mode to be applied by a renderer when rendering the six degrees of freedom audio scene when the at least one rendering mode is selected at the renderer. The information comprising at least one rendering modification to be applied by a renderer when rendering an output audio signal from the six degrees of freedom audio stream when the at least one rendering mode is selected at the renderer may comprise at least one modification parameter, the at least one modification parameter being configured to control a modification of at least one rendering process at the renderer. The at least one modification parameter may be further configured to control a modification of at least one default rendering process at the renderer, wherein the default rendering process may be applied by the renderer when rendering the six degrees of freedom audio scene when the at least one rendering mode is not selected. The at least one modification parameter may comprise at least one of: a reverberation modification configured to selectively enable reverberation for at least one audio source within the six degrees of freedom audio scene; a reflections modification configured to selectively enable reflections for at least one audio source within the six degrees of freedom audio scene; an occlusion modification configured to selectively enable occlusions for at least one audio source within the six degrees of freedom audio scene; a diffraction modification configured to selectively enable diffraction for at least one audio source within the six degrees of freedom audio scene; a heterogenous extent modification configured to selectively enable heterogenous propagation for at least one audio source within the six degrees for freedom audio scene; a homogenous extent modification configured to selectively enable homogenous propagation for at least one audio source within the six degrees for freedom audio scene; a portals modification configured to selectively enable portals for at least one audio source within the six degrees for freedom audio scene; a distance gain modification configured to selectively enable distance gains for at least one audio source within the six degrees for freedom audio scene; and a doppler modification configured to selectively enable doppler effects for at least one audio source within the six degrees for freedom audio scene. The at least one modification parameter may comprise at least one of: a disable effect modification configured to disable at least one rendering process; an attenuate effect modification configured to attenuate at least one rendering process; and an enhance effect modification configured to enhance at least one rendering process. Generating a bitstream configured to define a six-degrees of freedom rendering may comprise at least one of: receiving the information in an encoder input file format and generating an encoded MPEG-I format bitstream to be combined with an encoded six degrees of freedom audio scene bitstream; and obtaining the information in an MPEG-I format and combining the information with an encoded six degrees of freedom audio scene bitstream. According to a second aspect there is provided a method, comprising: obtaining a bitstream configured to define a six-degrees of freedom rendering, the