US-20230283790-A1 - PARALLELIZED VIDEO DECODING USING A NEURAL NETWORK

US20230283790A1US 20230283790 A1US20230283790 A1US 20230283790A1US-20230283790-A1

Abstract

In a method for decoding a data stream by way of an electronic device ( 10 ) including a processor ( 14 ), and a parallelized processing unit ( 16 ) designed to perform a plurality of operations of the same type in parallel at a given time, the data stream includes a first dataset (Fet) and a second dataset (Fnn) representative of audio or video content. The decoding method includes the processor ( 14 ) processing data from the first dataset (Fet), obtaining the audio or video content by processing (E 70 ) data from the second dataset (Fnn) using a process depending at least partially on the data from the first set (Fet) and using an artificial neural network ( 18 ) implemented by the parallelized processing unit ( 16 ).

Inventors

HENRY FéLIX
CLARE GORDON

Assignees

FOND B COM

Dates

Publication Date: 20230907
Application Date: 20210713
Priority Date: 20210713

Claims (20)

1 . A method for decoding a data stream by an electronic device including a first processor and a parallelized processor configured to perform in parallel, at a specific time, a plurality of a same type of operations, the data stream including a first set of data and a second set of data representative of an audio or video content, the method comprising: processing data from the first set of data by the first processor; and obtaining the audio or video content by processing data from the second set of data using a process at least partially depending on the data from the first set and using an artificial neural network implemented by the parallelized processor.
2 . The decoding method according to claim 1 , further comprising configuring the parallelized processor as a function of part-at least part of the data from the first set of data.
3 . The decoding method according to claim 2 , wherein the first set of data comprises data descriptive of the artificial neural network, and wherein, in the configuring, the first processor configures the parallelized processor based on said descriptive data.
4 . The decoding method according to claim 2 , wherein the electronic device comprises a storage configured to store a plurality of parameter sets respectively defining a plurality of artificial neural networks, wherein the first set of data comprises an identifier, and wherein, in the configuring, the first processor configures the parallelized processor based on a set of parameters associated with the identifier among the plurality of parameter sets.
5 . The decoding method according to claim 1 , wherein the data stream further comprises instructions executable within the electronic device, and wherein the processing of the data from the first data set is at least partially performed due toexecution of at least part of said instructions.
6 . The decoding method according to claim 2 , wherein the data stream further comprises instructions executable within the electronic device, and wherein the configuring the parallelized processor is performed due to execution of at least part of said instructions.
7 . The decoding method according to claim 1 , further comprising identifying the first set of data and the second set of data within the data stream.
8 . The decoding method according to claim 1 , wherein the first set of data comprises data representative of characteristics of a format of the audio or video content encoded by the data stream.
9 . The decoding method according to claim 1 , wherein the processing of the data from the second set produces at least one matrix representation of at least a part of an image.
10 . The decoding method according to claim 1 , wherein the artificial neural network receives, as an input, data from the second set of data.
11 . The decoding method according to claim 1 , wherein the artificial neural network receives, as an input, data previously produced as an output of the artificial neural network.
12 . An electronic device configured to decode a data stream including a first set of data and a second set of data representative of an audio or video content, the electronic device comprising: a first processor (configured to process data from the first set of data; and a parallelized processor configured to perform in parallel, at a specific time, a plurality of a same type of operations and configured to obtain the audio or video content by processing data from the second set of data using a process at least partially depending on the data from the first set and using an artificial neural network implemented by the parallelized processor.
13 . The electronic device according to claim 12 , wherein the first processor configures the parallelized processor as a function of at least part of the data from the first set of data.
14 . The electronic device according to claim 13 , further comprising a storage configured to store a plurality of parameter sets respectively defining a plurality of artificial neural networks, wherein the first set of data comprises an identifier, and wherein the first processor configures the parallelized processor based on a set of parameters associated with the identifier among the plurality of parameter sets.
15 . The electronic device according to claim 12 , wherein the parallelized processor is configured to produce at least one matrix representation of at least a part of an image.
16 . (canceled)
17 . The decoding method according to claim 2 , wherein the data stream further comprises instructions executable within the electronic device, and wherein the processing of the first data set is at least partially performed due to execution of at least part of said instructions.
18 . The decoding method according to claim 17 , wherein the configuring the parallelized processor is performed due to execution of at least part of said instructions.
19 . The decoding method according to claim 2 , further comprising identifying the first set of data and the second set of data within the data stream.
20 . The decoding method according to claim 2 , wherein the first set of data comprises data representative of characteristics of a format of the audio or video content encoded by the data stream.

Description

TECHNICAL FIELD OF THE INVENTION The present invention relates to the technical field of content decoding, in particular audio or video content. In particularly, it relates to a method and an electronic device for decoding a data stream, as well as an associated data stream. STATE OF THE ART It is known, in particular in the field of video decoding, to use an electronic device comprising both a processor (generally a central unit of the electronic device, or CPU for “Central Processing Unit”) and a parallelized processing unit designed to perform in parallel, at a given time, a plurality of operations of the same type. Such a parallelized processing unit is for example a Graphical Processing Unit or GPU, or a Tensor Processing Unit or TPU, as described for example in the article “Google’s Tensor Processing Unit explained: this is what the future of computing looks like”, by Joe Osborne, Techradar, Aug. 22, 2016. It has moreover been proposed to compress data representative of a video content by means of an artificial neural network. The decoding of the compressed data can then be performed by means of another artificial neural network, as described for example in the article “DVC: An End-to-end Deep Video Compression Framework”, by Guo Lu et al., 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 10998-11007. DISCLOSURE OF THE INVENTION In this context, the present invention proposes a method for decoding a data stream by means of an electronic device comprising a processor, and a parallelized processing unit designed to perform in parallel, at a given time, a plurality of operations of the same type, characterized in that the data stream comprises a first set of data and a second set of data representative of an audio or video content, and in that the method comprises the following steps: processing the data from the first set of data by the processor;obtaining the audio or video content by processing the data from the second set of data using a process depending on part at least of the data from the first set and using an artificial neural network implemented by the parallelized processing unit. The processing of the data from the second set by means of the artificial neural network implemented by the parallelized processing unit can thus be adapted as a function of the data from the first set contained in the data stream. This provides a flexible and efficient processing of the second data for their decoding by an artificial neural network. The method can hence comprise for example a step of configuring the parallelized processing unit as a function of part at least of the data from the first set of data. The configuration of the parallelized processing unit may comprise an allocation of memory of the parallelized processing unit and/or an instantiation of the memory of the parallelized processing memory and/or an assignment of values (as a function of said part at least of the data from the first set of data) to the processing operations implemented on the parallelized processing unit (here, in practice, assignment of weights and/or activation functions defining the artificial neural network, these weights and/or activation functions being determined as a function of said part at least of the data from the first set of data). According to a first possibility, the first set of data may comprise data descriptive of the artificial neural network (for example, data encoding the artificial neural network). In this case, at the configuration step, the processor may configure the parallelized processing unit on the basis of these descriptive data. According to a second possibility, the electronic device may comprise a storage unit for storing a plurality of parameter sets defining respectively a plurality of artificial neural networks. The first set of data may, in this case, comprise an identifier. At the configuration step, the processor can then configure the parallelized processing unit on the basis of a set of parameters associated with this identifier among the plurality of parameter sets. The processor is for example a microprocessor; the processor can thus execute successively a plurality of instructions from a computer program. Moreover, it may be provided that the data stream further comprises instructions executable (for example by the processor or, as an alternative, by a virtual machine) within the electronic device. The processing of the data from the first set of data may in this case be at least in part performed due to the execution of part at least of these instructions. In particular, the configuration step of the parallelized processing unit may then be performed due to the execution of part at least of these instructions. The decoding method may moreover include a step of identifying the first set of data and the second set of data within the data stream (for example thanks to the use of a predetermined binary length for the first set