Search

US-12626567-B2 - Method for carriage of binary wavelet streams in haptics interchange format

US12626567B2US 12626567 B2US12626567 B2US 12626567B2US-12626567-B2

Abstract

Methods, apparatuses, and systems for haptic signal processing are provided and include receiving a media stream comprising data in a haptics interchange format, obtaining, from the data in the haptics interchange format of the media stream, a wavelet effect value, and decoding the media stream based on the wavelet effect value.

Inventors

  • Iraj Sodagar

Assignees

  • Tencent America LLC

Dates

Publication Date
20260512
Application Date
20240416

Claims (14)

  1. 1 . A method for decoding video data, the method being performed by at least one processor, the method comprising: receiving a media stream comprising data in a haptics interchange format, the haptics interchange format is an .hjif format, and the data is in the .hjif format; obtaining, from the data in the haptics interchange format of the media stream, a wavelet effect value, the wavelet effect value is obtained from a “band type” property of the data in the .hjif format, and the wavelet effect value is indicated as “BinaryWavelet” in the “band type” property of the data in the .hjif format; and decoding the media stream based on the wavelet effect value.
  2. 2 . The method according to claim 1 , wherein the wavelet effect value indicates: that bands of the media stream are in binary encoded wavelet streams, and that entropy decoding is needed along with inverse wavelet transformation to decode a wave of the bands of the media stream.
  3. 3 . The method according to claim 1 , wherein the wavelet effect value indicates that bands of the media stream are in binary encoded wavelet streams.
  4. 4 . The method according to claim 3 , wherein the wavelet effect value indicates that entropy decoding is needed along with inverse wavelet transformation to decode a wave of bands of the media stream.
  5. 5 . The method according to claim 1 , wherein decoding the media stream comprises decoding a binary wavelet keyframe in the .hjif format by running a base64 decoding of the media stream.
  6. 6 . The method according to claim 5 , wherein decoding the media stream further comprises running “readWaveletEffect( )” of haptic data of the media stream.
  7. 7 . The method according to claim 6 , wherein the haptic data of the media stream is in an MPEG-I Haptic Stream (MIHS) format.
  8. 8 . An apparatus for decoding video data, the apparatus comprising: at least one memory configured to store program code; and at least one processor configured to read the program code and operate as instructed by the program code, the program code including: receiving code configured to cause the at least one processor to receive a media stream comprising data in a haptics interchange format; obtaining code configured to cause the at least one processor to obtain, from the data in the haptics interchange format of the media stream, a wavelet effect value, the wavelet effect value is obtained from a “band type” property of the data in the .hjif format, and the wavelet effect value is indicated as “BinaryWavelet” in the “band type” property of the data in the .hjif format; and decoding code configured to cause the at least one processor to decode the media stream based on the wavelet effect value.
  9. 9 . The apparatus according to claim 8 , wherein the wavelet effect value indicates: that bands of the media stream are in binary encoded wavelet streams, and that entropy decoding is needed along with inverse wavelet transformation to decode a wave of the bands of the media stream.
  10. 10 . The apparatus according to claim 8 , wherein the wavelet effect value indicates that bands of the media stream are in binary encoded wavelet streams.
  11. 11 . The apparatus according to claim 10 , wherein the wavelet effect value indicates that entropy decoding is needed along with inverse wavelet transformation to decode a wave of bands of the media stream.
  12. 12 . The apparatus according to claim 8 , wherein decoding the media stream comprises decoding a binary wavelet keyframe in the .hjif format by running a base64 decoding of the media stream.
  13. 13 . The apparatus according to claim 12 , wherein decoding the media stream further comprises running “readWaveletEffect( )” of haptic data of the media stream.
  14. 14 . A non-transitory computer-readable medium storing instructions, the instructions comprising: one or more instructions that, when executed by one or more processors of a device for decoding haptic data, cause the one or more processors to: receive a media stream comprising data in a haptics interchange format; obtain, from the data in the haptics interchange format of the media stream, a wavelet effect value, the wavelet effect value is obtained from a “band type” property of the data in the .hjif format, and the wavelet effect value is indicated as “BinaryWavelet” in the “band type” property of the data in the .hjif format; and decode the media stream based on the wavelet effect value.

Description

CROSS-REFERENCE TO RELATED APPLICATION This application claims priority from U.S. Provisional Application No. 63/459,929, filed on Apr. 17, 2023, the disclosure of which is incorporated herein by reference in its entirety. FIELD This disclosure is directed to a set of advanced video coding technologies. More specifically, the present disclosure is directed to encoding and decoding haptic experience for multimedia presentation and a method for the carriage of binary wavelet streams in haptics interchange format. BACKGROUND Haptics experience has become a part of multimedia presentation. In applications where multimedia presentation includes an aspect of haptic experience, haptic signals may be delivered to the device or wearable and the user may feel the haptic sensations during the use of the application in coordination with the visual and/or audio media experience. Recognizing the growing popularity of haptic experience in multimedia presentations, motion picture experts group (MPEG) has started working on a compression standard (both for MPEG-DASH and MPEG-I) for haptics as well as carriage of the compressed haptics signaling in ISO based media file format (ISOBMFF). One of the problems to be solved in involving an aspect of haptic experience within a multimedia presentation is that the timing model of the carriage of haptics tracks is not clear, i.e. it is not clear how the timing of the ISOBMFF tracks relates to the timing of the haptic elementary signal. Solutions addressing this problem are required. The haptic Committee Draft includes one JSON and one binary format. The current JSON format, as known as the haptics interchange format, carries quantized wavelet coefficients and not the binary wavelet-coded streams. Solutions addressing this problem are required. SUMMARY According to an aspect of the disclosure, there is an apparatus, and similarly a method and computer readable medium, including at least one memory configured to store computer program code; and at least one processor configured to access the computer program code and operate as instructed by the computer program code, the computer program code including: receiving code configured to cause the at least one processor to receive a media stream comprising data in a haptics interchange format; obtaining code configured to cause the at least one processor to obtain, from the data in the haptics interchange format of the media stream, a wavelet effect value; and decoding code configured to cause the at least one processor to decode the media stream based on the wavelet effect value. The haptics interchange format may be an .hjif format, and the data may be in the .hjif format. The wavelet effect value may be obtained from a “band_type” property of the data in the .hjif format. The wavelet effect value may be indicated as “BinaryWavelet” in the “band_type” property of the data in the .hjif format. The wavelet effect value may indicate: that bands of the media stream are in binary encoded wavelet streams, and that entropy decoding is needed along with inverse wavelet transformation to decode a wave of the bands of the media stream. The wavelet effect value may indicate that bands of the media stream are in binary encoded wavelet streams. The wavelet effect value may indicate that entropy decoding is needed along with inverse wavelet transformation to decode a wave of bands of the media stream. The decoding the media stream may include decoding a binary wavelet keyframe in an .hjif format by running a base64 decoding of the media stream. The decoding the media stream may include running “readWaveletEffect( )” of haptic data of the media stream. Additional embodiments will be set forth in the description that follows and, in part, will be apparent from the description, and/or may be learned by practice of the presented embodiments of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS Further features, the nature, and various advantages of the disclosed subject matter will be more apparent from the following detailed description and the accompanying drawings in which: FIG. 1 is a schematic illustration of a simplified block diagram of a communication system, in accordance with embodiments of the present disclosure; FIG. 2 is a schematic illustration of a simplified block diagram of a streaming system, in accordance with embodiments of the present disclosure; FIG. 3 is an example illustration in accordance with embodiments of the present disclosure; FIG. 4 is an example illustration in accordance with embodiments of the present disclosure; FIG. 5A is an example illustration in accordance with embodiments of the present disclosure; FIG. 5B is an example illustration in accordance with embodiments of the present disclosure; FIG. 6 is an exemplary flow diagram illustrating a process for processing haptic media in accordance with embodiments of the present disclosure; FIG. 7 is an exemplary diagram illustrating aspects in accordance with embodiments of