KR-20260066035-A - Method and device for providing real-time audio mixing service and playlist

KR20260066035AKR 20260066035 AKR20260066035 AKR 20260066035AKR-20260066035-A

Abstract

A server according to one embodiment of the present invention includes a communication module and a processor configured to communicate with a user terminal, wherein the processor receives sound source information and mixing information from the user terminal, selects an audio block corresponding to at least one stem item included in the sound source information based on the mixing information, combines audio information included in the selected audio block to generate a first session audio, generates a second session audio belonging to a different section from the first session audio, and individually adjusts at least one of the volume, speed, tonality, effect, and playback start point of at least one of the first session audio and the second session audio to generate a mixed sound source, and the communication module may be configured to transmit the generated mixed sound source to the user terminal so that it is played in real time.

Inventors

김태형
김근형
이종필
금상은

Assignees

뉴튠(주)

Dates

Publication Date: 20260512
Application Date: 20260430

Claims (12)

A communication module configured to communicate with a user terminal; and processor; Includes, The above processor is, Receiving sound source information and mixing information from the above user terminal, and Based on the above mixing information, select an audio block corresponding to at least one stem item included in the above sound source information, and A first session audio is generated by combining audio information included in the selected audio block, and Generate a second session audio belonging to a different section from the first session audio, and It is configured to generate a mixed sound source by individually adjusting at least one of the volume, speed, tonality, effect, and playback start point of at least one of the first session audio and the second session audio, and The communication module is configured to transmit the generated mixing sound source to the user terminal so that it can be played in real time. Server.
In paragraph 1, The above processor is, Load at least one audio version that is pre-stored for the above sound source information, and It is configured to generate an audio block corresponding to at least one pre-set stem item for each of the above-mentioned imported audio versions, and The communication module is configured to transmit information regarding the retrieved audio version and the generated audio block to the user terminal, and The user terminal is configured to display an audio mixing interface generated based on the received information on a display, and The above audio mixing interface includes an audio block screen in which the generated audio blocks are visually arranged. Server.
In paragraph 2, The above audio mixing interface is configured to visually distinguish the selected block from other audio blocks when there is a selected block among the audio blocks displayed on the audio block screen, and The above-mentioned visual distinguishing indicator includes at least one of displaying the selected block in a different shade from the other audio blocks or displaying a check mark on the selected block. Server.
In paragraph 1, The above stem item includes at least one of a Rhythm stem, a Bass stem, a Mid stem, a High stem, an FX stem, and a Melody stem, Server.
In paragraph 1, The communication module is configured to receive at least one of the user's voice information and conversation information input through the user terminal, and The above processor is, Analyze and process at least one of the received voice information and conversation information to map to music tag information, and Configured to generate the mixing information based on the mapped music tag information, Server.
In paragraph 1, The above processor is, Determining the similarity between the data stored in the memory module and the mixed sound source by considering the musical characteristics of the mixed sound source, and Configured to generate the next playlist to be played by selecting an audio block or session audio similar to the mixing source based on the similarity determined above. Server.
In a user terminal communicating with a server, A communication module configured to communicate with the above-mentioned server; Display; and processor; Includes, The communication module is configured to transmit sound source information and mixing information input through the display to the server, and to receive a mixed sound source from the server. The received mixing audio source is provided to the user terminal by the server selecting an audio block corresponding to at least one stem item included in the audio source information based on the mixing information, combining audio information included in the selected audio block to generate a first session audio, generating a second session audio belonging to a different section from the first session audio, and individually adjusting at least one of the volume, speed, tonality, effect, and playback start point of at least one of the first session audio and the second session audio. The above processor is configured to play the received mixing audio source in real time, User terminal.
In Paragraph 7, The communication module is configured to further receive information regarding the first session audio and the second session audio from the server, and The above processor further includes an audio mixing screen generation module, and The above audio mixing screen generation module is configured to generate an audio mixing interface based on the received information and display it on the display, and The above audio mixing interface includes a session audio screen that displays a block corresponding to the first session audio and a block corresponding to the second session audio at a position corresponding to the playback order. User terminal.
In paragraph 8, The communication module is configured to further receive information regarding audio versions and audio blocks from the server, and The information regarding the received audio version and audio block is provided to the user terminal by the server retrieving at least one audio version that is pre-stored for the sound source information, and generating an audio block corresponding to each of the at least one stem item that is pre-set for the retrieving audio version. The above audio mixing interface further includes an audio block screen in which the generated audio blocks are visually arranged according to the stem item and the audio version based on the received information. User terminal.
In claim 7, the stem item comprises at least one of a rhythm stem, a bass stem, a mid stem, a high stem, an FX stem, and a melody stem. User terminal.
In Paragraph 7, Including an input section; further The above processor is configured to transmit at least one of the user's voice information and conversation information input through the input unit to the server through the communication module, and The above mixing information is generated based on the mapped music tag information, wherein the server analyzes and processes at least one of the transmitted user's voice information and the conversation information to map it to music tag information. User terminal.
In Paragraph 7, The communication module is configured to receive information on the next playlist to be played from the server and display it on the display. The above playlist information is generated by the server determining the similarity between the data stored in the memory module and the mixing sound source by considering the musical characteristics of the mixing sound source, and selecting an audio block or session audio similar to the mixing sound source based on the determined similarity. User terminal.

Description

Method and device for providing real-time audio mixing service and playlist Method and device for providing real-time audio mixing service and playlist The present invention relates to a real-time audio mixing service and a method and apparatus for providing a playlist, and more specifically, to a technology that utilizes user information, including information input by a user or information about the user's surrounding environment, as mixing information, and accordingly selects and mixes stem data to provide real-time mixed audio. With the advancement of technology, it has become possible to digitize existing large-capacity media and convert them into low-capacity media. As a result, today, users can store various types of media on portable devices and conveniently select and enjoy the media they want while on the move. Furthermore, media digitized through digital compression technology enables media sharing among users over networks, which has explosively boosted online media services, and many applications and programs related to this are being developed. Music constitutes a significant portion of the vast amount of media available; compared to other media types, it requires less data and places a lower communication load, making it suitable for supporting real-time streaming services and resulting in high satisfaction for both service providers and users. Consequently, services providing online music to users through various methods are currently emerging. While existing online music services were limited to simply providing music to online-connected users in real time through methods such as delivering audio sources to user terminal devices or offering streaming services, recently, services that recommend highly preferred media to users are being provided by utilizing big data or artificial intelligence technology. However, current recommendation methods in online music services generate music charts by simply aggregating the number of tracks purchased, listened to, or searched by users, and then make recommendations based on these charts. This approach recommends music based on statistical criteria derived from mere access counts, thereby ignoring the diversity and variability of user preferences. Furthermore, because this method recommends music based on accumulated access counts, the volatility of the music charts is low; consequently, most of the music currently recommended overlaps with previously recommended tracks, significantly reducing its effectiveness. In addition, there are cases where users want to listen to different versions of the same music according to their preferences, but currently, unless the music distribution company distributes the music in different versions, users are unable to listen to versions of the music that have a different feel. FIG. 1 is a diagram illustrating a part of the configuration of a real-time audio mixing service providing system based on user information according to one embodiment of the disclosed invention. FIG. 2 is a diagram illustrating a part of the configuration of a real-time audio mixing service providing device based on user information according to one embodiment of the disclosed invention. FIG. 3 is a diagram illustrating a real-time audio mixing service and a playlist provision method based on user information according to one embodiment of the disclosed invention. FIG. 4 is a diagram illustrating a method for providing a real-time audio mixing service and a playlist according to an embodiment of the disclosed invention, wherein music characteristic information input by a user is generated as mixing information, and a mixing sound source is generated based on the generated mixing information. FIG. 5 is a diagram illustrating a method for providing a real-time audio mixing service and a playlist according to an embodiment of the disclosed invention, wherein music tag information entered by a user is generated as mixing information, and a mixing sound source is generated based on the generated mixing information. FIG. 6 is a diagram illustrating a method for providing a real-time audio mixing service and a playlist according to an embodiment of the disclosed invention, wherein user's surrounding environment information is generated as mixing information, and a mixing sound source is generated based on the generated mixing information. FIG. 7 is a diagram illustrating a method for generating a real-time mixed sound source by individually adjusting session audio in a real-time audio mixing service and playlist provision method according to one embodiment of the disclosed invention. FIG. 8 is a diagram illustrating a multi-client management method of a real-time audio mixing service providing device according to one embodiment of the disclosed invention. FIG. 9 is a drawing illustrating an interface screen displayed on a user terminal when a mixing audio service is provided in practice by applying the disclosed invention. Hereinafter, embodiments