CN-122003712-A - Information processing method, information processing system, and program

CN122003712ACN 122003712 ACN122003712 ACN 122003712ACN-122003712-A

Abstract

The information processing system includes an instruction data generation unit and a music data processing unit. The instruction data generation unit generates first instruction data including an instruction from a user concerning processing of music data, and attribute data indicating an instruction for one or more of a plurality of processing models that perform different processing, based on the first input data indicating an instruction for one or more of the plurality of processing models that perform the different processing, and the attribute data using the machine-learned generation model, and the music data processing unit performs the processing indicated by the first instruction data on the music data using one or more of the plurality of processing models that are indicated by the first instruction data. The instruction data generating unit generates second instruction data including an instruction from the user regarding the process of the music data on which the process indicated by the first instruction data is performed, using the generation model based on the second input data indicating an instruction to one or more of the plurality of process models and the attribute data, and the music data processing unit performs the process indicated by the second instruction data on the music data using one or more of the plurality of process models indicated by the second instruction data.

Inventors

Mae teruhisa
YAMAMOTO KAZUHIKO
ZHANG YIXIAO

Assignees

雅马哈株式会社

Dates

Publication Date: 20260508
Application Date: 20241009
Priority Date: 20231013

Claims (9)

1. An information processing method implemented by a computer system, the information processing method comprising: Generating first instruction data by using a machine-learned generation model based on first input data including an instruction from a user concerning processing of music data and attribute data indicating an attribute of music represented by the music data, the first instruction data indicating an instruction for one or more of a plurality of processing models that execute mutually different processing; Executing processing of the first instruction data representation on the music data using the one or more processing models of the plurality of processing models represented by the first instruction data; Generating second instruction data using the generation model based on second input data including an instruction from the user regarding the processing of the music data on which the processing represented by the first instruction data is performed and the attribute data, the second instruction data representing an instruction to one or more of the plurality of processing models, and And executing processing of the second instruction data representation on the music data using the one or more processing models of the plurality of processing models represented by the second instruction data.
2. The information processing method according to claim 1, further comprising: the attribute data is partially updated per processing of the music data.
3. The information processing method according to claim 2, wherein, In the updating of the attribute data, the attribute data is updated based on an instruction from the user and the music data after the processing of the first instruction data or the second instruction data representation is performed.
4. The information processing method according to claim 1, wherein, The attribute data includes at least one of genre, performance speed, tonality, impression, and performance instrument of the music.
5. The information processing method according to claim 1, wherein, Each of the first instruction data or the second instruction data instructs addition of a performance sound part in the music, deletion of a performance sound part, regeneration of a specified section in the music, addition of an effect sound, change of a pitch, or change of a performance speed as processing of the music data.
6. The information processing method according to claim 1, wherein, Each of the first input data and the second input data includes historical data representing a history of conversations including an indication from the user and an answer to the indication by the generative model.
7. The information processing method according to claim 1, wherein, Each of the first input data and the second input data further includes condition data, wherein the condition data specifies a condition related to the action of the generative model.
8. An information processing system comprising an instruction data generation unit and a music data processing unit, The instruction data generating unit generates first instruction data including an instruction from a user concerning processing of music data based on first input data representing an attribute of music represented by the music data and attribute data representing an instruction for one or more of a plurality of processing models that execute different processing, The music data processing unit executes processing indicated by the first instruction data on the music data using one or more of the plurality of processing models indicated by the first instruction data, The instruction data generating unit generates second instruction data including an instruction from the user regarding the processing of the music data on which the processing indicated by the first instruction data is performed, using the generation model, based on second input data indicating an instruction to one or more of the plurality of processing models and the attribute data, The music data processing unit executes processing indicated by the second instruction data on the music data using one or more of the plurality of processing models indicated by the second instruction data.
9. A program for causing a computer system to function as an instruction data generation unit and a music data processing unit, The instruction data generating unit generates first instruction data including an instruction from a user concerning processing of music data based on first input data representing an attribute of music represented by the music data and attribute data representing an instruction for one or more of a plurality of processing models that execute different processing, The music data processing unit executes processing indicated by the first instruction data on the music data using one or more of the plurality of processing models indicated by the first instruction data, The instruction data generating unit generates second instruction data including an instruction from the user regarding the processing of the music data on which the processing indicated by the first instruction data is performed, using the generation model, based on second input data indicating an instruction to one or more of the plurality of processing models and the attribute data, The music data processing unit executes processing indicated by the second instruction data on the music data using one or more of the plurality of processing models indicated by the second instruction data.

Description

Information processing method, information processing system, and program Technical Field The present disclosure relates to a technique of processing music data. Background Various techniques for generating music according to an instruction from a user have been proposed. For example, non-patent documents 1 to 3 disclose techniques for generating music using a deep neural network. Prior art literature Non-patent literature Non-patent literature 1：Ryan Louie, Andy Coenen, Cheng Zhi Huang, Michael Terry, and Carrie J Cai, "Novice-AI music co-creation via AI-steering tools for deep generative models" In Proceedings of the 2020 CHI conference on human factors in computing systems. 1-13. Non-patent literature 2：Simeon Rau, Frank Heyen, Stefan Wagner, and Michael Sedlmair, "Visualization for AI-Assisted Composing" In Proceedings of the 23th International Society for Music Information Retrieval Conference. Non-patent literature 3：Adam Roberts, Jesse Engel, Yotam Mann, Jon Gillick, Claire Kayacik, Signe Norly, Monica Dinculescu, Carey Radebaugh, Curtis Hawthorne, and Douglas Eck, "Magenta studio: Augmenting creativity with deep learning in ableton live" 2019. Disclosure of Invention Problems to be solved by the invention In a general music production scene, a plurality of processes such as adding a music piece (part) and mixing a plurality of audio tracks are repeated a plurality of times, thereby generating final music. However, in conventional music production using a model trained by a deep neural network or the like, music is produced by a single process according to an instruction from a user. Therefore, it is difficult to maintain musical consistency (i.e., uniformity) and repeatedly perform various editing. In view of the above, an object of one embodiment of the present disclosure is to repeatedly perform processing of music data while maintaining musical consistency. Means for solving the problems In order to solve the above problems, an information processing method according to one aspect of the present disclosure includes generating first instruction data based on first input data and attribute data, using a machine-learned generation model, wherein the first input data includes an instruction from a user regarding a process of music data, the attribute data indicates an attribute of music represented by the music data, the first instruction data indicates an instruction from the user regarding a process of the music data, the first instruction data indicates one or more of a plurality of process models that perform processes different from each other, the first instruction data indicates the one or more of the plurality of process models, the first instruction data indicates a process of the music data, the second instruction data indicates the one or more of the plurality of process models, and the second instruction data indicates the one or more of the plurality of process models, using the generation model, and generating second instruction data based on the second input data and the attribute data, wherein the second input data includes an instruction from the user regarding a process of the music data for which the process indicated by the first instruction data indicates the process, and the second instruction data indicates the one or more of the plurality of process models. An information processing system according to one aspect of the present disclosure is an information processing system including an instruction data generating section that generates first instruction data based on first input data including an instruction from a user concerning a process of music data and attribute data representing an instruction of a music attribute of the music data expressed by the instruction data, and a music data processing section that uses one or more of a plurality of process models in which mutually different processes are performed, the music data processing section executing a process of the first instruction data expressed by the one or more of the process models, based on second input data and the attribute data, using the generation model, the instruction data generating section generating second instruction data including an instruction of the music data expressed by the instruction data, the second instruction data representing a process of the music data expressed by the instruction data, using the one or more of the process models in which the one or more of the process models expressed by the instruction data are performed, using the one or more of the process models expressed by the instruction data, and the instruction data expressed by the one of the instruction data expressed by the instruction data. A program according to one aspect of the present disclosure is a program for causing a computer system to function as an instruction data generating unit that generates first instruction data including an instruction from a user concerning a process of music data based on first input d