US-20260127835-A1 - DYNAMIC CLOTHING ADJUSTMENT SYSTEM BASED ON MUSICAL ELEMENT AND METHOD THEREOF

US20260127835A1US 20260127835 A1US20260127835 A1US 20260127835A1US-20260127835-A1

Abstract

A dynamic clothing adjustment system based on musical element and a method thereof are disclosed. In the system, clothing items and first musical element messages, and a pre-trained musical element recognition model are stored; a computer host generates a virtual character that is allowed to wear the clothing item for display, and when a musical performance is detected, a music signal is continuously received, the music signal is input into a musical element recognition model to obtain a second musical element message corresponding to the music signal; the hardware processor loads the clothing item to be worn on the virtual character based on a comparison result of the first musical element message and the second musical element message, and continuously compares a difference between the second musical element messages at different times; when the difference exceeds a preset value, the clothing item worn on the virtual character is adjusted.

Inventors

Chuan-Cheng Chiu
Miao-Miao Liu

Assignees

SQ Technology (Shanghai) Corporation
INVENTEC CORPORATION

Dates

Publication Date: 20260507
Application Date: 20250115
Priority Date: 20241107

Claims (10)

1 . A dynamic clothing adjustment system based on musical element, comprising: a clothing database, configured to store clothing items and first musical element messages, wherein each of the first musical element messages corresponds to one of the clothing items; and a computer host, configured to connect the clothing database, and comprising: a non-transitory computer-readable storage medium, configured to store computer readable instructions, a pre-trained musical element recognition model, wherein the musical element recognition model is configured to receive a music signal, and output at least one second musical element message corresponding to the music signal; and a hardware processor, electrically connected to the non-transitory computer-readable storage medium, and configured to execute the computer readable instructions to operate: generating a virtual character through 3D modeling technology and displaying the virtual character, wherein the virtual character is allowed to wear one of the clothing items; when detecting that the virtual character performs a musical performance, continuously receiving a music signal and performing feature extraction, standardization and dimensionality reduction processing on the music signal, and inputting the processed music signal into the musical element recognition model to obtain the second musical element message corresponding to the music signal; comparing the obtained second musical element message with the first musical element message in the clothing database, and loading one of the clothing items corresponding to the matching first musical element message and wearing the loaded clothing item on the virtual character; and continuously comparing a difference between the second musical element messages obtained at different times to obtain an difference element amount, when the difference element amount exceeds an preset amount, repeating the previous steps to replace the clothing item currently worn on the virtual character with the loaded clothing item, and during the process of wearing the loaded clothing item, generating an dress-change special effects to obscure the virtual character until the clothing item is worn is completed.
2 . The dynamic clothing adjustment system based on musical element according to claim 1 , wherein each of the first musical element message and the second musical element message comprises at least one of genre, rhythm, melody, harmony, and timbre, and selectively comprises an ethnic religious message as a condition for selecting the clothing item to be loaded.
3 . The dynamic clothing adjustment system based on musical element according to claim 1 , wherein the hardware processor loads pieces of music data and label messages as training data, wherein each of the label messages corresponding to one of the pieces of music data from a music database, and the hardware processor inputs the training data into an artificial intelligence model for training, to generate the musical element recognition model, wherein each of the label messages comprises at least one of genre, rhythm, melody, harmony, and timbre.
4 . The dynamic clothing adjustment system based on musical element according to claim 1 , wherein the hardware processor is configured to detect an IP address, and when the IP address matches a specific country, the hardware processor loads at least one accessory item from the clothing database, and wears the loaded accessory item on the virtual character, the accessory item comprises at least one of a turban, a veil, a scarf, a mask, and a hat.
5 . The dynamic clothing adjustment system based on musical element according to claim 1 , wherein the hardware processor is allowed to continuously capture images of the virtual character and share the captured images to a social media, when the difference element amount does not exceed the preset amount, the hardware processor is allowed to select any of the loaded clothing items and wear the selected clothing items on the virtual character.
6 . A dynamic clothing adjustment method based on musical element, comprising: connecting a clothing database to a computer host, wherein the clothing database stores clothing items and first musical element messages, each of the clothing items corresponds to one of the first musical element messages, and the computer host stores a pre-trained musical element recognition model, the musical element recognition model receives a music signal and outputs at least one second musical element message corresponding to the music signal; generating a virtual character through a 3D modeling technology and displaying the virtual character, by the computer host, wherein the virtual character is allowed to wear at least one of the clothing items; when it is detected that the virtual character performs a musical performance, continuously receiving a music signal and performing feature extraction, standardization and dimensionality reduction processing on the music signal, inputting the processed music signal into the musical element recognition model to obtain the second musical element message corresponding to the music signal, by the computer host; comparing the obtained second musical element message with the first musical element message in the clothing database, loading one of the clothing items corresponding to the matching first musical element message, and wearing the loaded clothing item on the virtual character, by the computer host; and continuously comparing a difference between the second musical element messages obtained at different times to obtain an difference element amount, and when the difference element amount exceeds an preset amount, repeating the previous steps to replace the clothing item currently worn on the virtual character with the loaded clothing item, by the computer host, wherein during the process of wearing the clothing item, a dress-change special effects is generated to obscure the virtual character until the clothing item is worn completely.
7 . The dynamic clothing adjustment method based on musical element according to claim 6 , wherein each of the first musical element message and the second musical element message comprises at least one of genre, rhythm, melody, harmony, and timbre, and selectively comprises an ethnic religious message as a condition for selecting the clothing item to be loaded.
8 . The dynamic clothing adjustment method based on musical element according to claim 6 , wherein the hardware processor loads pieces of music data and label messages as training data, wherein each of the label messages corresponding to one of the pieces of music data from a music database, and the hardware processor inputs the training data into an artificial intelligence model for training, to generate the musical element recognition model, wherein each of the label messages comprises at least one of genre, rhythm, melody, harmony, and timbre.
9 . The dynamic clothing adjustment method based on musical element according to claim 6 , wherein the hardware processor is configured to detect an IP address, and when the IP address matches a specific country, the hardware processor loads at least one accessory item from the clothing database, and wears the loaded accessory item on the virtual character, the accessory item comprises at least one of a turban, a veil, a scarf, a mask, and a hat.
10 . The dynamic clothing adjustment method based on musical element according to claim 6 , wherein the hardware processor is allowed to continuously capture images of the virtual character and share the captured images to a social media, when the difference element amount does not exceed the preset amount, the hardware processor is allowed to select any of the loaded clothing items and wear the selected clothing items on the virtual character.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a dynamic clothing adjustment system and a method thereof, and more particularly to a dynamic clothing adjustment system based on musical element and a method thereof. 2. Description of the Related Art In recent years, with the popularization and rapid development of virtual technologies, various applications of virtual technologies have emerged, such as virtual singers and virtual streamers (such as VTubers). Conventionally, clothing on virtual singers or virtual streamers are usually fixed and unchanging. Even if clothing on virtual singers or virtual streamers can be changed, different types of clothing are switched through pre-configured way and cannot adapt to the music during a performance. In other words, while conventional methods maintain consistent images of virtual singers or virtual streamers, they often lack interaction with the music, and it may cause the problem that the clothing on virtual singers or virtual streamers cannot adapt to musical changes. In view of this, some companies have proposed technical solutions of changing the colors and patterns of the virtual character's clothing, LED light clothing, so that the displayed colors and patterns of the clothing can be dynamic adjusted according to the music, thereby enhancing the visual appeal of the virtual character. However, this solution cannot instantly achieve sufficient coordination between the clothing and the music, that is, simply changing colors and patterns is still insufficient for the audience to perceive coordination between the music and clothing. The patterns and colors, if disorganized, may even amplify the sense of discord. Thus, although the clothing on a virtual character can be changed with the music, the problem of insufficient coordination remains. According to above-mentioned contents, what is needed is to develop an improved solution to solve the problem of insufficient real-time coordination between music and clothing on a virtual character. SUMMARY OF THE INVENTION An objective of the present invention is to disclose a dynamic clothing adjustment system based on musical element and a method thereto, to solve the conventional problem of insufficient real-time coordination between music and clothing on a virtual character. To achieve the objective, the present invention discloses a dynamic clothing adjustment system based on musical element, the system includes a clothing database and a computer host. The clothing database is configured to store clothing items and first musical element messages, wherein each of the first musical element messages corresponds to one of the clothing items. The computer host is configured to connect the clothing database and comprises a non-transitory computer-readable storage medium and a hardware processor. The non-transitory computer-readable storage medium is configured to store computer readable instructions, a pre-trained musical element recognition model, wherein the musical element recognition model is configured to receive a music signal, and output at least one second musical element message corresponding to the music signal. The hardware processor is electrically connected to the non-transitory computer-readable storage medium, and configured to execute the computer readable instructions to operate: generating a virtual character through 3D modeling technology and displaying the virtual character, wherein the virtual character is allowed to wear one of the clothing items; when detecting that the virtual character performs a musical performance, continuously receiving a music signal and performing feature extraction, standardization and dimensionality reduction processing on the music signal, and inputting the processed music signal into the musical element recognition model to obtain the second musical element message corresponding to the music signal; comparing the obtained second musical element message with the first musical element message in the clothing database, and loading one of the clothing items corresponding to the matching first musical element message and wearing the loaded clothing item on the virtual character; continuously comparing a difference between the second musical element messages obtained at different times to obtain an difference element amount, when the difference element amount exceeds an preset amount, repeating the previous steps to replace the clothing item currently worn on the virtual character with the loaded clothing item, and during the process of wearing the loaded clothing item, generating an dress-change special effects to obscure the virtual character until the clothing item is worn is completed. The present invention discloses a dynamic clothing adjustment method based on musical element, the dynamic clothing adjustment method includes steps of connecting a clothing database to a computer host, wherein the clothing database stores clothing items a