Search

KR-102960282-B1 - method of providing recommendation of fashion goods by use of virtual clothing in smart home environment

KR102960282B1KR 102960282 B1KR102960282 B1KR 102960282B1KR-102960282-B1

Abstract

The present invention relates to a fashion product recommendation technology utilizing virtual clothing in a smart home environment, which generates virtual clothing based on characteristic information such as style and color inherent in clothing-wearing images collected through a camera installed on a smart home device, and recommends actual brand products similar to images in a database of in-store products to the user based on the generated virtual clothing. Since the present invention does not utilize the user's personal information, it is free from the disadvantages of conventional methods. In the present invention, when only an image of clothing being worn is provided through a smart home device in a situation where there is an intention to shop, various characteristics such as style, color, age, and seasonality are automatically detected to perform fashion product recommendations. Furthermore, the present invention has the advantage of guaranteeing the accuracy of product recommendations even if historical information such as purchase history or reviews is not sufficiently accumulated, as the method allows for the derivation of recommendation results that better satisfy the user's needs as the user provides additional information (e.g., speech, text). According to the present invention, a convenient and innovative online (mobile) shopping service can be provided.

Inventors

  • 박현용
  • 고영준
  • 김용진

Assignees

  • 주식회사 큐버

Dates

Publication Date
20260507
Application Date
20230829

Claims (6)

  1. A method for recommending fashion products using virtual clothing in a smart home environment, performed by a product recommendation server (100) to recommend fashion products to a user in a smart home environment, A first step in which a wear image receiving unit (110) receives a user’s clothing wear image (W) from a smart home device (10); A step in which a fashion text receiving unit (120) receives a fashion request text (T), which is a descriptive description of the clothing desired by the user, from a smart home device (10); A second step in which the image feature classification neural network (130N) of the image feature extraction unit (130) performs multi-label classification on the clothing wearing image (W) to generate multiple class vectors corresponding to multiple fashion attributes as image features; The semantic segmentation neural network (140N) of the image vector generation unit (140) performs clothing category classification on a pixel-by-pixel basis for the clothing wearing image (W) to form a segment feature map ( Step 3 for generating ); The feature vector generation unit (150) encodes the label of the main class information among the various labels of image features extracted by the image feature extraction unit (130) into a text vector to create a feature vector ( Set to ), and encode the label of the subclass information into a text vector to create a text embedding vector( Generated as ), and the above fashion requirement text (T) is encoded into a text vector which is a semantic embedding vector, and the above text embedding vector Adding to, the image feature extraction unit (130) concatenates a plurality of class vectors generated by the image feature extraction unit (130) to form an image feature vector ( Constructing ) and the above image feature vector ( ) and the above text embedding vector( Connecting the above feature vector ( By additionally generating ), a feature vector (for the above clothing wearing image (W) Step 4, which generates ); The virtual vector generation unit (160) is the segment feature map ( ) and the above feature vector( ) and Gaussian noise vector( Connect ) and perform convolution operations to create a virtual vector ( Step 5, which generates ); The virtual image generation unit (170) is the virtual vector ( ) and the feature vector of the above clothing wearing image (W) ) and Gaussian noise vector( Connect ) and perform deconvolution to obtain a virtual image vector ( Step 6, which generates ); The virtual image generation neural network (170N) of the virtual image generation unit (170) above generates a virtual image vector ( Step 7, generating multiple virtual clothing images (WV), which are images of virtual clothing, from ); Step 8, in which the user selection acquisition unit (180) provides the virtual clothing image (WV) to the user terminal (20) to identify the selected clothing image (S) according to the user selection; A ninth step in which a similar product extraction unit (190) selects a recommended product (R) from the store product DB (330) based on the similarity between the image of a plurality of store brand products stored in the store product DB (330) and the image of the selected outfit (S); and Step 10, in which the product recommendation providing unit (200) provides the recommended target product (R) to the user terminal (20) and recommends it to the user; A method for recommending fashion products using virtual clothing in a smart home environment configured including
  2. delete
  3. In claim 1, The virtual image generation unit (170) has an objective function for the generator G and discriminator D of the virtual image generation neural network (170N). The original above feature vector by ( ) and feature vector after update( A fashion product recommendation method using virtual clothing in a smart home environment, characterized by performing machine learning training to minimize the difference between ). (wherein λ is in the objective function Weight hyperparameters for adjusting the importance of
  4. In claim 1, The above 9th step is, A similar product extraction unit (190) inputs a store product image (P), which is an image of a plurality of store brand products stored in the store product DB (330), into an embedding model (190A) to convert it into a store product image vector (PV) and stores it in the store product DB (330); The above-mentioned similar product extraction unit (190) inputs the selected clothing image (S) into an embedding model (190B) and converts it into a selected clothing image vector (SV); The above similar product extraction unit (190) calculates the similarity between a plurality of selected clothing image vectors (SV) stored in the above store product DB (330) and the above store product image vector (PV) based on distance; The above-mentioned similar product extraction unit (190) selects a recommended product (R) from among the products of the brands based on similarity; A method for recommending fashion products using virtual clothing in a smart home environment, characterized by being configured to include
  5. In claim 1, The above 9th step is, A step in which a similar product extraction unit (190) inputs the selected clothing image (S) into an embedding model (190B) and converts it into a selected clothing image vector (SV); The above-mentioned similar product extraction unit (190) inputs the store product image (P), which is an image of a plurality of store brand products stored in the store product DB (330), into the embedding model (190A) to convert it into a store product image vector (PV); The above similar product extraction unit (190) calculates the similarity between the above multiple selected clothing image vectors (SV) and the above store product image vectors (PV) based on distance; The above-mentioned similar product extraction unit (190) selects a recommended product (R) from among the products of the brands based on similarity; A method for recommending fashion products using virtual clothing in a smart home environment, characterized by being configured to include
  6. A computer program stored on a storage medium to execute a method for recommending fashion products using virtual clothing in a smart home environment according to any one of claims 1, 3, 4, and 5 on a computer.

Description

Method of providing recommendation of fashion goods by use of virtual clothing in smart home environment The present invention generally relates to a technology that recommends fashion products for online shopping in a smart home environment. In particular, the present invention relates to a fashion product recommendation technology utilizing virtual clothing in a smart home environment, which generates virtual clothing through feature information such as style and color inherent in clothing wearing images collected through a camera installed on a smart home device, and recommends actual brand products similar to images in a database of in-store products to the user based on the generated virtual clothing. The online shopping (mobile shopping) sector has grown rapidly due to the development of communication networks and the widespread adoption of mobile devices. Online shopping holds a distinct advantage over offline shopping as it is not restricted by physical space and can offer customers virtually unlimited products. As the market grows, the number of online shopping companies is increasing dramatically, and competition among them is becoming fiercer day by day. Various efforts have been made to enhance the competitiveness of online shopping sites, and product recommendation features have become the core of this competitiveness. This involves curating the list of products presented on the online shopping site screen to be specialized for individual customers. Conventional product recommendation methods selected recommended products by matching customer personal information with product feature information. This process can be achieved by modeling the interaction between customer personal information and product feature information using vector-based linear algebra theory. Customer personal information (e.g., age, gender, preferences, purchase history, etc.) and product feature information (e.g., style, color, pattern, etc.) are each represented as vectors in a multidimensional space, and content-based, user-based, or hybrid-based recommendation algorithms are applied based on the distance or angle between these vectors in the multidimensional space. Artificial intelligence technology can be applied to model the interaction between customer personal information and product feature information. For example, deep learning recommendation algorithms utilizing Graph Neural Networks (GNNs) can finely model the complex interrelationships between users and products. In a GNN, a graph is constructed where users and products are nodes and their interactions are edges, and then the graph is trained using people's purchase data. Meanwhile, since the fashion product sector is an important part of online shopping, various ideas specialized in fashion product recommendations are being presented. For example, Korean registered patent No. 10-1418372, "Fashion product recommendation system and recommendation method using virtual fitting," is a technology that analyzes a user's product purchasing preferences to select recommended products suitable for the user from among the stocked products, generates a virtual fitting video by combining the video of these recommended products with the customer's avatar video, and transmits it to the user's terminal. Such conventional product recommendation technology has various problems. First, it relies excessively on past data because it is designed based on the customer's purchase history, browsing patterns, and the behavior of similar users. Furthermore, there are limitations in perfectly understanding and reflecting users' unique styles and preferences. While recommendation methods based on users' personal information and product characteristics can reflect user tastes to some extent, it is difficult to adequately reflect changes in user styles over time or their sensitivity to various fashion trends. Furthermore, users' personal information is sensitive and always carries inherent legal risks. When the collection of personal information is strictly managed to reduce legal risks, sufficient preference data may not be secured, leading to a cold-start problem, which degrades product recommendation performance. [Fig. 1] is a flowchart of the fashion product recommendation process according to the present invention. [Fig. 2] is a conceptual diagram of the fashion product recommendation process according to the present invention. [Fig. 3] is an overall configuration diagram of a fashion product recommendation system according to the present invention. [Fig. 4] is a configuration diagram of an artificial intelligence neural network used by a product recommendation server in the present invention. [Fig. 5] is a conceptual diagram of semantic segmentation. [Fig. 6] is a conceptual diagram of the data processing of the virtual image generation unit in the present invention. [Fig. 7] is a conceptual diagram of data processing according to the first embodiment of the similar product extraction