US-12626126-B2 - Cold-start recommendation system using shared neural item representations with fixed weight initialization

US12626126B2US 12626126 B2US12626126 B2US 12626126B2US-12626126-B2

Abstract

The present disclosure relates to an improved machine learning-based recommender system and method for cold-start predictions on an ecommerce platform. The improved system predicts user-item interactions with respect to cold-start items in which only side information is available. Item representations generated by an item neural network encoder from item side information are shared with a user neural network. The item representations are used, along with user feedback history, to generate user representations. Specifically, a weight matrix in the first layer of the user neural network encoder is fixed with the shared item embeddings. The effect of this is that, when the user neural network encoder is applied to an input user-item interaction vector, the output of the first layer of the user neural network encoder is a function of the item representations of the items for which the user provided positive feedback. The result is a recommender system that achieves better performance for cold-start items with fewer training iterations.

Inventors

Ramin Raziperchikolaei

Assignees

RAKUTEN GROUP, INC.

Dates

Publication Date: 20260512
Application Date: 20211220

Claims (16)

1 . A method, performed by a computer system, for predicting an interaction value for a user and an item, the method comprising: performing the following with respect to a training phase: (a) obtaining a training dataset for training users and items, wherein the training users and the items form user-item pairs, wherein the training dataset includes item data for each item of the items in the training dataset and a user interaction vector for each training user of the training users, wherein the user interaction vector is obtained from an interaction matrix with an interaction value for each user-item pair of the user-item pairs in the training dataset; (b) using an item neural network encoder to compute an item vector representation for each item of the items in the training dataset for which at least one training user has a positive interaction value, wherein the item neural network encoder is a multilayer perceptron; (c) setting a weight matrix in a first layer of a user neural network encoder using item vector representations comprising the item vector representation for each item of the items, wherein the user neural network encoder is also a multilayer perceptron; (d) for each user-item pair of the user-item pairs in the training dataset, performing the following: computing a user vector representation by applying the user neural network encoder to the user interaction vector for the training user, wherein in computing the user vector representation for the training user, output of the first layer of user neural network encoder is a function of the item vector representations of the items with which the training user has a positive interaction value; computing an item vector representation by applying the item neural network encoder to item data for an item of the items in each user-item pair of the user-item pairs; and computing a predicted interaction value for the user-item pair based on the user and item vector representations generated for each user-item pair of the user-item pairs; (e) calculating a loss between predicted interaction values and actual interaction values for the user-item pairs in the training dataset; (f) adjusting trainable parameters of the item and user neural network encoders to minimize the loss, wherein the weight matrix in the first layer of the user neural network encoder is fixed and excluded from said trainable parameters; (g) repeating steps (b)-(f) for a number of iterations; performing the following with respect to a prediction phase: (h) computing item vector representations for a plurality of items, wherein the item vector representations are generated by applying the item neural network encoder to item data for the plurality of items; (i) setting the weight matrix in the first layer of the user neural network encoder using said item vector representations; (j) performing the following with respect to a test user: obtaining a user interaction vector for the test user; obtaining item data for an item k for which there is no known interaction value with the test user; computing a user vector representation for the test user by applying the user neural network encoder to the user interaction vector, wherein output of the first layer of the user neural network encoder is a function of the item vector representations for items of the plurality of items with which the test user has a positive interaction value; computing an item vector representation for item k by applying the item neural network encoder to item data for item k; and computing a predicted interaction value for the test user and item k based on the user and item vector representations computed for the test user and item k.
2 . The method of claim 1 , further comprising: repeating step (j) with respect to a plurality of items for which there is no known interaction value for the test user; and recommending one or more items to the test user based on the predicted interaction values computed for the test user and the plurality of items.
3 . The method of claim 1 further comprising, repeating step (j) with respect to a plurality of different test users; recommending item k to one or more of the test users based on the predicted interaction values for test users and item k.
4 . The method of claim 1 , wherein computing the output of the first layer of the user neural network encoder comprises: applying a fusion function to the item vector representations of the items with which the user has a positive interaction value to map said item vector representations to a single vector; and using said single vector as the output of the first layer of the user neural network encoder.
5 . The method claim 4 , wherein the fusion function is an average function that averages the item vector representations.
6 . The method of claim 4 , wherein the fusion function is a weighted average function that computes a weighted average of the item vector representations based on the similarity of said item vector representations to the item vector representation of the item for which an interaction value is being predicted.
7 . The method of claim 1 , wherein the plurality of items in step (h) are items for sale on an ecommerce platform.
8 . The method of claim 1 , wherein item k is a cold start item.
9 . The method of claim 1 , wherein the predicted interaction value for a user-item pair is based on the dot product between the user vector representation and the item vector representation for the pair.
10 . A non-transitory computer-readable medium comprising a computer program, that, when executed by a computer system, enables the computer system to perform the following method for training a machine-learning model to predict an interaction value for a user and an item, the method comprising: (a) obtaining a training dataset for training users and items, wherein the training users and the items form user-item pairs, wherein the training dataset includes item data for each item of the items in the training dataset and a user interaction vector for each training user of the training users, wherein the user interaction vector is obtained from an interaction matrix with an interaction value for each user-item pair of the user-item pairs in the training dataset; (b) using an item neural network encoder to compute an item vector representation for each item of the items in the training dataset for which at least one training user has a positive interaction value, wherein the item neural network encoder is a multilayer perceptron; (c) setting a weight matrix in a first layer of a user neural network encoder using item vector representations comprising the item vector representation for each item of the items, wherein the user neural network encoder is also a multilayer perceptron; (d) for each user-item pair of the user-item pairs in the training dataset, performing the following: computing a user vector representation by applying the user neural network encoder to the user interaction vector for the training user, wherein in computing the user vector representation for the training user, output of the first layer of user neural network encoder is a function of the item vector representations of the items with which the training user has a positive interaction value; computing an item vector representation by applying the item neural network encoder to item data for an item of the items in each user-item pair of the user-item pairs; and computing a predicted interaction value for the user-item pair based on the user and item vector representations generated for each user-item pair of the user-item pairs; (e) calculating a loss between predicted interaction values and actual interaction values for the user-item pairs in the training dataset; (f) adjusting trainable parameters of the item and user neural network encoders to minimize the loss, wherein the weight matrix in the first layer of the user neural network encoder is fixed and excluded from said trainable parameters; and (g) repeating steps (b)-(f) for a number of iterations.
11 . The non-transitory computer-readable medium of claim 10 , wherein computing the output of the first layer of the user neural network encoder comprises: applying a fusion function to the item vector representations of the items with which the user has a positive interaction value to map said item vector representations to a single vector; and using said single vector as the output of the first layer of the user neural network encoder.
12 . The non-transitory computer-readable medium of claim 11 , wherein the fusion function is an average function that averages the item vector representations.
13 . The non-transitory computer-readable medium of claim 11 , wherein the fusion function is a weighted average function that computes a weighted average of the item vector representations based on the similarity of said item vector representations to the item vector representation of the item for which an interaction value is being predicted.
14 . A non-transitory computer-readable medium comprising a computer program, that, when executed by a computer system, enables the computer system to perform the following method for predicting an interaction value for a user and an item, the method comprising: (a) computing item vector representations for a plurality of items, wherein the item vector representations are generated by applying an item neural network encoder to item data for the plurality of items, wherein the item neural network encoder is a multilayer perceptron; (b) setting a weight matrix in a first layer of a user neural network encoder using said item vector representations, wherein the user neural network encoder is also a multilayer perceptron; (c) performing the following with respect to a test user: obtaining a user interaction vector for the test user; obtaining item data for an item k for which there is no known interaction value with the test user; computing a user vector representation for the test user by applying the user neural network encoder to the user interaction vector, wherein output of the first layer of the user neural network encoder is a function of the item vector representations for items of the plurality of items with which the test user has a positive interaction value; computing an item vector representation for item k by applying the item neural network encoder to item data for item k; and computing a predicted interaction value for the test user and item k based on the user and item vector representations computed for the test user and item k; (d) repeating step (c) with respect to a plurality of items for which there is no known interaction value for the test user, and with respect to a plurality of different test users; (e) in response to repeating step (c) with respect to a plurality of items for which there is no known interaction value for the test user, recommending one or more items to the test user based on the predicted interaction values computed for the test user and the plurality of items; and (f) in response to repeating step (c) with respect to a plurality of different test users, recommending item k to one or more of the test users based on the predicted interaction values for test users and item k.
15 . A system for predicting user-item interaction values on an ecommerce platform comprising: a processor configured to execute instructions programmed using a set of machine codes; one or more memory units coupled to the processor; and a machine-learning model, stored in the one or more memory units of the system, that takes a user input and an item input and outputs a predicted user-interaction value for users and items on the ecommerce platform, wherein the machine-learning model includes computational instructions implemented in the machine code for generating the output, and wherein the machine-learning model is trained according to a method that comprises the following: (a) obtaining a training dataset for training users and items, wherein the training users and the items form user-item pairs, wherein the training dataset includes item data for each item of the items in the training dataset and a user interaction vector for each training user of the training users, wherein the user interaction vector is obtained from an interaction matrix with an interaction value for each user-item pair of the user-item pairs in the training dataset; (b) using an item neural network encoder to compute an item vector representation for each item of the items in the training dataset for which at least one training user has a positive interaction value, wherein the item neural network encoder is a multilayer perceptron; (c) setting a weight matrix in a first layer of a user neural network encoder using item vector representations comprising the item vector representation for each item of the items, wherein the user neural network encoder is also a multilayer perceptron; (d) for each user-item pair of the user-item pairs in the training dataset, performing the following: computing a user vector representation by applying the user neural network encoder to the user interaction vector for the training user, wherein in computing the user vector representation for the training user, output of the first layer of user neural network encoder is a function of the item vector representations of the items with which the training user has a positive interaction value; computing an item vector representation by applying the item neural network encoder to item data for an item of the items in each user-item pair of the user-item pairs; and computing a predicted interaction value for the user-item pair based on the user and item vector representations generated for each user-item pair of the user-item pairs; (e) calculating a loss between predicted interaction values and actual interaction values for the user-item pairs in the training dataset; (f) adjusting trainable parameters of the item and user neural network encoders to minimize the loss, wherein the weight matrix in the first layer of the user neural network encoder is fixed and excluded from said trainable parameters; and (g) repeating steps (b)-(f) for a number of iterations.
16 . The system of claim 15 , wherein computing the output of the first layer of the user neural network encoder comprises: applying a fusion function to the item vector representations of the items with which the user has a positive interaction value to map said item vector representations to a single vector; and using said single vector as the output of the first layer of the user neural network encoder.

Description

RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Application No. 63/230,676 filed on Aug. 6, 2021, and titled “Shared Neural Item Representations for Completely Cold Start Problem,” the contents of which are incorporated by reference herein as if fully disclosed herein. This application also claims the benefit of U.S. Provisional Application No. 63/234,651 filed on Aug. 18, 2021, and titled “Shared Neural Item Representations for Completely Cold Start Problem,” the contents of which are incorporated by reference herein as if fully disclosed herein. BACKGROUND OF THE INVENTION 1. Field of the Invention This invention relates generally to machine learning-based recommender systems and, more specifically, to a recommender system that uses shared neural item representations for cold-start recommendations. 2. Description of the Background Art Recommender systems are used on many shopping platforms. The goal of a recommender system is to take known user feedback and predict unknown user feedback on an item. The feedback, often referred to as a user-item interaction, can be implicit (e.g., purchased/likely to purchase) or explicit (e.g., a rating between 1 and 5). The predicted feedback can be used to either recommend items to users or to provided targeted advertising for items on a shopping platform. Many recommender systems use machine learning models to make predictions. For example, neural collaborative filtering (NCF) systems apply neural network encoders to user and item data to generate representations of the user and item data, and then use the user and item representations to predict user feedback for the items. Recommender systems may use various input sources to fulfill their tasks. One important source of information is users' previous feedback on items. For items with high sales volumes on a platform, best prediction results are often achieved when both the inputs to the user and item neural network includes feedback data. However, this does not work for “cold-start items.” Cold-start items are items for which there is either no feedback history or insufficient history from which to make predictions related to the item. To predict user-item interactions for cold start items, one known solution is to use user feedback for input to the user neural network encoder and to use item side information as input to the item neural network encoder. During training, the parameters of the neural network encoders are learned such that the encoders generate similar user and item representations for user and item pairs with known positive interactions and dissimilar representations otherwise. Training neural networks is computationally intensive. The more training data, the better predictions. However, in the system described above, the number of encoder parameters that need to be learned during training increases almost linearly as the number of user and items increases. As entities rely more and more on recommender system, there is demand for continued performance improvement in such systems with fewer training iterations. SUMMARY OF THE DISCLOSURE The present disclosure relates to an improved machine learning-based recommender system and method for cold-start predictions on an ecommerce platform. The improved system predicts user-item interactions with respect to cold-start items in which only side information is available. Like known solutions, the system predicts user-item interaction values based on user and item representations generated by user and item neural network encoders, respectively. However, unlike known solutions, the item representations generated by the item neural network encoder from item side information are shared with the user neural network encoder. Specifically, instead of initializing a weight matrix in the first layer of the user neural network encoder with randomly-generated embeddings and then learning the optimal weights during training, the weight matrix is fixed with the item embeddings generated by the item neural network encoder and shared with the user neural network encoder. To generate user representations, the user neural network is applied to an input user-item interaction vector. The input vector has a “1” value for each item with which a user has had a positive interaction and a “0” value for all other items. The combined effect of using the user-item interaction vector as input and using the shared item representations as weights in the first layer of the user neural network encoder is that the output of the first layer of the user neural network encoder is a function of the item representations of the items for which the user provided positive feedback. This is different from known systems in which the output of the first layer is a function of randomly-initialized weights that have to be learned during training. The improvements result in better performance with fewer training iterations. In one embodiment, a method for predicting an interactio