US-12626288-B2 - Boosting scores for ranking items matching a search query

Abstract

An online system receives a search query from a client device associated with a user and queries a database including item data for a set of items matching the query, in which the set of items is at a retailer location associated with a retailer type and each item is associated with an item category. For each item of the set, a machine learning model is applied to predict a probability of conversion for the user and item and a score is computed based on an expected value, in which the expected value is based on a value associated with the item and the probability. The score for each item is boosted based on the item category, retailer type, or a user segment that is based on the user's historical order data. The items are ranked based on the boosted scores and the ranking is sent to the client device.

Inventors

• Vinesh Reddy Gudla
• Tyler Russell Tate
• Tejaswi TENNETI
• Akshay Nair

Assignees

• MAPLEBEAR INC.

Dates

Publication Date

20260512

Application Date

20220928

Claims (17)

1. 1 . A method comprising: at a computer system comprising a processor and a computer-readable medium: receiving a search query from a client device associated with a user of an online system; querying a database comprising item data for a set of items that match at least a portion of the search query, wherein each item of the set of items is associated with an item category; for each item of the set of items: computing an initial score for a corresponding item by applying a first machine learning model to a set of attributes of the user and an additional set of attributes of a corresponding item to predict a probability of the user interacting with the corresponding item through a user interface of the client device, wherein the first machine model comprises a neural network; training a second machine learning model for the item of the corresponding items, wherein the second machine learning model comprises a regression model, wherein the second machine learning model is trained to predict the probability of the user interacting with the corresponding item through a user interface of the client device based on the item category associated with the corresponding item and a user segment associated with the user, wherein the user segment associated with the user is based on historical order data describing the set of orders received from the user; computing a boosting hyperparameter for the second machine learning model, wherein the boosting hyperparameter is a hyperparameter for the second machine learning model for boosting the initial score for the corresponding item, wherein computing the boosting hyperparameter comprises: applying the second machine learning model to order data associated with the user using a first candidate boosting hyperparameter of a plurality of candidate boosting hyperparameters; measuring a performance of the second machine learning model using the first candidate boosting hyperparameter; and adjusting the first candidate boosting hyperparameter to another boosting hyperparameter based on the measured performance of the second machine learning model; and boosting the initial score for the corresponding item to generate a boosted score based on the boosting hyperparameter, the item category associated with the corresponding item, and the user segment associated with the user, wherein the user segment associated with the user is based at least in part on the historical order data describing the set of orders received from the user; ranking the set of items based at least in part on the boosted score for each item; sending the ranking to the client device associated with the user which causes the client device to generate a user interface in a display area of the client device by: generating a plurality of user interface elements that each correspond to one item from the set of items, each user interface element for accessing computer functionality of the client device responsive to a selection of the user interface element by the user; arranging the plurality of user interface elements according to the ranking sent to the client device that is based at least in part on the boosted score for each item from the set of items; and displaying the plurality of user interface elements that are arranged according to the ranking in the user interface; and receiving, from the client device, a selection of one of the plurality of user interface elements that is displayed in the user interface, wherein the computer functionality of the client device that corresponds to the one of the plurality of user interface elements is accessed responsive to receiving the selection.
2. 2 . The method of claim 1 , wherein the boosting hyperparameter for boosting the initial score for the corresponding item is further determined based on the historical order data describing the set of orders received from the user.
3. 3 . The method of claim 1 , wherein the boosting hyperparameter changes an expected value associated with the corresponding item and the boosting hyperparameter is selected from a group consisting of: a coefficient and an exponent.
4. 4 . The method of claim 3 , wherein the expected value, associated with the corresponding item is based at least in part on the boosting hyperparameter a measure of relevance of the corresponding item to the search query, and a price associated with the corresponding item.
5. 5 . The method of claim 1 , wherein the item category associated with the corresponding item is selected from a group consisting of: groceries, alcohol, tobacco, pharmaceuticals, gifts, clothing, accessories, personal care, books, electronics, furniture, houseware, toys, sporting goods, pet supplies, baby supplies, arts and crafts supplies, office supplies, party supplies, cleaning supplies, home improvement, and appliances.
6. 6 . The method of claim 1 , wherein the user segment associated with the user is selected from a group consisting of: a value-oriented user segment and a premium user segment.
7. 7 . The method of claim 6 , wherein the user segment associated with the user is determined based at least in part on one or more features associated with the user selected from a group consisting of: an average total amount spent on each order of the set of orders received from the user, an average percentage of discounted items included in each order of the set of orders received from the user, an average amount spent on each item included in each order of the set of orders received from the user, an average number of items associated with at least a threshold price included in each order of the set of orders received from the user, an average number of items associated with less than a threshold price included in each order of the set of orders received from the user, an average number of organic items included in each order of the set of orders received from the user, a frequency with which the user placed one or more orders included among the set of orders, and one or more retailer types associated with the set of orders received from the user.
8. 8 . The method of claim 7 , further comprising: generating a plurality of data points associated with a plurality of users of the online system, wherein a data point is generated based at least in part on the one or more features associated with a corresponding user of the online system; classifying each of the plurality of data points into a user segment of a plurality of user segments based at least in part on a clustering algorithm; and determining the user segment associated with the user based at least in part on a classification of a data point associated with the user.
9. 9 . A computer program product comprising a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by a processor, cause the processor to: receive a search query from a client device associated with a user of an online system; query a database comprising item data for a set of items that match at least a portion of the search query, wherein the set of items is included among an inventory of a retailer location, the retailer location is associated with a retailer type, and each item of the set of items is associated with an item category; for each item of the set of items: computing an initial score for a corresponding item by applying a first machine learning model to a set of attributes of the user and an additional set of attributes of a corresponding item to predict a probability of the user interacting with the corresponding item through a user interface of the client device, wherein the first machine model comprises a neural network; train a second machine learning model for the item of the corresponding items, wherein the second machine learning model comprises a regression model, wherein the second machine learning model is trained to predict the probability of the user interacting with the corresponding item through a user interface of the client device based on the item category associated with the corresponding item and a user segment associated with the user, wherein the user segment associated with the user is based on historical order data describing the set of orders received from the user; compute a boosting hyperparameter for the second machine learning model, wherein the boosting hyperparameter is a hyperparameter for the second machine learning model for boosting the initial score for the corresponding item, wherein computing the boosting hyperparameter comprises; applying the second machine learning model to order data associated with the user using a first candidate boosting hyperparameter of a plurality of candidate boosting hyperparameters; measuring a performance of the second machine learning model using the first candidate boosting hyperparameter; and adjusting the first candidate boosting hyperparameter to another boosting hyperparameter based on the measured performance of the second machine learning model; and boost the initial score for the corresponding item to generate a boosted score based on the boosting hyperparameter the item category associated with the corresponding item, and the user segment associated with the user, wherein the user segment associated with the user is based at least in part on the historical order data describing the set of orders received from the user; rank the set of items based at least in part on the boosted score for each item; send the ranking to the client device associated with the user which causes the client device to generate a user interface in a display area of the client device by: generating a plurality of user interface elements that each correspond to one item from the set of items, each user interface element for accessing computer functionality of the client device responsive to a selection of the user interface element by the user; arranging the plurality of user interface elements according to the ranking sent to the client device that is based at least in part on the boosted score for each item from the set of items; and displaying the plurality of user interface elements that are arranged according to the ranking in the user interface; and receive, from the client device, a selection of one of the plurality of user interface elements that is displayed in the user interface, wherein the computer functionality of the client device that corresponds to the one of the plurality of user interface elements is accessed responsive to receiving the selection.
10. 10 . The computer program product of claim 9 , wherein the boosting hyperparameter for boosting the initial score for the corresponding item is further determined based on the historical order data describing the set of orders received from the user.
11. 11 . The computer program product of claim 9 , wherein the boosting hyperparameter changes an expected value associated with the corresponding item and the boosting hyperparameter is selected from a group consisting of: a coefficient and an exponent.
12. 12 . The computer program product of claim 11 , wherein the expected value associated with the corresponding item is based at least in part on the boosting hyperparameter, a measure of relevance of the corresponding item to the search query, and a price associated with the corresponding item.
13. 13 . The computer program product of claim 9 , wherein the item category associated with the corresponding item is selected from a group consisting of: groceries, alcohol, tobacco, pharmaceuticals, gifts, clothing, accessories, personal care, books, electronics, furniture, houseware, toys, sporting goods, pet supplies, baby supplies, arts and crafts supplies, office supplies, party supplies, cleaning supplies, home improvement, and appliances.
14. 14 . The computer program product of claim 9 , wherein the retailer type associated with the retailer location is selected from a group consisting of: traditional grocery store, organic grocery store, convenience store, warehouse club store, specialty store, department store, discount store, premium store, and outlet store.
15. 15 . The computer program product of claim 9 , wherein the user segment associated with the user is selected from a group consisting of: a value-oriented user segment and a premium user segment.
16. 16 . The computer program product of claim 15 , wherein the user segment associated with the user is determined based at least in part on one or more features associated with the user selected from a group consisting of: an average total amount spent on each order of the set of orders received from the user, an average percentage of discounted items included in each order of the set of orders received from the user, an average amount spent on each item included in each order of the set of orders received from the user, an average number of items associated with at least a threshold price included in each order of the set of orders received from the user, an average number of items associated with less than a threshold price included in each order of the set of orders received from the user, an average number of organic items included in each order of the set of orders received from the user, a frequency with which the user placed one or more orders included among the set of orders, and one or more retailer types associated with the set of orders received from the user.
17. 17 . A computer system comprising: a processor; and a non-transitory computer readable storage medium storing instructions that, when executed by the processor, perform actions comprising: receiving a search query from a client device associated with a user of an online system; querying a database comprising item data for a set of items that match at least a portion of the search query, wherein each item of the set of items is associated with an item category; for each item of the set of items: computing an initial score for a corresponding item by applying a first machine learning model to a set of attributes of the user and an additional set of attributes of a corresponding item to predict a probability of the user interacting with the corresponding item through a user interface of the client device, wherein the first machine model comprises a neural network; training a second machine learning model for the item of the corresponding items, wherein the second machine learning model comprises a regression model, wherein the second machine learning model is trained to predict the probability of the user interacting with the corresponding item through a user interface of the client device based on the item category associated with the corresponding item and a user segment associated with the user, wherein the user segment associated with the user is based on historical order data describing the set of orders received from the user; computing a boosting hyperparameter for the second machine learning model, wherein the boosting hyperparameter is a hyperparameter for the second machine learning model for boosting the initial score for the corresponding item, wherein computing the boosting hyperparameter comprises: applying the second machine learning model to order data associated with the user using a first candidate boosting hyperparameter of a plurality of candidate boosting hyperparameters; measuring a performance of the second machine learning model using the first candidate boosting hyperparameter; and adjusting the first candidate boosting hyperparameter to another boosting hyperparameter based on the measured performance of the second machine learning model; and boosting the initial score for the corresponding item to generate a boosted score based on the boosting hyperparameter, the item category associated with the corresponding item, and the user segment associated with the user, wherein the user segment associated with the user is based at least in part on the historical order data describing the set of orders received from the user; ranking the set of items based at least in part on the boosted score for each item; sending the ranking to the client device associated with the user which causes the client device to generate a user interface in a display area of the client device by: generating a plurality of user interface elements that each correspond to one item from the set of items, each user interface element for accessing computer functionality of the client device responsive to a selection of the user interface element by the user; arranging the plurality of user interface elements according to the ranking sent to the client device that is based at least in part on the boosted score for each item from the set of items; and displaying the plurality of user interface elements that are arranged according to the ranking in the user interface; and receiving, from the client device, a selection of one of the plurality of user interface elements that is displayed in the user interface, wherein the computer functionality of the client device that corresponds to the one of the plurality of user interface elements is accessed.

Description

BACKGROUND Online systems, such as online concierge systems and online retailers, may receive requests from their users to search for items available through the online systems in the form of search queries. In response, the online systems may return lists of search results matching the search queries. Items included in a list of results may be ordered based on their relevance to a search query, such that items that are more relevant appear in more prominent positions in the list of results than less relevant items. For example, if a user provides a search query with the words “chocolate chips” in an ordering interface provided by an online system, the online system may return a list of items ordered based on their relevance to the search query, (e.g., with chocolate chips at the top, followed by chocolate chip cookies, chocolate chip ice cream, etc.). Alternatively, since online systems may receive compensation that is proportional to values associated with items purchased by their users (e.g., in the form of commissions for online concierge systems), online systems may order items included in search results based on their values to maximize revenue. In the above example, the online system may order items based on their prices instead of their relevance, such that more expensive items appear higher in the list than less expensive items. However, since items included in the most prominent positions of lists of search results that are ordered based on values associated with items may not be that relevant to users searching for the items, the users may become frustrated and purchase fewer items or even refrain from purchasing items altogether, reducing revenue earned by the online systems. In the above example, if the chocolate chips for which the user is searching are relatively inexpensive, several items that appear higher in the list than the chocolate chips (e.g., chocolate chip cakes, chocolate chip cookies, chocolate chip ice cream, chocolate chip brownies, etc.) may make it difficult for the user to find them among the list of results. Additionally, in the above example, if different brands of chocolate chips have very different prices such that they do not appear near each other in the search results, the user may refrain from purchasing the chocolate chips if they wanted to compare different brands of chocolate chips before purchasing them and their ordering within the list makes it difficult for the user to do so. SUMMARY In accordance with one or more aspects of the disclosure, an online system ranks items matching a search query based on an item category, a retailer type, or a user segment. More specifically, the online system receives a search query from a client device associated with a user of the online system and queries a database including item data for a set of items matching the search query, in which the set of items is included among an inventory of a retailer location associated with a retailer type and each of the set of items is associated with an item category. For each item included among the set of items, the online system applies a machine learning model to a set of attributes of the user and an additional set of attributes of the item to predict a probability of conversion for the user and the item and computes a score for the item based on an expected value associated with the item. The expected value associated with an item is based on a value (e.g., a price) associated with the item and the probability of conversion for the user and the item. The online system also boosts the score for each item based on the item category associated with the item, the retailer type, or a user segment associated with the user, in which the user segment is based on historical order data associated with the user. The online system then ranks the set of items based on the boosted scores and sends the ranking to a display area of the client device. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 illustrates an example system environment for an online system, in accordance with one or more embodiments. FIG. 2 illustrates an example system architecture for an online system, in accordance with one or more embodiments. FIG. 3 is a flowchart of a method for ranking items matching a search query based on an item category, a retailer type, or a user segment, in accordance with one or more embodiments. FIGS. 4A and 4B illustrate a conceptual diagram of a method for ranking items matching a search query based on an item category, a retailer type, or a user segment, in accordance with one or more embodiments. DETAILED DESCRIPTION FIG. 1 illustrates an example system environment for an online system 140, such as an online concierge system, in accordance with one or more embodiments. The system environment illustrated in FIG. 1 includes a customer client device 100, a picker client device 110, a retailer computing system 120, a network 130, and an online system 140. Alternative embodiments may include more, fewer, o