US-12619618-B1 - Query dependent threshold generation for search result filtering

Abstract

Query dependent threshold generation for search result filtering is described. In one or more implementations, a user query entered via a search platform is received, and in response, items are retrieved from a storage device based on the user query. Using a first machine learning model, relevance scores are generated for the items, and the relevance scores represent degrees of relevance of respective items with respect to the user query. Using a second machine learning model, a relevance threshold is generated for the user query based on one or more features of the user query. The items are filtered based on the relevance scores and the relevance threshold, and the filtered items are communicated over a network for display in a user interface of the search platform.

Inventors

• Shreya Gupta
• Nadia G Vase
• John Degenhardt

Assignees

• EBAY INC.

Dates

Publication Date

20260505

Application Date

20241218

Claims (20)

1. 1 . A method implemented by at least one computing device, the method comprising: receiving a user query entered via a search platform; retrieving, from a storage device, items based on the user query; generating, using a first machine learning model, a relevance threshold for the user query based on one or more features of the user query which capture a degree of specificity of the user query; distributing the items across a plurality of database shards hosted by different servers; processing, by the different servers, the items in parallel across the plurality of database shards by: generating, using a second machine learning model, relevance scores for the items representing degrees of relevance of respective items with respect to the user query; and filtering the items based on the relevance scores and the relevance threshold; and communicating, over a network, the filtered items for display in a user interface of the search platform.
2. 2 . The method of claim 1 , wherein generating the relevance threshold includes: generating, using a first function of the first machine learning model, an intermediate relevance threshold based on the one or more features; and generating, using a second function of the first machine learning model, the relevance threshold by reducing the intermediate relevance threshold by a relaxation factor.
3. 3 . The method of claim 2 , further comprising: receiving training data including a plurality of training samples, each training sample including a training query, and training items having been engaged with by users of the search platform responsive to the training query; and training the first function and the second function of the first machine learning model using the training data.
4. 4 . The method of claim 3 , wherein training the first function includes: generating, using the second machine learning model, training relevance scores for the training items of a training sample of the plurality of training samples; generating, using the first function, a predicted intermediate relevance threshold based on the one or more features of the training query of the training sample; and training the first function based on a comparison of a target value of the training relevance scores and the predicted intermediate relevance threshold.
5. 5 . The method of claim 4 , wherein training the second function includes: repeating the generating the training relevance scores and the generating the predicted intermediate relevance threshold for each of the plurality of training samples; computing, for each of the plurality of training samples, a difference between the predicted intermediate relevance threshold and the target value; and determining, as the relaxation factor, a function of a standard deviation of the differences.
6. 6 . The method of claim 4 , wherein the target value is a minimum value of the training relevance scores.
7. 7 . The method of claim 4 , further comprising ranking the training items for display in the user interface in a display order, wherein the target value is a weighted sum of the training relevance scores having weights determined based on the display order.
8. 8 . The method of claim 3 , further comprising tuning the relaxation factor using the training data.
9. 9 . The method of claim 1 , wherein the one or more features include a number of the items retrieved based on the user query, a number of categories to which the items belong, a number of tokens in the user query, constraints associated with the user query, and one or more categories into which the user query is classified.
10. 10 . The method of claim 1 , wherein filtering the items includes: identifying relevant items having one or more of the relevance scores that exceed the relevance threshold; ranking the items in a display order for display in the user interface, the display order including the relevant items ranked above irrelevant items having the relevance scores that are below the relevance threshold; and communicating, over the network, the items for display in the user interface in the display order.
11. 11 . The method of claim 10 , wherein ranking the items includes: determining a first ranking of the items, resulting in one or more top-ranked subsets of the items; determining a second ranking of the relevant items in the one or more top-ranked subsets; and determining the display order based on the first ranking and the second ranking.
12. 12 . The method of claim 1 , wherein filtering the items includes discarding irrelevant items having the relevance scores that are below the relevance threshold.
13. 13 . A non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations including: receiving a user query entered via a search platform; retrieving, from a storage device, items based on the user query; generating, using a first machine learning model, a relevance threshold for the user query based on one or more features of the user query which capture a degree of specificity of the user query; distributing the items across a plurality of database shards hosted by different servers; processing, by the different servers, the items in parallel across the plurality of database shards by: generating, using a second machine learning model, relevance scores for the items representing degrees of relevance of respective items with respect to the user query; and filtering the items based on the relevance scores and the relevance threshold; and communicating, over a network, the filtered items for display in a user interface of the search platform.
14. 14 . The non-transitory computer-readable media of claim 13 , the operations further including: receiving a training sample including a training query, and training items having been engaged with by users of the search platform responsive to the training query; generating, using the second machine learning model, training relevance scores for the training items; generating, using the first machine learning model, a predicted relevance threshold based on the one or more features of the training query; and training the first machine learning model based on a minimum value of the training relevance scores and the predicted relevance threshold.
15. 15 . The non-transitory computer-readable media of claim 14 , wherein training the first machine learning model includes: determining a delta value based on the training relevance scores; determining a target relevance threshold by reducing the minimum value by the delta value; and training the first machine learning model based on a comparison of the predicted relevance threshold and the target relevance threshold.
16. 16 . The non-transitory computer-readable media of claim 15 , wherein determining the delta value includes computing, as the delta value, a function of a standard deviation of the training relevance scores.
17. 17 . The non-transitory computer-readable media of claim 15 , wherein determining the delta value includes: determining differences between the minimum value and the training relevance scores; and computing, as the delta value, a function of a standard deviation of the differences.
18. 18 . The non-transitory computer-readable media of claim 15 , wherein determining the delta value includes: determining differences between the minimum value and the training relevance scores; and computing, as the delta value, a function of a lower bound of a confidence interval of the differences.
19. 19 . The non-transitory computer-readable media of claim 15 , wherein determining the delta value includes computing, as the delta value, a function of a fraction distance between a highest value of the training relevance scores and a lowest value of the training relevance scores.
20. 20 . A system comprising: one or more processors; and memory storing instructions that, when executed by the one or more processors, cause the system to: receive a user query entered via a search platform; retrieve, from a storage device, items based on the user query; generate, using a first machine learning model, a relevance threshold for the user query based on one or more features of the user query which capture a degree of specificity of the user query; distribute the items across a plurality of database shards hosted by different servers; process, by the different servers, the items in parallel across the plurality of database shards by: generating, using a second machine learning model, relevance scores for the items representing degrees of relevance of respective items with respect to the user query; and filtering the items based on the relevance scores and the relevance threshold; and communicate, over a network, the filtered items for display in a user interface of the search platform.

Description

BACKGROUND Search engines allow users to retrieve relevant information from a dataset by entering user queries, which the search engine matches against indexed content to deliver search results that satisfy the searching user's intent as defined by a user query. To enhance user experience, modern search engines use ranking algorithms to evaluate and prioritize results based on a multitude of factors, one example of which is relevance of search results to the user query. Indeed, search result quality is heavily influenced by whether search results surfaced to a searching user are relevant to the user's query, and accurately reflect the user's searching intent. SUMMARY Query dependent threshold generation for search result filtering is described. As part of this, a search platform receives a user query, and retrieves items from a storage device that match the user query. Using a relevance scoring model, the search platform generates relevance scores for the items representing degrees of relevance of respective items with respect to the user query. In addition, the search platform uses the contextual thresholding model to generate a relevance threshold for the user query based on query features of the user query. In various implementations, the query features capture a degree of specificity of the user query. Once the relevance threshold is determined, the search platform filters the retrieved items based on the relevance scores of the retrieved items and the relevance threshold. For example, the search platform filters out (e.g., discards) the items having relevance scores that do not meet the relevance threshold. Furthermore, the search platform communicates the filtered items for display in a user interface of the search platform. Notably, the contextual thresholding model is trained on a dataset including a plurality of training samples, each having a training query previously entered via the search platform, and engagement items having been engaged with responsive to the training query being entered via the listing platform. In some examples, the contextual thresholding model is trained to generate intermediate relevance thresholds that reflect a minimum value of the relevance scores (e.g., as determined by the relevance scoring model) exhibited by the engagement items, and reduce the intermediate relevance threshold by a learned relaxation factor. Additionally or alternatively, the contextual thresholding model is trained to generate relevance thresholds that reflect a relaxed (e.g., reduced) version of a minimum relevance score (e.g., as determined by the relevance scoring model) exhibited by the engagement items. This Summary introduces a selection of concepts in a simplified form that are further described below in the Detailed Description. As such, this Summary is not intended to identify essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS The detailed description is described with reference to the accompanying figures. FIG. 1 is an illustration of an environment in an example implementation that is operable to employ techniques described herein. FIG. 2 depicts an example of a search platform determining a display order for search results in accordance with one or more implementations. FIG. 3 depicts an example of a contextual thresholding model outputting a relevance threshold in accordance with one or more implementations. FIG. 4 depicts an example of training a first function of a contextual thresholding model in accordance with one or more implementations. FIG. 5 depicts an example of training a second function of a contextual thresholding model in accordance with one or more implementations. FIG. 6 depicts an example of a search platform training a contextual thresholding model in accordance with one or more implementations. FIG. 7 depicts an example of a user interface displayable in accordance with the described techniques. FIG. 8 depicts a procedure in an example implementation of query dependent threshold generation for search result filtering. FIG. 9 depicts a procedure in an example implementation of query dependent threshold generation for search result filtering. FIG. 10 illustrates an example of a system that may implement the various techniques described herein. DETAILED DESCRIPTION Overview Search platforms utilize thresholding to filter search results returned responsive to a user query that are irrelevant to a user query. For instance, search platforms filter out search results that fail to meet a minimum quality and/or relevance threshold. However, conventional techniques for threshold-based filtering use predefined thresholds for all user queries entered via a search platform, and as such, fail to account for the nuances, context, and breadth of the user query when crafting the filtering threshold. These conventional approaches, thus, do not consider the