US-12625922-B2 - Greedy inference for resource-efficient matching of entities

US12625922B2US 12625922 B2US12625922 B2US 12625922B2US-12625922-B2

Abstract

Methods, systems, and computer-readable storage media for determining a set of potential probability thresholds based on a set of inference results provided by processing testing data through the ML model, for each potential probability threshold in the set of potential probability thresholds, determining an accuracy, selecting a probability threshold from the set of potential probability thresholds, processing an inference job including sets of entity pairs through the ML model to assign a label to each entity pair in the sets of entity pairs, each label being associated with a probability and including a type of multiple types, and for each entity pair having a label of one or more specified types, selectively removing an entity of the entity pair from further processing of the inference job by the ML model based on whether the probability associated with the label meets or exceeds the probability threshold.

Inventors

Sundeep Gullapudi

Assignees

SAP SE

Dates

Publication Date: 20260512
Application Date: 20211116

Claims (17)

1 . A computer-implemented method for matching entities using a machine learning (ML) model, the method being executed by one or more processors and comprising: during a pre-deployment phase: determining a set of potential probability thresholds based on a set of inference results provided by processing testing data through the ML model, each inference result being associated with a probability indicating a confidence that the respective inference result is correct, each probability being included in the set of potential probability thresholds; for each potential probability threshold in the set of potential probability thresholds, determining an accuracy as a number of correct values predicted at or above the respective potential probability threshold among all values predicted at or above the respective potential probability threshold; and selecting a probability threshold from the set of potential probability thresholds; deploying the ML model for inference in a deployment phase, the ML model being deployed with the probability threshold to reduce a number of entities from inference jobs processed by the ML model; and during the deployment phase: processing an inference job comprising a first set of entity pairs through the ML model to assign a label to each entity pair in the set entity pairs, each label being associated with a probability and comprising a type of multiple types; and for each entity pair having a label of one or more specified types, selectively removing an entity of the entity pair from further processing of the inference job by the ML model based on whether the probability associated with the label meets or exceeds the probability threshold; wherein selectively removing an entity of the entity pair from further processing of the inference job by the ML model comprises adding a key of the entity to a set of matched keys in response to determining that the probability associated with the label, wherein the set of matched keys is used to selectively filter entities from being processed in the inference job.
2 . The method of claim 1 , wherein the probability threshold is selected as a lowest potential probability threshold in the set of potential probability thresholds having an accuracy that meets or exceeds a target accuracy.
3 . The method of claim 1 , wherein the one or more specified types comprise one or more of a single match and a multi-match.
4 . The method of claim 1 , wherein the set of potential probability thresholds comprises unique probabilities included in the inference results.
5 . The method of claim 1 , further comprising: determining a set of keys for a set of entities, each key in the set of keys uniquely identifying an entity; comparing keys in the set of keys to matched keys in a set of matched keys; and removing an entity from the set of entities in response to determining that a key identifying the entity is included in the set of matched keys.
6 . The method of claim 1 , wherein each entity pair comprises a query entity and a target entity, the target entity being selectively removed based on whether the probability associated with the label meets or exceeds the probability threshold.
7 . A non-transitory computer-readable storage medium coupled to one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations for matching entities using a machine learning (ML) model, the operations comprising: during a pre-deployment phase: determining a set of potential probability thresholds based on a set of inference results provided by processing testing data through the ML model, each inference result being associated with a probability indicating a confidence that the respective inference result is correct, each probability being included in the set of potential probability thresholds; for each potential probability threshold in the set of potential probability thresholds, determining an accuracy as a number of correct values predicted at or above the respective potential probability threshold among all values predicted at or above the respective potential probability threshold; and selecting a probability threshold from the set of potential probability thresholds; deploying the ML model for inference in a deployment phase, the ML model being deployed with the probability threshold to reduce a number of entities from inference jobs processed by the ML model; and during the deployment phase: processing an inference job comprising a first set of entity pairs through the ML model to assign a label to each entity pair in the set entity pairs, each label being associated with a probability and comprising a type of multiple types; and for each entity pair having a label of one or more specified types, selectively removing an entity of the entity pair from further processing of the inference job by the ML model based on whether the probability associated with the label meets or exceeds the probability threshold, wherein selectively removing an entity of the entity pair from further processing of the inference job by the ML model comprises adding a key of the entity to a set of matched keys in response to determining that the probability associated with the label, wherein the set of matched keys is used to selectively filter entities from being processed in the inference job.
8 . The non-transitory computer-readable storage medium of claim 7 , wherein the probability threshold is selected as a lowest potential probability threshold in the set of potential probability thresholds having an accuracy that meets or exceeds a target accuracy.
9 . The non-transitory computer-readable storage medium of claim 7 , wherein the one or more specified types comprise one or more of a single match and a multi-match.
10 . The non-transitory computer-readable storage medium of claim 7 , wherein the set of potential probability thresholds comprises unique probabilities included in the inference results.
11 . The non-transitory computer-readable storage medium of claim 7 , wherein operations further comprise: determining a set of keys for a set of entities, each key in the set of keys uniquely identifying an entity; comparing keys in the set of keys to matched keys in a set of matched keys; and removing an entity from the set of entities in response to determining that a key identifying the entity is included in the set of matched keys.
12 . The non-transitory computer-readable storage medium of claim 7 , wherein each entity pair comprises a query entity and a target entity, the target entity being selectively removed based on whether the probability associated with the label meets or exceeds the probability threshold.
13 . A system, comprising: a computing device; and a computer-readable storage device coupled to the computing device and having instructions stored thereon which, when executed by the computing device, cause the computing device to perform operations for matching entities using a machine learning (ML) model, the operations comprising: during a pre-deployment phase: determining a set of potential probability thresholds based on a set of inference results provided by processing testing data through the ML model, each inference result being associated with a probability indicating a confidence that the respective inference result is correct, each probability being included in the set of potential probability thresholds; for each potential probability threshold in the set of potential probability thresholds, determining an accuracy as a number of correct values predicted at or above the respective potential probability threshold among all values predicted at or above the respective potential probability threshold; and selecting a probability threshold from the set of potential probability thresholds; deploying the ML model for inference in a deployment phase, the ML model being deployed with the probability threshold to reduce a number of entities from inference jobs processed by the ML model; and during the deployment phase: processing an inference job comprising a first set of entity pairs through the ML model to assign a label to each entity pair in the set entity pairs, each label being associated with a probability and comprising a type of multiple types; and for each entity pair having a label of one or more specified types, selectively removing an entity of the entity pair from further processing of the inference job by the ML model based on whether the probability associated with the label meets or exceeds the probability threshold: wherein selectively removing an entity of the entity pair from further processing of the inference job by the ML model comprises adding a key of the entity to a set of matched keys in response to determining that the probability associated with the label, wherein the set of matched keys is used to selectively filter entities from being processed in the inference job.
14 . The system of claim 13 , wherein the probability threshold is selected as a lowest potential probability threshold in the set of potential probability thresholds having an accuracy that meets or exceeds a target accuracy.
15 . The system of claim 13 , wherein the one or more specified types comprise one or more of a single match and a multi-match.
16 . The system of claim 13 , wherein the set of potential probability thresholds comprises unique probabilities included in the inference results.
17 . The system of claim 13 , wherein operations further comprise: determining a set of keys for a set of entities, each key in the set of keys uniquely identifying an entity; comparing keys in the set of keys to matched keys in a set of matched keys; and removing an entity from the set of entities in response to determining that a key identifying the entity is included in the set of matched keys.

Description

BACKGROUND Enterprises continuously seek to improve and gain efficiencies in their operations. To this end, enterprises employ software systems to support execution of operations. Recently, enterprises have embarked on the journey of so-called intelligent enterprise, which includes automating tasks executed in support of enterprise operations using machine learning (ML) systems. For example, one or more ML models are each trained to perform some task based on training data. Trained ML models are deployed, each receiving input (e.g., a computer-readable document) and providing output (e.g., classification of the computer-readable document) in execution of a task (e.g., document classification task). ML systems can be used in a variety of problem spaces. An example problem space includes autonomous systems that are tasked with matching items of one entity to items of another entity. Examples include, without limitation, matching questions to answers, people to products, bank statements to invoices, and bank statements to customer accounts. In a traditional approach, during inference, each entity (record) from a query set is compared all of the entities in a target set to get the probabilities of matches between the respective entity pairs. That is, each record (entity) of the query set is compared to all records (entities) of the target set. By this inference process, the traditional approach duplicates comparison of entities, which increases the time required to conduct inference as well as computing resources (e.g., processors, memory). Accordingly, the traditional approach is not optimized and results in significant computational costs (e.g., expending processors, memory). This problem is exacerbated when the entity matching task involves large numbers of entities (e.g., millions of target entities and over a hundred thousand query entities). Further, scaling of query entities to target entities is limited because, as the number of entities increases, the computational costs exponentially increase. SUMMARY Implementations of the present disclosure are directed to decreasing resource consumption in matching of entities using one or more ML models. More particularly, implementations of the present disclosure are directed to using greedy inference for resource-efficient matching of entities by one or more ML models. In some implementations, actions include determining a set of potential probability thresholds based on a set of inference results provided by processing testing data through the ML model, for each potential probability threshold in the set of potential probability thresholds, determining an accuracy, selecting a probability threshold from the set of potential probability thresholds, processing an inference job including sets of entity pairs through the ML model to assign a label to each entity pair in the sets of entity pairs, each label being associated with a probability and including a type of multiple types, and for each entity pair having a label of one or more specified types, selectively removing an entity of the entity pair from further processing of the inference job by the ML model based on whether the probability associated with the label meets or exceeds the probability threshold. Other implementations of this aspect include corresponding systems, apparatus, and computer programs, configured to perform the actions of the methods, encoded on computer storage devices. These and other implementations can each optionally include one or more of the following features: the probability threshold is selected as a lowest potential probability threshold in the set of potential probability thresholds having an accuracy that meets or exceeds a target accuracy; selectively removing an entity of the entity pair from further processing of the inference job by the ML model includes adding a key of the entity to a set of matched keys in response to determining that the probability associated with the label, wherein the set of matched keys is used to selectively filter entities from being processed in the inference job; the one or more specified types include one or more of a single match and a multi-match; the set of potential probability thresholds includes unique probabilities included in the inference results; actions further include determining a set of keys for a set of entities, each key in the set of keys uniquely identifying an entity, comparing keys in the set of keys to matched keys in a set of matched keys, and removing an entity from the set of entities in response to determining that a key identifying the entity is included in the set of matched keys; and each entity pair includes a query entity and a target entity, the target entity being selectively removed based on whether the probability associated with the label meets or exceeds the probability threshold. The present disclosure also provides a computer-readable storage medium coupled to one or more processors and having instructions