US-20260127606-A1 - MACHINE LEARNING MODEL TRAINING USING CONTRASTIVE LANGUAGE ANOMALY PRETRAINING APPROACH

Abstract

Transaction data associated with a user is identified. A first embedding representing the transaction data is generated using a deep learning machine learning (ML) model. A second embedding interpretable by a large language model is generated using an adaptor ML model based on the first embedding. A text summary associated with one or more transactions is identified. A third embedding representing the text summary is generated using the large language model. An adaptor ML model is trained based on a similarity between the second embedding and the third embedding.

Inventors

• Zhichao Han
• Yang Zhao
• Weiming Liang
• Yinan Shan
• Zitao ZHANG
• Hang Yin
• Shan Jiang
• Alok Lal

Assignees

• EBAY INC.

Dates

Publication Date

20260507

Application Date

20241104

Claims (20)

1. 1 . A system comprising: one or more hardware processors; and at least one machine-storage medium for storing instructions that, when executed by the one or more hardware processors, cause the one or more hardware processors to perform operations comprising: identifying transaction data associated with a user; generating, using a deep learning machine learning (ML) model, a first embedding that represents the transaction data; generating, using an adaptor ML model, a second embedding that represents the transaction data based on the first embedding, the second embedding being interpretable by a large language model; identifying a text summary associated with one or more transactions; generating, using the large language model, a third embedding that represents the text summary; and training the adaptor ML model based on a similarity between the second embedding and the third embedding.
2. 2 . The system of claim 1 , wherein the deep learning ML model comprises a multi-modal encoder that transforms a plurality of modalities into a shared space for jointly analyzing and processing similarities and relationships between the plurality of modalities.
3. 3 . The system of claim 2 , wherein the plurality of modalities comprises one or more of tabular data, graph data, text, and images.
4. 4 . The system of claim 1 , wherein the adaptor ML model facilitates compatibility between upstream ML models and downstream ML models.
5. 5 . The system of claim 1 , the training of the adaptor ML model based on the similarity between the second embedding and the third embedding comprises: determining, using a cosine similarity formula, a value that represents a degree of similarity between the second embedding and the third embedding.
6. 6 . The system of claim 1 , wherein the user is a first user, and wherein the operations comprise: using a trained adaptor ML model to generate a fourth embedding representing transaction data of a second user; and generating a text summary of the transaction data based on the fourth embedding.
7. 7 . The system of claim 1 , wherein the text summary of the transaction data comprises at least one of an observation description, a transaction pattern, a fraud risk determination, a confidence score of the fraud risk determination, and a rationale description associated with the fraud risk determination, and wherein one or more of the observation description, the transaction pattern, the fraud risk determination, the confidence score of the fraud risk determination, and the rationale description associated with the fraud risk determination are generated via manual review.
8. 8 . The system of claim 1 , wherein the large language model corresponds to an open-source large language model.
9. 9 . The system of claim 1 , wherein the transaction data comprises a plurality of paired data between different types of modalities.
10. 10 . The system of claim 1 , wherein the text summary associated with one or more transactions corresponds to one or more suspended user accounts.
11. 11 . A method comprising: identifying transaction data associated with a user; generating, using a deep learning machine learning (ML) model, a first embedding that represents the transaction data; generating, using an adaptor ML model, a second embedding that represents the transaction data based on the first embedding, the second embedding being interpretable by a large language model; identifying a text summary associated with one or more transactions; generating, using the large language model, a third embedding that represents the text summary; and training the adaptor ML model based on a similarity between the second embedding and the third embedding.
12. 12 . The method of claim 11 , wherein the deep learning ML model comprises a multi-modal encoder that transforms a plurality of modalities into a shared space for jointly analyzing and processing similarities and relationships between the plurality of modalities.
13. 13 . The method of claim 12 , wherein the plurality of modalities comprises one or more of tabular data, graph data, text, and images.
14. 14 . The method of claim 11 , wherein the adaptor ML model facilitates compatibility between upstream ML models and downstream ML models.
15. 15 . The method of claim 11 , the training of the adaptor ML model based on the similarity between the second embedding and the third embedding comprises: determining, using a cosine similarity formula, a value that represents a degree of similarity between the second embedding and the third embedding.
16. 16 . The method of claim 11 , wherein the user is a first user, comprising: using a trained adaptor ML model to generate a fourth embedding representing transaction data of a second user; and generating a text summary of the transaction data based on the fourth embedding.
17. 17 . The method of claim 11 , wherein the text summary of the transaction data comprises at least one of an observation description, a transaction pattern, a fraud risk determination, a confidence score of the fraud risk determination, and a rationale description associated with the fraud risk determination, and wherein one or more of the observation description, the transaction pattern, the fraud risk determination, the confidence score of the fraud risk determination, and the rationale description associated with the fraud risk determination are generated via manual review.
18. 18 . The method of claim 11 , wherein the large language model corresponds to an open-source large language model.
19. 19 . The method of claim 11 , wherein the transaction data comprises a plurality of paired data between different types of modalities, and wherein the text summary associated with one or more transactions corresponds to one or more suspended user accounts.
20. 20 . A machine-storage medium for storing instructions that, when executed by one or more hardware processors, cause the one or more hardware processors to perform operations comprising: identifying transaction data associated with a user; generating, using a deep learning machine learning (ML) model, a first embedding that represents the transaction data; generating, using an adaptor ML model, a second embedding that represents the transaction data based on the first embedding, the second embedding being interpretable by a large language model; identifying a text summary associated with one or more transactions; generating, using the large language model, a third embedding that represents the text summary; and training the adaptor ML model based on a similarity between the second embedding and the third embedding.

Description

TECHNICAL FIELD The present disclosure generally relates to data processing using machine learning technologies. More particularly, various embodiments described herein provide for systems, methods, techniques, instruction sequences, and devices that facilitate machine learning model training using a contrastive language anomaly pretraining approach. BACKGROUND The field of anomaly detection in data science involves identifying unusual patterns in datasets that do not conform to expected behavior. The integration of machine learning with language processing technologies has expanded the scope of data analysis for anomaly detection, allowing for more complex interpretations of structured and unstructured data across a variety of modalities. As technology evolves, the machine learning used in anomaly detection continues to become more refined, leveraging advancements in computational power and algorithmic complexity to improve detection capabilities. BRIEF DESCRIPTION OF THE DRAWINGS In the drawings, which are not necessarily drawn to scale, like numerals may describe similar components in different views. To easily identify the discussion of any particular element or act, the most significant digit or digits in a reference number refer to the figure number in which that element is first introduced. Some embodiments are illustrated by way of examples, and not limitations, in the accompanying figures. FIG. 1 is a block diagram showing an example data system that includes a data management system, according to various embodiments of the present disclosure. FIG. 2 is a block diagram illustrating an example data management system that facilitates machine learning model training using a contrastive language anomaly pretraining approach, according to various embodiments of the present disclosure. FIG. 3 is a flowchart illustrating an example method for facilitating machine learning model training using a contrastive language anomaly pretraining approach, according to various embodiments of the present disclosure. FIG. 4 is a flowchart illustrating an example method for facilitating machine learning model training using a contrastive language anomaly pretraining approach, according to various embodiments of the present disclosure. FIG. 5 is a diagram illustrating data flow within an example data management system that facilitates machine learning model training using a contrastive language anomaly pretraining approach, according to various embodiments of the present disclosure. FIG. 6 is a block diagram illustrating a representative software architecture, which may be used in conjunction with various hardware architectures herein described, according to various embodiments of the present disclosure. FIG. 7 is a block diagram illustrating components of a machine able to read instructions from a machine storage medium and perform any one or more of the methodologies discussed herein according to various embodiments of the present disclosure. DETAILED DESCRIPTION The description that follows includes systems, methods, techniques, instruction sequences, and computing machine program products that embody illustrative embodiments of the present disclosure. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of embodiments. It will be evident, however, to one skilled in the art that the present inventive subject matter may be practiced without these specific details. Reference in the specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present subject matter. Thus, the appearances of the phrase “in one embodiment” or “in an embodiment” appearing in various places throughout the specification are not necessarily all referring to the same embodiment. For purposes of explanation, specific configurations and details are set forth in order to provide a thorough understanding of the present subject matter. However, it will be apparent to one of ordinary skill in the art that embodiments of the subject matter described may be practiced without the specific details presented herein, or in various combinations, as described herein. Furthermore, well-known features may be omitted or simplified in order not to obscure the described embodiments. Various embodiments may be given throughout this description. These are merely descriptions of specific embodiments. The scope or meaning of the claims is not limited to the embodiments given. Various embodiments include systems, methods, and non-transitory computer-readable media that facilitate machine learning model training using a contrastive language anomaly pretraining approach, according to various embodiments of the present disclosure. Specifically, various embodiments relate to a Contrastive Language Anomaly Pretraining (CLAP) approach that enh