US-12625908-B1 - Categorization with graph neural network and language model

Abstract

Certain aspects of the disclosure provide techniques for categorization by a device. An example method includes receiving input information regarding a plurality of classification targets and a plurality of categories for classification of the plurality of classification targets; generating a plurality of embeddings for the input information using a first model, the plurality of embeddings including: a set of first embeddings associated with the plurality of classification targets, and a set of second embeddings associated with the plurality of categories; determining that a similarity score for the set of first embeddings and the set of second embeddings fails to satisfy a threshold; generating, based on the similarity score failing to satisfy the threshold and using a graph neural network (GNN), a classification of the plurality of classification targets in accordance with the plurality of categories; and outputting information regarding the classification.

Inventors

• Kaiwen DONG
• Xiang Gao
• Maria Kissa
• Ayan Acharya
• Hilaf Hasson
• Mauricio Flores
• Heather Simpson
• Byron Tang
• Kamalika Das

Assignees

• INTUIT INC.

Dates

Publication Date

20260512

Application Date

20250730

Priority Date

20241224

Claims (20)

1. 1 . A method of categorization by a processing system, comprising: receiving input information regarding a plurality of classification targets and a plurality of categories for classification of the plurality of classification targets; generating a plurality of embeddings for the input information using a first model, the plurality of embeddings including: a set of first embeddings associated with the plurality of classification targets, and a set of second embeddings associated with the plurality of categories; determining that a similarity score for the set of first embeddings and the set of second embeddings fails to satisfy a threshold; generating, based on the similarity score failing to satisfy the threshold and using a graph neural network (GNN), a classification of the plurality of classification targets in accordance with the plurality of categories; and outputting information regarding the classification.
2. 2 . The method of claim 1 , wherein the input information further includes data associated with the plurality of classification targets, the data comprising at least one of a transaction direction, a transaction amount, or a memo.
3. 3 . The method of claim 1 , wherein the GNN is configured to use a heterogeneous graph structure that includes a plurality of nodes representing users, transactions, and categories, and a plurality of edges representing relationships between the plurality of nodes.
4. 4 . The method of claim 3 , wherein each node of the plurality of nodes is one of a plurality of node types, wherein each node type of the plurality of node types is associated with a respective data structure of a plurality of data structures of a relational database.
5. 5 . The method of claim 4 , wherein a first data structure of the plurality of data structures is associated with a first parameter of the input information and a second data structure of the plurality of data structures is associated with a second parameter of the input information.
6. 6 . The method of claim 3 , further comprising generating the heterogeneous graph structure.
7. 7 . The method of claim 6 , wherein the heterogeneous graph structure is based on relationships of a relational database.
8. 8 . The method of claim 3 , wherein generating the classification further comprises performing a link prediction using the heterogeneous graph structure.
9. 9 . The method of claim 1 , wherein generating the classification is based on the similarity score.
10. 10 . The method of claim 1 , wherein the GNN is based on a weighted negative sampling technique.
11. 11 . The method of claim 1 , wherein the set of first embeddings is based on a tokenizer that is specialized for transaction data.
12. 12 . A method of categorization by a processing system, comprising: receiving input information regarding a plurality of classification targets and a plurality of categories for classification of the plurality of classification targets; training a first model to generate a plurality of embeddings for the input information, the plurality of embeddings including: a set of first embeddings associated with the plurality of classification targets, and a set of second embeddings associated with the plurality of categories; generating a heterogeneous graph structure for a graph neural network (GNN), the heterogeneous graph structure including: a plurality of nodes representing users, transactions, and categories of the input information, and a plurality of edges representing relationships between the plurality of nodes; and providing the first model and the GNN.
13. 13 . The method of claim 12 , wherein each node of the plurality of nodes is one of a plurality of node types, wherein each node type of the plurality of node types is associated with a respective data structure of a plurality of data structures of a relational database.
14. 14 . The method of claim 13 , wherein a first data structure of the plurality of data structures is associated with a first parameter of the input information and a second data structure of the plurality of data structures is associated with a second parameter of the input information.
15. 15 . The method of claim 12 , wherein the GNN uses a message-passing mechanism based on relationships being treated as edges between nodes representing transactions.
16. 16 . The method of claim 12 , wherein the first model is a transformer-based language model.
17. 17 . The method of claim 12 , wherein training the first model comprises training the first model based on a tokenizer that is specialized for transaction data.
18. 18 . An apparatus, comprising a processing system that includes one or more processors and one or more memories, the processing system configured to cause the apparatus to: receive input information regarding a plurality of classification targets and a plurality of categories for classification of the plurality of classification targets; generate a plurality of embeddings for the input information using a first model, the plurality of embeddings including: a set of first embeddings associated with the plurality of classification targets, and a set of second embeddings associated with the plurality of categories; generate, based on a similarity score for the set of first embeddings and the set of second embeddings failing to satisfy a threshold and using a graph neural network (GNN), a classification of the plurality of classification targets in accordance with the plurality of categories; and output information regarding the classification.
19. 19 . The apparatus of claim 18 , wherein the information regarding the classification indicates a suggested category, not included in the plurality of categories, for a classification target of the plurality of classification targets.
20. 20 . The apparatus of claim 19 , wherein the suggested category is based on historical information of the input information.

Description

CROSS-REFERENCE TO RELATED APPLICATION(S) This application claims benefit of and priority to European Patent Application No. 24386153.1, filed Dec. 24, 2024, which is herein incorporated by reference in its entirety for all applicable purposes. BACKGROUND Field Aspects of the present disclosure relate to intelligent systems and machine learning platforms, particularly to methods and systems for categorizing classification targets with a graph neural network and a language model. Description of Related Art It is beneficial in various arts to classify a set of objects (such as descriptions of transactions, text strings, or other objects), such as by assigning each object of the set of objects to a category. There are various methods for classification, ranging from simple text matching to complex machine learning based approaches. Different methods have tradeoffs with regard to computational complexity, accuracy, and versatility. SUMMARY Certain aspects provide a method of categorization by a processing system. The method includes receiving input information regarding a plurality of classification targets and a plurality of categories for classification of the plurality of classification targets; generating a plurality of embeddings for the input information using a first model, the plurality of embeddings including: a set of first embeddings associated with the plurality of classification targets, and a set of second embeddings associated with the plurality of categories; determining that a similarity score for the set of first embeddings and the set of second embeddings fails to satisfy a threshold; generating, based on the similarity score failing to satisfy the threshold and using a graph neural network (GNN), a classification of the plurality of classification targets in accordance with the plurality of categories; and outputting information regarding the classification. Certain aspects provide a method of categorization by a processing system. The method includes receiving input information regarding a plurality of classification targets and a plurality of categories for classification of the plurality of classification targets; training a first model to generate a plurality of embeddings for the input information, the plurality of embeddings including: a set of first embeddings associated with the plurality of classification targets, and a set of second embeddings associated with the plurality of categories; generating a heterogeneous graph structure for a GNN, the heterogeneous graph structure including: a plurality of nodes representing users, transactions, and categories of the input information, and a plurality of edges representing relationships between the plurality of nodes; and providing the first model and the GNN. Certain aspects provide a method by a processing system. The method includes receiving input information regarding a plurality of classification targets and a plurality of categories for classification of the plurality of classification targets; generating a plurality of embeddings for the input information using a first model, the plurality of embeddings including: a set of first embeddings associated with the plurality of classification targets, and a set of second embeddings associated with the plurality of categories; generating, based on a similarity score for the set of first embeddings and the set of second embeddings failing to satisfy a threshold and using a GNN, a classification of the plurality of classification targets in accordance with the plurality of categories; and outputting information regarding the classification. Other aspects provide processing systems configured to perform the aforementioned methods as well as those described herein; non-transitory, computer-readable media comprising instructions that, when executed by a processors of a processing system, cause the processing system to perform the aforementioned methods as well as those described herein; a computer program product embodied on a computer readable storage medium comprising code for performing the aforementioned methods as well as those further described herein; and a processing system comprising means for performing the aforementioned methods as well as those further described herein. The following description and the related drawings set forth in detail certain illustrative features of one or more aspects. DESCRIPTION OF THE DRAWINGS The appended figures depict certain aspects and are therefore not to be considered limiting of the scope of this disclosure. FIG. 1 depicts an example system supporting microservices. FIG. 2 is a diagram illustrating an example of categorization of classification targets using a graph neural network and a language model. FIG. 3 is a diagram illustrating an example of training and implementing a tokenizer and a transformer for embedding of input information comprising classification targets. FIG. 4 illustrates an example relational database of input information and