US-12620250-B2 - Generalized anomaly detection

US12620250B2US 12620250 B2US12620250 B2US 12620250B2US-12620250-B2

Abstract

Described are methods and systems for training a system for detecting anomalies in images of documents in a class of documents. A plurality of training document images of training documents in a class of documents are obtained. For each training document image, the training document image is segmented into a plurality of region of interest (ROI) images, each ROI image corresponding to a respective ROI of the training document. For each ROI image, a plurality of transformations are applied to the ROI image to generate respective transform-specific features for the ROI image and respective transform-specific anomaly scores from the transform-specific features. Based on the respective anomaly scores of the plurality of training document images, a transform-specific threshold is computed for each transformation to separate document images containing an anomaly from document images not containing an anomaly.

Inventors

Olivier Koch
Philip Botros
Christos SAGONAS
Francesco Picciotti

Assignees

ONFIDO LTD.

Dates

Publication Date: 20260505
Application Date: 20221220
Priority Date: 20210614

Claims (16)

1 . A computer-implemented method of sourcing training images for a system for detecting anomalies in images of documents in a class of documents, the method comprising: (a) obtaining one or more document images of a fraudulent document; (b) for each document image: (i) segmenting the document image into one or more region of interest (ROI) images, each ROI image corresponding to a respective ROI of the fraudulent document; (ii) generating a vector representation of each ROI image in an embedding space, the embedding space comprising vector representations of one or more other ROI images from a plurality of sample document images; (iii) determining a similarity between each ROI image from the fraudulent document and one or more other ROI images from sample document images using the vector representations in the embedding space; and (iv) selecting a plurality of sample document images that have ROI images similar to the ROI images from the fraudulent document and generating a training data set using the selected sample document images; and (c) in response to a determination that a size of the training data set exceeds a predetermined size threshold, training a supervised anomaly detection machine learning model using the training data set.
2 . The method of claim 1 , wherein the vector representation of each ROI image is generated using a vision transformer (ViT) model.
3 . The method of claim 1 , wherein the similarity is determined using a distance metric.
4 . The method of claim 1 , wherein the one or more document images of the fraudulent document are obtained from an external application programming interface (API) or a user interface of a production anomaly detection system.
5 . The method of claim 1 , wherein at least one of the one or more ROI images comprises an image of the entire fraudulent document.
6 . The method of claim 1 , wherein at least one of the one or more ROI images comprises an image of a portion of the fraudulent document.
7 . The method of claim 1 , wherein generating a vector representation of each ROI image in an embedding space comprises projecting the vector representation into the embedding space.
8 . The method of claim 1 , wherein training a supervised anomaly detection machine learning model using the training data set comprises: training a plurality of supervised anomaly detection machine learning models using the training data set, each supervised anomaly detection machine learning model corresponding to a respective ROI of the fraudulent document.
9 . A system for sourcing training images for a system for detecting anomalies in images of documents in a class of documents, the system comprising a computing device having a memory for storing computer-executable instructions and a processor that executes the computer-executable instructions to: (a) obtain one or more document images of a fraudulent document; (b) for each document image: (i) segment the document image into one or more region of interest (ROI) images, each ROI image corresponding to a respective ROI of the fraudulent document; (ii) generate a vector representation of each ROI image in an embedding space, the embedding space comprising vector representations of one or more other ROI images from a plurality of sample document images; (iii) determine a similarity between each ROI image from the fraudulent document and one or more other ROI images from sample document images using the vector representations in the embedding space; and (iv) select a plurality of sample document images that have ROI images similar to the ROI images from the fraudulent document and generating a training data set using the selected sample document images; (c) determine whether a size of the training data set exceeds a predetermined size threshold; and (d) in response to a determination that the size of the training data set exceeds the predetermined size threshold, train a supervised anomaly detection machine learning model using the training data set; or (e) in response to a determination that the size of the training data set does not exceed the predetermined size threshold, train an unsupervised anomaly detection machine learning model using the training data set.
10 . The system of claim 9 , wherein the vector representation of each ROI image is generated using a vision transformer (ViT) model.
11 . The system of claim 9 , wherein the similarity is determined using a distance metric.
12 . The system of claim 9 , wherein the one or more document images of the fraudulent document are obtained from an external application programming interface (API) or a user interface of a production anomaly detection system.
13 . The system of claim 9 , wherein at least one of the one or more ROI images comprises an image of the entire fraudulent document.
14 . The system of claim 9 , wherein at least one of the one or more ROI images comprises an image of a portion of the fraudulent document.
15 . The system of claim 9 , wherein generating a vector representation of each ROI image in an embedding space comprises projecting the vector representation into the embedding space.
16 . The system of claim 9 , wherein training a supervised anomaly detection machine learning model using the training data set comprises: training a plurality of supervised anomaly detection machine learning models using the training data set, each supervised anomaly detection machine learning model corresponding to a respective ROI of the fraudulent document.

Description

RELATED APPLICATIONS This application claims priority to European Patent Application No. EP22204513.0, filed on Oct. 28, 2022. This application is also a continuation-in-part of U.S. patent application Ser. No. 17/830,208, filed on Jun. 1, 2022, now U.S. Pat. No. 12,423,788; which claims priority to European Patent Application No. EP21179304.7, filed on Jun. 14, 2021. Each of these applications is incorporated herein by reference in its entirety. TECHNICAL FIELD The subject matter of the application relates generally to a method for detecting anomalies in images of a document in a class of documents. In particular, the disclosure relates to applying a plurality of transformations to images of regions of interest in the document to detect anomalies in images of a document. BACKGROUND Digital identity verification systems require users to submit images of government issued documents remotely using mobile phones, computers or any other digital device. This information is then used to grant users access to services or products of different kinds such as online banking, brokering, conveyancing, property letting or car rentals. The remote nature of digital onboarding combined with the potential high value of these services leaves such verification systems open to fraudsters from around the world who may attempt to defraud the systems. For example, a fraudster may impersonate another person by tampering with identity documents, resulting in documents containing anomalies. There are, of course, other ways of digitally defrauding a system such as submitting false biometric data by impersonating someone else or intercepting results from a service provider to its clients and unauthorized tampering. However, the focus of the present application is on solving the problem of detecting anomalies in documents, wherein the anomalies may be indicative of fraudulent tampering with the documents. There are several major challenges that arise when attempting to detect anomalies in documents. First, there is a high variability across different documents types and across different countries. Moreover, as technology evolves and new fraud attacks are revealed, additional security features or design changes are permanently added and new versions of certain document types are created as a result. Thus, a need for a modular anomaly detection model that can be changed on a continuous basis arises. Second, images of such documents contain high levels of intrinsic noise (e.g., certain security features such as holograms appear differently from one document to another) and extrinsic noise (e.g., sensor noise, differing light levels, blur etc.), as well as a convolution of both. Third, there is a lack of genuine reference documents for certain countries, which fraudsters might therefore target, creating a need for easily scalable approaches. Fourth, comprehensive fraudulent data are frequently lacking or extremely limited, especially as new, specific fraud types are continuously being developed, thus creating a need for an anomaly detection model that can be trained without necessarily using fraudulent data for training. Because models are trained across thousands of different document classes, when a new type of fraud is incorporated, there is a need for a global model update, which can alter performance and can delay a prompt response. Additionally, general fraud detection models do not provide explanations about the decision to mark documents as fraudulent or not, therefore making human-driven verification of the anomalies flagged by the system inefficient and ineffective. Finally, as fraudsters develop new fraud attacks, the performance of fraud-specific anomaly detection models (e.g. picture swap, font anomaly) is severely affected as any unknown type of fraud can easily pass undetected by the model. There is therefore a need for a new automated approach that can rapidly deliver modular, adaptable, scalable, interpretable and customized solutions. SUMMARY The invention, in one aspect, features a computer implemented method of training a system for detecting anomalies in images of documents in a class of documents—for example, British passports, American driver's licenses, French identity cards, etc.—is disclosed. The first stage of the method involves obtaining plurality of training document images of a training documents in the class of documents (e.g., driver's licenses, passports, ID cards). During the next stage of the method, each training document image is segmented into a plurality of region of interest (ROI) images, each ROI image corresponding to a respective ROI of the training document. A region of interest of an image represents a region deemed to be rich in information about the image and can be a portion of or the whole image. For example, an ROI of a document can be a portion of a document that is rich in information about the document, unique and hard to reproduce. In practice, ROI images of a document could comprise i