Search

EP-4377906-B1 - OPTICAL FRAUD DETECTOR FOR AUTOMATED DETECTION OF FRAUD IN DIGITAL IMAGINARY-BASED AUTOMOBILE CLAIMS, AUTOMATED DAMAGE RECOGNITION, AND METHOD THEREOF

EP4377906B1EP 4377906 B1EP4377906 B1EP 4377906B1EP-4377906-B1

Inventors

  • TISSEUR, Riccardo
  • ARORA, AMIT
  • GUPTA, ABHINAV

Dates

Publication Date
20260506
Application Date
20220729

Claims (20)

  1. Automated optical fraud detector (100) for automated detection of damage of at least one damaged object (1083) of a motor vehicle (108) based on digital image data (1042) and for automated detection of fraudulent manipulations of the digital image data (1042), wherein the automated optical fraud detector (100) comprises an electronic claim portal (1041) for capturing one or more digital images (1042), wherein the one or more digital images (1042) at least comprise digital images (1042) associated with one or more damaged objects (1083) of the motor vehicle (108), and wherein the automated optical fraud detector (100) comprises an incident damage zone and location unit (130), wherein the received digital images (1042) are processed by the incident damage zone and location unit (130) determining at least damage location zones (1301, wherein the automated optical fraud detector (100) comprises a machine-learning based module for processing the one or more images by using an Artificial Intelligence (Al) structure, wherein by means of the machine-learning based module a first set of attributes indicative of damage to one or more zones of the automobile is determined, the first set of attributes comprising details damage to at least one zone of the one or more zones including front center zone, front left zone, front right zone, left zone, right zone, rear left zone, rear right zone, rear center zone, and windshield zone; and wherein a second attribute is determined, the second attribute comprising details the damage to the roof of the automobile, the automated optical fraud detector (100) comprises an unusua damage pattern detector (134) for processing said one or more digital images (1042) for existing image alteration by using an unusual pattern identification structure (1341) and providing a first fraud detection (1342), wherein the unusual damage pattern detector (134) detects unusual damage pattern of the motor vehicle (108) indicating unusual damage pattern to the automobile not happening in at least an accident or natural calamities based on determining by the unusual damage pattern detector (134) whether damages are to one or more zones on opposite sides of the motor vehicle (108) and damage to the roof of the motor vehicle (108), and wherein a rule-based fraud identifier (13421) is set, if an unusual damage pattern is detected by the unusual damage pattern detector (134), the automated optical fraud detector (100) comprises a machine-learning based fraud detection unit (136), wherein by means of a RGB recognition module (1361) said one or more digital images (1042) are processed for fraud detection using RGB image input, and wherein the RGB values (1362) are used for (i) CNN-based pre-existing damage detection, (ii) parallel CNN-based color matching, and (iii) double JPEG compression (DJCD) detection using custom CNN, providing output of the CNN-based detection, the automated optical fraud detector (100) comprises a seconc fraud detection unit (1363) having as input the output of the of the CNN-based damage detection triggering a machine-learning-based fraud identifier (13631) based on the output of the CNN-based damage detection, wherein the machine-learning-based fraud identifier (13631) is set, if a fraud is indicated by the second fraud detection unit (1363), and wherein the automated optical fraud detector (100) comprises a signa assembling unit (140) for generating a fraud signaling output (1401) based upon detecting the rule-based fraud identifier (13421) and the machine-learning-based fraud identifier (13631).
  2. Automated optical fraud detector (100) according to claim 1, characterized in that the automated optical fraud detector (100) comprises an automated feed-back loop (1402) for verifying the generated fraud signaling output (13631) by a human expert, wherein a verified signaling output (1403) is generated fed back to the machine-learning-based fraud detection unit (136) updating machine learning parameters of the CNN-based pre-existing damage detection and/or of the parallel CNN-based color matching and/or of the double JPEG compression detection using custom CNN.
  3. An automated optical fraud detection method for automated detection of damage of at least one damaged object (1083) of a motor vehicle (108) based on digital image data (1042) and for automated detection of fraudulent manipulations of the digital image data (1042), wherein an automated optical fraud detector (100) comprises an electronic claim portal (1041) for capturing one or more digital images (1042), wherein the one or more digital images (1042) at least comprise digital images (1042) associated with one or more damaged objects (1083) of the motor vehicle (108), and wherein the automated optical fraud detector (100) comprises an incident damage zone and location unit (130), wherein the received digital images (1042/1043/1044) are processed by the incident damage zone and location unit (130) determining at least damage location zones (1301), performing the steps of: processing by a machine-learning based module the one or more images by using an Artificial Intelligence (AI) structure, wherein by means of the machine-learning based module a first set of attributes indicative of damage to one or more zones of the automobile is determined, the first set of attributes comprising details damage to at least one zone of the one or more zones including front center zone, front left zone, front right zone, left zone, right zone, rear left zone, rear right zone, rear center zone, and windshield zone; and wherein a second attribute is determined, the second attribute comprising details the damage to the roof of the automobile, processing by an unusual damage pattern detector (134) of the automated optical fraud detector (100) said one or more digital images (1042) for existing image alteration by using an unusual pattern identification structure (1341) and providing a first fraud detection (1342), wherein the unusual damage pattern detector (134) detects unusual damage pattern of the motor vehicle (108) indicating unusual damage pattern to the automobile not happening in at least an accident or natural calamities based on determining by the unusual damage pattern detector (134) whether damages are to one or more zones on opposite sides of the motor vehicle (108) and damage to the roof of the motor vehicle (108), and wherein a rule-based fraud identifier (13421) is set, if an unusual damage pattern is detected by the unusual damage pattern detector (134), processing by a machine-learning based fraud detection unit (136) of the automated optical fraud detector (100) by means of a RGB recognition module (1361) said one or more digital images (1042) for fraud detection using RGB image input, and wherein the RGB values (1362) are used for (i) CNN-based pre-existing damage detection, (ii) parallel CNN-based color matching, and (iii) double JPEG compression (DJCD) detection using custom CNN, providing output of the CNN-based detection, triggering by a second fraud detection unit (1363) of the automated optical fraud detector (100) a machine-learning-based fraud identifier (13631) having as input the output of the of the CNN-based damage detection based on the output of the CNN-based damage detection, wherein the machine-learning-based fraud identifier (13631) is set, if a fraud is indicated by the second fraud detection unit (1363), and generating by a signal assembling unit (140) of the automated optical fraud detector (100) a fraud signaling output (1401) based upon detecting the rule-based fraud identifier (13421) and the machine-learning-based fraud identifier (13631).
  4. The automated optical fraud detection method according to claim 3, wherein the one or more servers are configured to process the text data using natural language processing techniques to determine a third set of attributes, the third set of attributes comprising at least one of car location, date of accident, time of accident, and damaged side of automobile and parts.
  5. The automated optical fraud detection method according to one of the claims 1 to 4, wherein the automated method comprises the step of processing mobile sensor data to obtain a floating car data (FCD) and/or processing automobile sensor data to obtain on-board units (OBUs) sensors data.
  6. The automated optical fraud detection method according to one of the claims 1 to 5, wherein the automated method comprises the steps of processing the FCD to determine a fourth set of attributes, the fourth set of attributes comprising at least one of a passenger's route, trip travel time, estimate traffic state, and global positioning system (GPS) data, and processing the set of attributes, the third attribute, and the fourth set of attributes to obtain a sixth set of attributes, the sixth set of attributes comprising at least one of information of damage to one or more zones of the automobile and FCD attributes, wherein the FCD attributes comprises timestamped geo-localization and speed data.
  7. The automated optical fraud detection method according to one of the claims 1 to 6, wherein the automated method comprises the steps of processing OBUs sensors data to obtain a fifth set of attributes, the fifth set attributes comprising at least one of a camera data, a speed data, engine revolutions per minute (RPM) data, rate of fuel consumption, the GPS data, moving direction, impact sensor data, and airbag deployment data and processing the fifth set of attributes to obtain a seventh set of attributes that provide damage information associated with the automobile, and location information, the seventh set of attributes comprising at least one of information of damage to one or more zones of the automobile and the GPS data.
  8. The automated optical fraud detection method according to one of the claims 1 to 7, wherein the automated method comprises the steps of performing a first analysis on the first set of attributes, the sixth set of attributes, and the seventh set of attributes to determine whether there are damages to one or more zones on opposite sides of the automobile when the third attribute indicates a damage to the automobile roof, and performing a second analysis on the first set of attributes, the sixth set of attributes, and the seventh set of attributes to determine whether there are damages to one or more zones on opposite sides of the automobile when the third attribute indicates no damage to the automobile roof, and determining the unusual damage pattern based on the first analysis and the second analysis.
  9. The automated optical fraud detection method according to one of the claims 1 to 8, wherein the automated method comprises the steps of processing the one or more images by using a convolutional neural network (CNN) structure to (i) process the one or more images to determine a pre-existing damage, perform color matching and a double joint photographic experts group (JPEG) compression, and (ii) determine the indication of the fraud based on determining at least one of pre-existing damage, color matching and double JPEG compression.
  10. The automated optical fraud detection method according to one of the claims 1 to 9, wherein the automated method comprises the steps of providing a machine learning (ML) parameter updates to the CNN based on human verification.
  11. Automated optical fraud detector (100) according to one of the claims 1 or 2, characterized in that the incident damage zone and location unit (130) comprises, for processing the received data (1042/1043/1044) and predicting and/or determining at least damage location zones (1301) and/or incident geographic location (1302) and/or incident date (1303) and/or incident time (1304), a recognition apparatus for automated damage identification for vehicle or property damages based on image processing, comprising a damage data upload section, at least one data storage unit, an image processing section, a damage identification section and a damage information output section, wherein the upload section is designed to receive one or more images captured of a damage at a vehicle or property in form of digital image data that in an uploading step (1) is uploaded into the at least one data storage unit, wherein the image processing section is designed to process the image data in an image processing step by independently applying at least two different visual modeling data processing structures (3; 4) to the image data to independently identify damaged parts of the vehicle or property and/or damage types at the vehicle or property and wherein each visual model provides an independent subset of damage data (3.5; 4.5) corresponding to the identified damaged parts and/or damage types, wherein the damage identification section is designed for automatically combining the subsets of damage data (3.5; 4.5) in a combining step (5) to define a single domain of damage data (6) that provides enhanced inference accuracy for identifying damaged parts of the vehicle or property and/or damage types, and wherein the damage identification output section is designed to provide damage information based on the single domain of damage data (6).
  12. The automated optical fraud detector (100) according to claim 11, characterized in that in the combining step (5) the individual sub sets of damage data (3.5; 4.5) are checked for data deficiencies regarding the damaged parts of the vehicle or property, and such data deficiencies in one sub set of damage data (3.5) are compensated by damage data of another sub set of damage data (4.5) to provide enhanced inference accuracy of the single domain of damage data (6).
  13. The automated optical fraud detector (100) according to one of the claims 11 or 12, characterized in that a data storage unit provides a master list of damage nomenclature and the single domain of damage data (6) representing the identified damaged parts and/or damage types is compared to the master list of damage nomenclature to associate the identified damage to corresponding damage nomenclature.
  14. The automated optical fraud detector (100) according to one of the claims 11 to 13, characterized in that in the image processing step the image data is further processed by a gradient boosted decision tree model (7).
  15. The automated optical fraud detector (100) according to one of the claims 11 to 14, characterized in that the damage information is subject to human expert validation and the combining step (5) is augmented by a validation factor corresponding to the expert validation.
  16. An automated optical fraud detection method for detecting a fraud in one or more digital images associated with an automobile claim and a related risk-transfer by means of an automated system according to one of the claims 3 to 10, the method comprising as first steps the following steps of automated damage identification for vehicle or property damages based on image processing, wherein one or more images captured of a damage at a vehicle or property provide image data that that in an uploading step (1) is uploaded into at least one data storage unit, in an image processing step the image data is processed by independently applying at least two different visual models (3; 4) to the image data to independently identify damaged parts of the vehicle or property and/or damage types at the vehicle or property and wherein each visual model (3; 4) provides an independent subset of damage data (3.5; 4.5) corresponding to the identified damaged parts and/or damage types, in a combining step (5) the subsets of damage data (3.5; 4.5) are automatically combined to define a single domain of damage data (6) that provides enhanced inference accuracy for identifying damaged parts of the vehicle or property and/or damage types, and damage information based on the single domain of damage data (6) is provided that indicates damaged parts and/or damage types.
  17. The automated optical fraud detection method according to claim 16, wherein in the image processing step the image data is prepared for the processing by the at least two different visual models (3; 4) in that the image data is provided with a case identifier, a damage part number and/or an image identifier.
  18. The automated optical fraud detection method according to claim 16 or 17, wherein the image processing step of at least one of the visual models (3; 4) provides steps of damage part identification, damage class identification and/or assigning a damage confidence metric to the damage data.
  19. The automated optical fraud detection method according to any of the preceding claims 16 to 17, wherein in the image processing step of at least one of the visual models (3; 4) provides steps of data cleaning and/or data correction including correction of the damage part and/or damage classification.
  20. The automated optical fraud detection method according to any of the preceding claims 16 to 19, wherein in the image processing step of at least one of the visual models (3; 4) provides steps of associating the damage data with a predefined damage nomenclature and/or predefined damage classification, wherein the predefined damage nomenclature and/or classification is selected from a master list of damage nomenclature.

Description

Field of the Invention The present invention relates to devices, systems, and methods for automated detecting and/or assessing damage to an object such as, for example, a vehicle, wherein possible fraudulent manipulations of digital image and/or fraudulent claims are automatically detected. Thus, it relates to an optical fraud detector and detection device for automated detection of fraud in digital imaginary-based automobile claim processing and automated damage recognition systems. In addition, it relates to devices, systems, and methods for detecting / analyzing / assessing damage to an object such as, for example, a vehicle providing estimates on repair / replacement costs, as well as, in addition to the evaluation on potential image manipulation and fraud. In particular, the present invention relates to a fully automated method for detecting and/or assessing damage to an object from image data provided by a user. Further, the present invention relates to a recognition apparatus and a recognition method based on image processing for automated damage identification for vehicle or property damages. Furthermore, the present invention also generally relates to image based damage recognition for processing damage claims by an insurance system. Background of the Invention (A) Automated Automobile Claims Fraud Detector When insured property is damaged, the owner may file a claim with the risk-transfer system or insurance company concerned with the risk-transfer. However, conventional processing of insurance claims is a complex process including, inter alia, the involvement of experts such as accident assessors in order to inspect, analyze and assess any damage to the insured object and provide the amount of damage, as well as costs required to repair or replace the damaged object. Thus, there is a heavy reliance on manual inspections by an expert to provide a repair cost estimate, which may come with significant cost and delay in processing time, as a person (assessor) must view the asset in order to assess the damage and decide upon an outcome, e.g. view a vehicle, and decide if the vehicle is repairable or not. Further, an insured may want to know the full extent of the damage before involving the insurance or assessor in order to decide, whether it is worth submitting an insurance claim or more cost effective to simply pay the cost of repair themselves. For instance, is the damage panel repairable or does it need a replacement. There has been some advancement across the industry over the last years in the use of images to assist with assessing vehicles or other property without the need of a physical inspection. However, these advancements still rely on the technical expertise required to first capture suitable images (e.g. required technical standard format) and then incorporate the images with additional data from third parties, to allow a trained assessor or engineer to manually inspect the images and generate, for example, a repair estimate report. This is a costly, time consuming process particularly when there are finite technical resources. In the prior art, there are also systems allowing a consumer to capture the images in accordance with given instructions and process the initial claim by providing detailed information of the damage (e.g. following a protocol of questions to determine the location, type, and description of the damage), making the process very time consuming and very subjective to the consumer's incentive. First is to be noted, that fighting against insurance fraud is a challenging problem both technically and operationally. It is reported that approximately 21%-36% auto-insurance claims contain elements of suspected fraud but only less than 3% of the suspected fraud is prosecuted. Traditionally, insurance fraud detection relies heavily on auditing and expert inspection. Since manually detecting fraud cases is costly and inefficient and fraud need to be detected prior to the claim payment, data mining analytics is increasingly recognized as a key in fighting against fraud. This is due to the fact that data mining and machine learning techniques have the potential to detect suspicious cases in a timely manner, and therefore potentially significantly reduce economic losses, both to the insurers and policy holders. Indeed there is great demand for effective predictive methods which maximize the true positive detection rate, minimize the false positive rate, and are able to quickly identify new and emerging fraud schemes. In summary, conventional insurance claims processing is a complex process that typically starts with a first notification of loss related to an insured item. Upon notification of loss, the claim may be routed to multiple claims adjusters that analyze different aspects of the damage associated with the insured item in order to determine whether compensation for the loss is appropriate. In general, conventional claims adjustment can involve paperwork processing, telepho