Search

CN-122022990-A - Distributed batch deduction method, system, program product and device

CN122022990ACN 122022990 ACN122022990 ACN 122022990ACN-122022990-A

Abstract

The invention provides a distributed batch deduction method, a distributed batch deduction system, a program product and a distributed batch deduction device. The method comprises the steps of S100, conducting validity pre-checking on data to be processed when a daily batch deduction task is triggered, S200, terminating the batch deduction process through pre-checking if no valid to-be-deducted records exist, otherwise entering into S300, extracting valid to-be-deducted records, predicting the execution cost of each to-be-deducted record based on multi-dimensional characteristics of each to-be-deducted record, distributing the to-be-deducted records to a plurality of fragments by adopting an equalizing algorithm to generate a load-equalizing fragment task list, S400, generating a globally unique idempotent key for each fragment task of the fragment task list, conducting idempotent verification before the execution of the globally unique idempotent key to ensure that the same fragment task is only executed once within a preset time window, S500, grading the failure records according to failure codes returned by deduction results, and executing differentiated processing according to different levels, wherein the differentiated processing comprises retry processing, final state failure processing or external post-loan system interface.

Inventors

  • LI ZHENZHEN

Assignees

  • 河北幸福消费金融股份有限公司

Dates

Publication Date
20260512
Application Date
20260227

Claims (10)

  1. 1. The distributed batch deduction method is characterized by comprising the following steps of: S100, when a daily batch buckling task is triggered, performing validity pre-detection on data to be processed; s200, through pre-checking, terminating the batch deduction flow if no effective record to be deducted exists, otherwise, entering step S300; s300, extracting effective to-be-deducted records, predicting the execution cost of each to-be-deducted record based on the multidimensional characteristics of the to-be-deducted records, distributing the to-be-deducted records to a plurality of fragments by adopting an equalization algorithm, and generating a load-balanced fragment task list; S400, generating globally unique idempotent keys for each slicing task of the slicing task list, and performing idempotent verification before executing the globally unique idempotent keys to ensure that the same slicing task is executed only once in a preset time window; S500, when a failure code returned by the deduction result is received, grading the failure record according to the failure code, and executing differentiation processing according to different grades, wherein the differentiation processing comprises retry processing, final failure processing or calling an external post-credit system interface.
  2. 2. The method according to claim 1, wherein in step S100, the performing validity pre-check on the data to be processed includes: acquiring product information and time batch information of batch buckling to be executed on the same day; And generating a pre-check task record for each product matched with the current time, and executing a pre-check script to inquire the number of the valid records to be deducted.
  3. 3. The method of claim 1, wherein predicting an execution cost of each to-be-deducted record based on its multidimensional feature comprises: Extracting corresponding multidimensional features from each record to be deducted, wherein the multidimensional features comprise historical repayment behavior features, current risk portrait features and single money features of a user, and the single money features comprise loan amount and overdue amount; vectorizing the extracted multi-dimensional features and inputting vector data into a pre-trained machine learning model to predict the execution time consumption of each record as an execution cost.
  4. 4. The method of claim 1, wherein in step S300, distributing the record to be deducted to the plurality of segments using an equalization algorithm comprises: The records to be deducted are sorted in descending order according to the predicted execution cost, Based on the preset number of fragments, an improved boxing algorithm is adopted to distribute the records to be buckled to a plurality of fragment containers, so that the predicted total consumption time of each fragment tends to be balanced.
  5. 5. The method of claim 3, wherein the machine learning model comprises a XGBoost regression model.
  6. 6. The method of claim 1, wherein in step S400, the generating globally unique idempotent keys for each shard task comprises: splicing the content for each record to be buckled according to the format of 'batch buckling date: product number: customer number: batch buckling record ID'; Performing hash calculation on the spliced content to generate idempotent keys, and The idempotent key is written to the cache system and set with a validity period and a status code value, The state code value is updated according to a predefined state circulation rule, and the state circulation rule limits that states can only circulate from an initial state to a processing final state in sequence.
  7. 7. The method of claim 6, wherein in step S400, performing an idempotent check prior to task execution comprises: And checking the state code value of the idempotent key in the cache, allowing corresponding tasks to be executed only when the state code value is displayed as an initial state or is not present, and updating the state code value according to the state circulation rule according to the execution of the tasks.
  8. 8. A distributed batch payment system, comprising: The space-division pre-checking module is used for carrying out validity pre-checking on the data to be processed when the daily batch deduction task is triggered, terminating the batch deduction flow if no valid record to be deducted exists, and otherwise triggering the subsequent module; the dynamic fragment generation module is used for extracting effective pending deduction records, predicting the execution cost of each pending deduction record based on the multidimensional characteristics of the each pending deduction record, distributing the pending deduction records to a plurality of fragments by adopting an equalization algorithm, and generating a load-balanced fragment task list; the idempotent control module is used for generating a globally unique idempotent key for each slicing task of the slicing task list, and performing idempotent verification before the execution of the overall unique idempotent key so as to ensure that the same slicing task is executed only once in a preset time window; and the failure grading processing module is used for grading the failure records according to the failure codes when the failure codes returned by the deduction result are received, and executing differentiation processing according to different grades, wherein the differentiation processing comprises retry processing, final failure processing or calling an external post-credit system interface.
  9. 9. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, realizes the steps of the method according to any one of claims 1 to 7.
  10. 10. A computer apparatus, comprising: A memory having a computer program stored thereon; a processor which when executing the computer program carries out the steps of the method of any one of claims 1 to 7.

Description

Distributed batch deduction method, system, program product and device Technical Field The invention relates to the field of financial science and technology, in particular to a distributed batch deduction method, a system, a program product and a device Background In the field of financial science and technology, post-credit management of credit businesses commonly employs an automated deduction repayment mechanism. The loan institution typically initiates a batch fund deduction to a large number of borrower accounts through the payment channel on each payoff day to ensure timeliness of the fund return and stability of the property quality. Such batch deduction operations have the typical characteristics of high concurrency, strong consistency and low fault tolerance, namely, on one hand, the funds transfer of hundreds of thousands or millions of users is completed in a short time (such as several hours), and on the other hand, any repeated deduction, missing deduction or invalid retry can directly cause customer complaints, supervision penalties and even funds loss. Currently, batch deduction modules in mainstream credit systems mostly adopt static fragmentation strategies based on borrower ID hashing or contract number segmentation, and rely on a timing task scheduling framework (such as XXL-JOB and Quartz) for driving and executing. However, in actual operation, the static slicing mode described above exposes many drawbacks, such as: In the prior art, a static fragmentation mechanism based on user ID hash or contract number segmentation is generally adopted, and the dynamic change of data distribution cannot be perceived. When the high-activity or high-failure rate users are concentrated in a certain hash interval, the time consumption of the corresponding slicing process is obviously higher than that of other slicing, a long tail effect is formed, the whole batch buckling operation completion time is dragged slowly, and the lateral expansion capability of the system is restricted. The existing scheme usually realizes idempotent only in a single transaction layer, and lacks idempotent guarantee for executing the hierarchical tasks. When the same slicing task is repeatedly delivered due to abnormal scheduler, network retransmission or misoperation of operation and maintenance, the system can initiate deduction requests for the same batch of users for a plurality of times, and the risk of repeated fund deduction is caused. The existing batch deduction system adopts unified retry logic for all failure records, and can not carry out differentiated treatment according to the root cause of failure. The instantaneous failure and the service final failure are processed in a mixed mode, so that not only is the payment gateway resource wasted, but also the system load can be aggravated due to invalid retry. Because the slicing rules are fixed in advance, even if some slices have no record to be processed in a specific batch, the scheduling framework still allocates the executor resources for the slices and starts the task flow, so that the waste of computing resources is caused. Disclosure of Invention One of the objectives of the present invention is to provide a distributed batch payment method, which solves at least one of the above drawbacks of the prior art. One embodiment of the present invention relates to a distributed batch deduction method, herein "batch deduction" is abbreviated as "batch deduction", comprising the steps of: s100, when the daily batch buckling task is triggered, the validity pre-detection is carried out on the data to be processed, S200, through pre-checking, terminating the batch deduction flow if no effective record to be deducted exists, otherwise, entering step S300; s300, extracting effective to-be-deducted records, predicting the execution cost of each to-be-deducted record based on the multidimensional characteristics of the to-be-deducted records, distributing the to-be-deducted records to a plurality of fragments by adopting an equalization algorithm, and generating a load-balanced fragment task list; S400, generating globally unique idempotent keys for each slicing task of the slicing task list, and performing idempotent verification before executing the globally unique idempotent keys to ensure that the same slicing task is executed only once in a preset time window; S500, when a failure code returned by the deduction result is received, grading the failure record according to the failure code, and executing differentiation processing according to different grades, wherein the differentiation processing comprises retry processing, final failure processing or calling an external post-credit system interface. In some embodiments, in step S100, the performing validity pre-checking on the data to be processed includes obtaining product information and time batch information of batch deductions to be performed on the same day, generating a pre-checking task record for each produc