CN-121979647-A - Method, device, equipment, medium and program product for analyzing batch job duration

CN121979647ACN 121979647 ACN121979647 ACN 121979647ACN-121979647-A

Abstract

The disclosure provides an analysis method for batch operation duration, which can be applied to the technical field of big data and relates to the field of distributed artificial intelligence. The analysis method comprises the steps of responding to an analysis request of batch job duration, obtaining a job list of a target application from a distributed batch processing database, the execution duration of each job in the job list and the calling relation among each job, determining the weight of each job according to the execution duration, determining the execution path of each job according to the calling relation, determining the shortest weighted value path of each job from the execution paths based on the weight, and determining the batch job duration of each job according to the shortest weighted value path. The disclosure also provides an analysis device, equipment, medium and program product for batch job duration.

Inventors

BI PING
DING JIE
YUAN YAHUI
Ge Chengzhe

Assignees

中国工商银行股份有限公司

Dates

Publication Date: 20260505
Application Date: 20250620

Claims (11)

1. A method for analyzing a length of a batch job, the method comprising: responding to an analysis request of batch job duration, and acquiring a job list of a target application, the execution duration of each job in the job list and the calling relation among each job from a distributed batch processing database; Determining the weight of each job according to the execution duration; determining the execution path of each job according to the calling relation; determining the shortest weighted value path of each job from the execution paths based on the weights, and And determining the batch job duration of each job according to the shortest weighted value path.
2. The method according to claim 1, wherein the obtaining, from the distributed batch processing database, a job list of the target application, execution time lengths of each job in the job list, and call relations between each job, includes: acquiring a job information data table of batch jobs from a distributed batch processing database, wherein the job information data table comprises job identifiers of different single jobs, application identifiers of forward jobs and execution time lengths of the forward jobs; Determining the application identifier of the application as the operation identifier of the single operation of the target application according to the operation identifier of each single operation and the application identifier of the application; obtaining a job list of the target application according to the job identification of the single job of the target application; Determining the forward operation of each operation in the operation list according to the operation identification of the forward operation of each single operation; determining the application identifier of the forward operation of each operation in the operation list according to the application identifier of the forward operation of each single operation; and determining the calling relation among the jobs in the job list according to the forward jobs of the jobs in the job list and the application identifiers of the forward jobs of the jobs in the job list.
3. The analysis method according to claim 2, wherein determining the call relationship between the jobs in the job list based on the forward jobs of the jobs in the job list and the application identifications of the forward jobs of the jobs in the job list includes: setting the forward operation of the operation in the operation list as a root node when the application identifier of the forward operation of the operation in the operation list is not the target application; When the application identifier of the forward operation of the operation in the operation list is the target application, obtaining a calling relation between the operation in the operation list and the forward operation; and determining the calling relation among the jobs in the job list according to the calling relation between the jobs in the job list and the forward jobs.
4. The method according to claim 3, wherein the number of execution paths of each job is at least one; A process for determining an execution path of a job in a job list according to the call relationship, comprising: taking one job in the job list as a current job, and determining a previous job of the current job according to the calling relation; under the condition that the previous job of the current job is not the root node, the previous job of the current job is used as the current job, the current job is updated, and the previous job of the current job is continuously determined according to the calling relation until the previous job of the current job is the root node; In the case where the previous job of the current job is the root node, a forward job included between the job in the job list and the previous job of the current job constitutes one execution path of one job of the job list.
5. The method of analyzing of claim 4, wherein determining a shortest weighted path for each job from the execution paths based on the weights comprises: Calculating a weighted value of each execution path of all execution paths of one job in the job list based on the weight for the one job in the job list; Selecting an execution path with the smallest weighted value from all execution paths of one job in a job list, and taking the execution path with the smallest weighted value as the shortest weighted value path of one job in the job list; The shortest weight path for each job is determined based on the shortest weight path for one job in the job list.
6. The method of claim 5, wherein calculating weighted values for all execution paths of a job in a job list based on the weights comprises: determining the weight of the previous job included in each execution path of one job in a job list based on the weight; And determining the weight value of each execution path of one job in the job list according to the weight of the previous job included in each execution path of one job in the job list.
7. The method of claim 4, wherein the batch job duration for each job is determined based on the execution duration of the preceding job included in the shortest weighted value path.
8. An analysis device for a length of a batch job, the analysis device comprising: the data acquisition module is used for responding to the analysis request of the batch job duration, and acquiring a job list of a target application, the execution duration of each job in the job list and the calling relation among each job from the distributed batch processing database; the weight determining module is used for determining the weight of each job according to the execution time length, wherein the weight of each job is inversely proportional to the execution time length; The path determining module is used for determining the execution path of each job according to the calling relation; a target path determining module for determining the shortest weighted value path of each job from the execution paths based on the weights, and And the duration determining module is used for determining the batch job duration of each job according to the shortest weighted value path.
9. An electronic device, comprising: One or more processors; a memory for storing one or more computer programs, Characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1-7.
10. A computer-readable storage medium, on which a computer program or instructions is stored, which, when executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.
11. A computer program product comprising a computer program or instructions which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 7.

Description

Method, device, equipment, medium and program product for analyzing batch job duration Technical Field The present disclosure relates to the field of big data, to the field of distributed, artificial intelligence, and more particularly to a method, apparatus, device, medium, and program product for analyzing a batch job duration. Background In the distributed technology field, as host traffic moves down, the distributed framework carries (Distributed Batch Framework, DBF) more and more core applications. From the statistics of DBF operation, it can be presumed that the resource consumption of all applications is large. By means of batch tasks, batch operation can be started at random, and the resource utilization rate of the distributed batch program is improved. In the batch task execution process, the batch execution window is divided by estimating the waiting time and the calling path of a single job. The current evaluation of the waiting time of a single job to be called mainly depends on manual experience, and the evaluation precision is low. Disclosure of Invention In view of the foregoing, the present disclosure provides an analysis method, apparatus, device, medium, and program product for a batch job duration that improves the division accuracy of a batch execution window. According to a first aspect of the disclosure, an analysis method for a batch job duration is provided, which includes obtaining a job list of a target application, execution durations of jobs in the job list and call relations among the jobs from a distributed batch processing database in response to an analysis request for the batch job duration, determining weights of the jobs according to the execution durations, determining execution paths of the jobs according to the call relations, determining shortest weighted value paths of the jobs from the execution paths based on the weights, and determining batch job durations of the jobs according to the shortest weighted value paths. According to the embodiment of the disclosure, the method for acquiring the job list of the target application from the distributed batch processing database, the execution time of each job in the job list and the calling relation among the jobs comprise the steps of acquiring a job information data table of batch jobs from the distributed batch processing database, wherein the job information data table comprises job identifiers of different single jobs, application identifiers, forward job identifiers and forward job identifiers, the execution time, determining the application identifiers as the job identifiers of the single jobs of the target application according to the job identifiers of the single jobs and the application identifiers, acquiring the job list of the target application according to the job identifiers of the single jobs of the target application, determining the forward job of each job in the job list according to the forward job identifiers of the single jobs, determining the application identifiers of the forward job in the job list according to the application identifiers of the forward job of each single job, and the calling relation among the jobs in the application list according to the forward job of each job in the job list and the forward job identifiers of each job in the job list. According to the embodiment of the disclosure, according to the forward operation of each operation in the operation list and the application identifier of the forward operation of each operation in the operation list, the calling relationship between each operation in the operation list is determined, wherein the calling relationship between each operation in the operation list is determined according to the calling relationship between each operation in the operation list and the forward operation when the application identifier of the forward operation of each operation in the operation list is not the target application, the forward operation of each operation in the operation list is set as a root node, and the calling relationship between each operation in the operation list is obtained when the application identifier of the forward operation of each operation in the operation list is the target application. According to the embodiment of the disclosure, the number of execution paths of each job is at least one, the process of determining one execution path of one job in a job list according to the calling relation comprises the steps of determining the front job of the current job according to the calling relation by taking one job in the job list as the current job, updating the current job by taking the front job of the current job as the current job when the front job of the current job is not the root node, and continuing to determine the front job of the current job according to the calling relation until the front job of the current job is the root node, wherein when the front job of the current job is the root node, the front job i