CN-121979642-A - Lightweight monomer high concurrency scheduling method and system

CN121979642ACN 121979642 ACN121979642 ACN 121979642ACN-121979642-A

Abstract

The invention provides a lightweight single body high concurrency scheduling method and system, which relate to the technical field of high concurrency task scheduling and comprise the steps of pre-reading task configuration information when a program is started, comparing current system time with task next triggering time, adding an un-timed-out task into a memory-based time scaler to align next triggering time and calculate trigger expiration time to perform time compensation, inquiring an execution state identification of the task in a memory database when the triggering time arrives, adjusting a triggering strategy if the task is in a blocking state, distributing the task to a fast thread pool or a slow thread pool for execution according to task historical execution time if the task is not blocked, recording a task triggering time stamp, an execution state and next triggering time by using the memory database, and refreshing the next execution time immediately after the task is triggered. The invention has the beneficial effects of realizing task triggering with millisecond precision and concurrency processing capacity of more than 1500, and solving the technical problem of task blocking and insufficient time precision in a single environment.

Inventors

YONG HONGCHENG
YU CHUANHAO
PENG BIAO

Assignees

成都天奥电子股份有限公司

Dates

Publication Date: 20260505
Application Date: 20260210

Claims (10)

1. The lightweight monomer high concurrency scheduling method is characterized by comprising the following steps of: Pre-reading task configuration information when a program is started, comparing the current system time with the next trigger time of a task, adding an un-overtime task into a time scaler based on a memory to align the next trigger time, and calculating the expiration time of the trigger to perform time compensation; When the trigger time arrives, inquiring the execution state identification of the task in the memory database, if the task is in a blocking state, adjusting the trigger strategy, and if the task is not blocked, distributing the task to a fast thread pool or a slow thread pool for execution according to the historical execution time of the task; and recording the task trigger time stamp, the execution state and the next trigger time by using the memory database, and refreshing the next execution time immediately after the task is triggered.
2. The method for high concurrency scheduling of lightweight monomers as recited in claim 1, wherein the assigning tasks to fast thread pool or slow thread pool execution based on task history execution time consumption comprises: and when the average execution time of the task history is lower than a preset threshold, the task history is distributed to a fast thread pool, and when the average execution time of the task history is higher than the preset threshold, the task history is distributed to a slow thread pool, the number of core threads configured by the fast thread pool is smaller than that of the slow thread pool, and the preset threshold is dynamically adjusted based on the sliding average of the execution time of the task history.
3. The method for high concurrency scheduling of lightweight monomers according to claim 1, wherein the step of pre-reading task configuration information at program start comprises loading task identifiers, trigger rules and historical average execution time length into a memory for cache pre-heating, and calibrating system time based on clock synchronization service to ensure millisecond alignment accuracy.
4. The method for scheduling lightweight single bodies according to claim 1, wherein calculating the expiration time of the trigger for time compensation comprises calculating a deviation value between the current system time and the theoretical trigger time of the task, calibrating the next trigger time by a compensation algorithm when the deviation value is within a preset compensation threshold, and marking the task as a delay state and triggering compensation execution when the deviation value exceeds the compensation threshold.
5. The method for high concurrency scheduling of lightweight entities according to claim 1, wherein the querying the execution state identifier of the task in the memory database, and if in the blocking state, adjusting the trigger policy, comprises: and if the execution state is marked as 'in-process', updating the execution state as 'in-process' and allowing the thread pool to execute.
6. The method for high concurrency scheduling of lightweight monomers according to claim 1, wherein the recording task trigger time stamp, execution state and next trigger time by using an in-memory database comprises: and adopting a transaction mechanism to ensure the atomicity operation of the task trigger time stamp and the state update, setting a refreshing strategy to periodically asynchronously refresh the task execution log into a persistent storage medium, and simultaneously keeping the trigger time data only stored in a memory layer.
7. The method for high concurrency scheduling of lightweight monomers of claim 1, wherein said adjusting the trigger policy if in a blocking state further comprises: when the task is detected to be in a blocking state after being continuously triggered for a plurality of times, automatically lifting the task to the priority of the slow thread pool; and when the accumulation task number of the single-thread pool exceeds the warning threshold value, dynamically expanding the maximum thread number of the corresponding thread pool and triggering a task distribution rebalancing mechanism.
8. The method for scheduling high concurrency of lightweight monomers according to claim 1, wherein adding the non-timeout task into the memory-based time scaler to align the next trigger time comprises implementing the time scaler by using a layered time-round algorithm or a delay queue, wherein the default configuration core thread number of the fast thread pool and the slow thread pool is 5 and the maximum thread number is 20, and supporting task concurrency above 1500 in a non-dilatation state.
9. The method of claim 1, further comprising providing a real-time interface of the Web interface to return the trigger time, the waiting time, the execution state, the next scheduling time and the thread pool load index of all the current tasks, and refreshing a time axis sequence for showing task execution every second through a long connection or a polling mechanism.
10. A lightweight single body high concurrency scheduling system based on the lightweight single body high concurrency scheduling method of claim 1, comprising: the pre-reading processing module is used for pre-reading task configuration information when a program is started, and comparing the current system time with the next trigger time of the task; The time alignment engine is used for adding the non-overtime task into the memory-based time scaler to align the next trigger time and calculating the expiration time of the trigger for time compensation; the double-thread pool scheduler is used for inquiring the execution state identification of the task in the memory database when the trigger time arrives, adjusting the trigger strategy if the trigger time is in a blocking state, and distributing the task to a fast thread pool or a slow thread pool for execution according to the historical execution time of the task if the trigger time is not blocked; The memory database management layer is used for recording the task trigger time stamp, the execution state and the next trigger time, and refreshing the next execution time immediately after the task is triggered.

Description

Lightweight monomer high concurrency scheduling method and system Technical Field The invention relates to the technical field of high concurrency task scheduling, in particular to a lightweight single body high concurrency scheduling method and system. Background With the rapid development of internet services, the number of timing tasks to be carried by a single application program increases exponentially, and the precision requirements of scenes such as financial transaction, real-time monitoring, accurate control and the like on task triggering time reach millisecond level. In the prior art, task triggering is generally realized by adopting a mode of matching a single global thread pool with system clock polling, all tasks enter the same waiting queue no matter how long the execution time is, or trigger tasks are compared only through simple time stamps. When facing thousands of concurrent tasks, the scheme is extremely easy to cause task triggering delay due to I/O blocking, thread contention and clock drift, and even causes task accumulation and system halt in severe cases. In order to alleviate the problems, the prior art is laterally expanded by means of increasing the scale of a thread pool without limit, deploying a distributed scheduling cluster or introducing a heavyweight message middleware, and the like, however, the means not only significantly increase the system resource consumption and the operation and maintenance complexity, but also can not solve the fundamental contradiction of clock precision loss and task execution blocking in a single machine scene, and are difficult to meet the dual requirements of light deployment and high-precision triggering. Disclosure of Invention The invention aims to provide a lightweight single body high concurrency scheduling method and system so as to solve the problems. In order to achieve the above purpose, the technical scheme adopted by the invention is as follows: in a first aspect, the present application provides a lightweight single body high concurrency scheduling method, including the following steps: Pre-reading task configuration information when a program is started, comparing the current system time with the next trigger time of a task, adding an un-overtime task into a time scaler based on a memory to align the next trigger time, and calculating the expiration time of the trigger to perform time compensation; When the trigger time arrives, inquiring the execution state identification of the task in the memory database, if the task is in a blocking state, adjusting the trigger strategy, and if the task is not blocked, distributing the task to a fast thread pool or a slow thread pool for execution according to the historical execution time of the task; and recording the task trigger time stamp, the execution state and the next trigger time by using the memory database, and refreshing the next execution time immediately after the task is triggered. Preferably, the task is allocated to the fast thread pool or the slow thread pool for execution according to the task history execution time, which comprises the following steps: and when the average execution time of the task history is lower than a preset threshold, the task history is distributed to a fast thread pool, and when the average execution time of the task history is higher than the preset threshold, the task history is distributed to a slow thread pool, the number of core threads configured by the fast thread pool is smaller than that of the slow thread pool, and the preset threshold is dynamically adjusted based on the sliding average of the execution time of the task history. Preferably, the pre-reading task configuration information when the program is started comprises loading task identification, trigger rules and historical average execution duration into a memory for cache pre-heating, and calibrating system time based on clock synchronization service to ensure millisecond alignment accuracy. Preferably, calculating the expiration time of the trigger for time compensation comprises calculating a deviation value of the current system time and the theoretical trigger time of the task, calibrating the next trigger time by a compensation algorithm when the deviation value is within a preset compensation threshold value range, marking the task as a delay state when the deviation value exceeds the compensation threshold value, and triggering compensation execution. Preferably, the querying the execution state identifier of the task in the memory database, and if the task is in the blocking state, adjusting the trigger policy includes: and if the execution state is marked as 'in-process', updating the execution state as 'in-process' and allowing the thread pool to execute. Preferably, the recording the task trigger time stamp, the execution state and the next trigger time by using the in-memory database includes: and adopting a transaction mechanism to ensure the atomicity operation of