CN-116225867-B - Blacklist generation method and device, electronic equipment and storage medium

CN116225867BCN 116225867 BCN116225867 BCN 116225867BCN-116225867-B

Abstract

The disclosure provides a blacklist generation method, a blacklist generation device, electronic equipment and a storage medium, relates to the technical field of computers, and particularly relates to the technical field of artificial intelligence such as automatic driving, data processing and simulation testing. The method comprises the steps of responding to a simulation task initiating request, calling a current target blacklist, shielding machines located in the target blacklist in a cluster, submitting the simulation task to the cluster to obtain operation result data of the simulation task, mining the operation result data to obtain a fault machine with a problem in the simulation task, and updating the target blacklist based on the fault machine. According to the scheme disclosed by the invention, the blacklist can be automatically generated in the resource scheduling, so that the machine in the current latest target blacklist is shielded in the next resource scheduling, and the running stability and efficiency of the simulation task are improved.

Inventors

ZHOU JIE
LI BIN

Assignees

北京百度网讯科技有限公司

Dates

Publication Date: 20260512
Application Date: 20221229

Claims (17)

1. A blacklist generation method is applied to an automatic driving simulation test cluster and comprises the following steps: In response to a simulation task initiating request, a current target blacklist is called, wherein the target blacklist comprises a machine identifier, a mining rule identifier, a version time stamp, an expected release time, a mining time and a time window corresponding to the mining time, and the time window comprises a starting time and an ending time; shielding machines in the cluster, which are located in the target blacklist; Submitting the simulation task to the cluster, and acquiring operation result data of the simulation task, wherein the operation result data comprises machine performance data and operation error codes; excavating the operation result data to obtain a fault machine with a problem in the simulation task; Triggering an excavating task at regular time by an excavating program, and updating the target blacklist based on each faulty machine; The updating the target blacklist includes: traversing the mining rules corresponding to the fault machine, setting the predicted release time of the fault machine as the maximum value of the predicted release time corresponding to each mining rule, and dynamically updating the predicted release time based on the mining rules and/or maintenance state data; Updating the target blacklist based on the machine identification, the mining rule identification, the version timestamp, the mining time and the time window when the predicted release time is greater than a current time; and when the predicted release time is less than or equal to the current time, moving the faulty machine out of the target blacklist.
2. The method of claim 1, further comprising: Acquiring a release schedule, wherein the release schedule comprises predicted release time corresponding to each machine; Any machine in the target blacklist is removed from the target blacklist in response to detecting that the any machine reaches an expected release time for the any machine.
3. The method of claim 2, further comprising: Acquiring maintenance state data; updating the release schedule based on the repair status data.
4. The method of claim 1, further comprising: According to the machine type and/or the simulation task type, making an excavating rule; the mining of the operation result data to obtain a fault machine with problems in the simulation task comprises the following steps: And mining the operation result data according to the mining rule to obtain a fault machine with a problem in the simulation task.
5. The method of claim 4, wherein the mining the operation result data according to the mining rules results in a failed machine that has a problem in the simulation task, comprising: And mining based on the operation error code in the operation result data to obtain a first type of fault machine with the target operation error code.
6. The method of claim 4, wherein the mining the operation result data according to the mining rules results in a failed machine that has a problem in the simulation task, comprising: and mining based on the number of kinds of the operation error codes of the same machine in the operation result data to obtain a second kind of fault machine with the number of kinds larger than a first threshold value.
7. The method of claim 4, wherein the mining the operation result data according to the mining rules results in a failed machine that has a problem in the simulation task, comprising: And excavating based on the frequency of task execution failure of the same machine in a preset time period, and obtaining a third class of fault machine with the frequency larger than a second threshold value.
8. A blacklist generation device is applied to an automatic driving simulation test cluster and comprises: The pulling module is used for responding to a simulation task initiating request and calling a current target blacklist, wherein the target blacklist comprises a machine identifier, an excavating rule identifier, a version timestamp, an expected release time, an excavating time and a time window corresponding to the excavating time, and the time window comprises a starting time and an ending time; the shielding module is used for shielding the machines in the cluster, which are positioned in the target blacklist; The first acquisition module is used for submitting the simulation task to the cluster and acquiring operation result data of the simulation task, wherein the operation result data comprises machine performance data and operation error codes; The mining module is used for mining the operation result data to obtain a fault machine with a problem in the simulation task; the first updating module is used for triggering an excavating task at fixed time through an excavating program and updating the target blacklist based on each faulty machine; The first update module includes: The time updating sub-module is used for traversing the mining rules corresponding to the fault machine, setting the predicted release time of the fault machine to be the maximum value of the predicted release time corresponding to each mining rule, and dynamically updating the predicted release time based on the mining rules and/or maintenance state data; A list adding sub-module, configured to update the target blacklist based on the machine identifier, the mining rule identifier, the version timestamp, the mining time and the time window when the expected release time is greater than a current time; and the list removing sub-module is used for removing the fault machine from the target blacklist when the predicted release time is less than or equal to the current time.
9. The apparatus of claim 8, further comprising: the second acquisition module is used for acquiring a release schedule, wherein the release schedule comprises the expected release time corresponding to each machine; And a removal module for removing any machine from the target blacklist in response to detecting that any machine in the target blacklist reaches an expected release time of the any machine.
10. The apparatus of claim 9, further comprising: The third acquisition module is used for acquiring maintenance state data; and a second updating module for updating the release schedule based on the maintenance status data.
11. The apparatus of claim 8, further comprising: the formulating module is used for formulating the mining rule according to the machine type and/or the simulation task type; the mining module is specifically configured to mine the operation result data according to the mining rule, so as to obtain a fault machine with a problem in the simulation task.
12. The apparatus of claim 11, wherein the mining module comprises: the first mining submodule is used for mining based on the operation error code in the operation result data to obtain a first type of fault machine with the target operation error code.
13. The apparatus of claim 11, wherein the mining module comprises: The second mining sub-module is used for mining based on the type number of the running error codes of the same machine in the running result data, and obtaining a second type of fault machine with the type number larger than a first threshold value.
14. The apparatus of claim 11, wherein the mining module comprises: and the third excavating sub-module is used for excavating based on the frequency of the task failure executed by the same machine in a preset time period, and obtaining a third type of fault machine with the frequency larger than a second threshold value.
15. An electronic device, comprising: at least one processor, and A memory communicatively coupled to the at least one processor, wherein, The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.
16. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-7.
17. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any of claims 1-7.

Description

Blacklist generation method and device, electronic equipment and storage medium Technical Field The disclosure relates to the field of computer technology, and in particular to the technical field of artificial intelligence such as automatic driving, data processing, simulation testing and the like. Background In autopilot simulation cluster operation, each simulation batch computing task is scheduled to multiple different clusters, which are a resource pool consisting of hundreds of physical machines. Some machines in the cluster may be in a false active state, and no anomaly is seen at the monitoring level, but two problems occur when the actual task runs on such machines. The machine performance is obviously reduced compared with other normal machines, and the operation efficiency of simulation tasks is affected. Such machines frequently appear in clusters, resulting in various degrees of degradation in stability and performance of business calculations. Disclosure of Invention The disclosure provides a blacklist generation method, a blacklist generation device, electronic equipment and a storage medium. According to a first aspect of the present disclosure, there is provided a blacklist generation method, including: Responding to a simulation task initiating request, and calling a current target blacklist; Shielding machines in the cluster, which are located in the target blacklist; Submitting simulation tasks to the clusters, and obtaining operation result data of the simulation tasks; excavating operation result data to obtain a fault machine with problems in the simulation task; the target blacklist is updated based on the failed machine. According to a second aspect of the present disclosure, there is provided a blacklist generating apparatus including: the pulling module is used for responding to the simulation task initiating request and calling the current target blacklist; the shielding module is used for shielding the machines positioned in the target blacklist in the cluster; the first acquisition module is used for submitting simulation tasks to the cluster and acquiring operation result data of the simulation tasks; the mining module is used for mining the operation result data to obtain a fault machine with a problem in the simulation task; and the first updating module is used for updating the target blacklist based on the fault machine. According to a third aspect of the present disclosure, there is provided an electronic device comprising: at least one processor, and A memory communicatively coupled to the at least one processor, wherein, The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of the embodiments of the present disclosure. According to a fourth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium storing computer instructions for causing the computer to perform a method according to any one of the embodiments of the present disclosure. According to a fifth aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements a method according to any of the embodiments of the present disclosure. According to the scheme disclosed by the invention, the blacklist can be automatically generated in the resource scheduling, so that the machine in the current latest target blacklist is shielded in the next resource scheduling, and the running stability and efficiency of the simulation task are improved. It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification. Drawings In the drawings, the same reference numerals refer to the same or similar parts or elements throughout the several views unless otherwise specified. The figures are not necessarily drawn to scale. It is appreciated that these drawings depict only some embodiments according to the disclosure and are not therefore to be considered limiting of its scope. Fig. 1 is a flow diagram of a blacklist generation method according to an embodiment of the present disclosure; FIG. 2 is a general flow diagram of automated blacklist generation in accordance with an embodiment of the present disclosure; FIG. 3 is a schematic diagram of a blacklist generation mechanism according to an embodiment of the present disclosure; fig. 4 is a schematic diagram of a blacklist validation mechanism according to an embodiment of the present disclosure; FIG. 5 is a schematic diagram one of mining a failed machine according to mining rules, according to an embodiment of the present disclosure; FIG. 6 is a schematic diagram II of mining a failed machine according to mining rule