Search

CN-116340350-B - Method, device and storage medium for generating computing job

CN116340350BCN 116340350 BCN116340350 BCN 116340350BCN-116340350-B

Abstract

The embodiment of the application provides a method, equipment and storage medium for generating a computing job. The method comprises the steps of obtaining a first structured query language to create a draft entity containing the first structured query language, obtaining first configuration information of the first deployment entity based on the first deployment entity creating a stream mode, starting the first deployment entity to create a first instance of the stream mode and a first computing job corresponding to the first instance and running according to the first configuration information, obtaining second configuration information of the second deployment entity based on the second deployment entity creating a batch mode, starting the second deployment entity to create a second instance of the batch mode and a second computing job corresponding to the second instance and running according to the second configuration information, and guaranteeing consistency of results of the stream job and the batch job.

Inventors

  • XIE YADONG

Assignees

  • 阿里巴巴（中国）有限公司

Dates

Publication Date
20260505
Application Date
20230328

Claims (12)

  1. 1. A method of generating a computing job, comprising: obtaining a first structured query language to create a draft entity containing the first structured query language; creating a first deployment entity of a stream mode based on the draft entity, acquiring first configuration information of the first deployment entity, and starting the first deployment entity to create a first instance of the stream mode and a first computing job corresponding to the first instance and running according to the first configuration information; and creating a second deployment entity in a batch mode based on the draft entity, acquiring second configuration information of the second deployment entity, and starting the second deployment entity to create a second instance in the batch mode and a second computing job corresponding to the second instance and running according to the second configuration information.
  2. 2. The method as recited in claim 1, further comprising: acquiring a second structured query language, and updating the first structured query language in the draft entity into the second structured query language; Updating a target deployment entity based on the updated draft entity, wherein the target deployment entity comprises the first deployment entity and/or the second deployment entity.
  3. 3. The method of claim 2, wherein the target deployment entity comprises the first deployment entity, the method further comprising: stopping the first instance entity; and starting the updated first deployment entity to create a third instance of the streaming mode and a third computing job corresponding to the third instance and running according to the first configuration information.
  4. 4. The method of claim 2, wherein the target deployment entity comprises the second deployment entity, the method further comprising: And starting the updated second deployment entity to create a fourth instance of the batch mode and a fourth computing job corresponding to the fourth instance and running according to the second configuration information.
  5. 5. The method as recited in claim 1, further comprising: modifying configuration information of a target deployment entity into target configuration information, wherein the target deployment entity is the first deployment entity, the configuration information of the target deployment entity is the first configuration information, the target configuration information is the third configuration information, or the target deployment entity is the second deployment entity, the configuration information of the target deployment entity is the second configuration information, and the target configuration information is the fourth configuration information.
  6. 6. The method of claim 5, wherein the target deployment entity is the first deployment entity, the method further comprising: stopping the first instance entity; and starting the updated first deployment entity to create a fifth instance of the streaming mode and a fifth computing job corresponding to the fifth instance and running according to the third configuration information.
  7. 7. The method of claim 5, wherein the target deployment entity is the second deployment entity, the method further comprising: And starting the updated second deployment entity to create a sixth instance of the batch mode and a sixth computing job corresponding to the sixth instance and running according to the fourth configuration information.
  8. 8. A method of generating a computing job, comprising: Creating a draft entity containing a first structured query language in response to a draft entity creation request, wherein the draft creation request comprises the first structured query language; Responding to a stream mode deployment entity creation request, and creating a first deployment entity of a stream mode based on the draft entity; responding to a first starting instruction, starting the first deployment entity to create a first instance of a stream mode and a first computing job corresponding to the first instance and running according to the first configuration information; The method comprises the steps of receiving a first configuration instruction for the first deployment entity, responding to a first configuration instruction, starting the first deployment entity to create a first instance of the batch mode and a first computing job corresponding to the first instance and running according to the first configuration information, wherein the first configuration instruction comprises first configuration information, and the first computing job corresponds to the first instance.
  9. 9. A method of generating a computing job, comprising: sending a draft entity creation request to a computing platform, wherein the draft entity creation request comprises a first structured query language input by a user, and the first structured query language is used for creating a draft entity; Sending a stream pattern deployment entity creation request to the computing platform, so that the computing platform creates a first deployment entity based on the stream pattern of the draft entity; sending a first configuration instruction aiming at the first deployment entity to the computing platform, wherein the first configuration instruction comprises first configuration information; sending a first starting instruction to the computing platform so that the computing platform starts the first deployment entity to create a first instance of a streaming mode and a first computing job corresponding to the first instance and running according to the first configuration information; the method comprises the steps of receiving a batch mode deployment entity creation request from a computing platform, enabling the computing platform to create a batch mode second deployment entity based on a draft entity, sending a second configuration instruction for the second deployment entity to the computing platform, wherein the second configuration instruction comprises second configuration information, and sending a second starting instruction to the computing platform, so that the computing platform starts the second deployment entity to create a batch mode second instance and a second computing job corresponding to the second instance and running according to the second configuration information.
  10. 10. A computing platform comprising a memory and a processor; The memory is used for storing a computer program; The processor being configured to execute a computer program stored in the memory, the computer program when run causing the processor to perform the method of any one of claims 1-8.
  11. 11. A client is characterized by comprising a memory and a processor; The memory is used for storing a computer program; the processor configured to execute a computer program stored in the memory, the computer program when run causes the processor to perform the method of claim 9.
  12. 12. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a computer program which, when executed by a processor, causes the processor to perform the method according to any of claims 1-9.

Description

Method, device and storage medium for generating computing job Technical Field The present application relates to the field of cloud computing, and more particularly, to a method, apparatus, and storage medium for generating a computing job. Background In big data processing, batch computing and stream computing are very important concepts, in general, data is collected and then processed in batch computing, while stream computing is processing data in real-time, and a computing engine supports running computing jobs in stream mode or batch mode. The current mainstream architecture is that a set of stream calculation schemes and a set of batch calculation schemes are parallel, and the two schemes are independently developed and maintained, but the inventor discovers that the calculation results obtained by adopting stream operation and batch operation are often different for the same calculation requirement and the same data source. Disclosure of Invention The application provides a method, equipment and storage medium for generating a computing job, so as to ensure the consistency of results of a stream job and a batch job. In a first aspect, the present application provides a method for generating a computing job, including: obtaining a first structured query language to create a draft entity containing the first structured query language; creating a first deployment entity of a stream mode based on the draft entity, acquiring first configuration information of the first deployment entity, and starting the first deployment entity to create a first instance of the stream mode and a first computing job corresponding to the first instance and running according to the first configuration information; and creating a second deployment entity in a batch mode based on the draft entity, acquiring second configuration information of the second deployment entity, and starting the second deployment entity to create a second instance in the batch mode and a second computing job corresponding to the second instance and running according to the second configuration information. Optionally, the method further comprises: acquiring a second structured query language, and updating the first structured query language in the draft entity into the second structured query language; Updating a target deployment entity based on the updated draft entity, wherein the target deployment entity comprises the first deployment entity and/or the second deployment entity. Optionally, the target deployment entity includes the first deployment entity, and the method further includes: stopping the first instance entity; and starting the updated first deployment entity to create a third instance of the streaming mode and a third computing job corresponding to the third instance and running according to the first configuration information. Optionally, the target deployment entity includes the second deployment entity, and the method further includes: And starting the updated second deployment entity to create a fourth instance of the batch mode and a fourth computing job corresponding to the fourth instance and running according to the second configuration information. Optionally, the method further comprises: modifying configuration information of a target deployment entity into target configuration information, wherein the target deployment entity is the first deployment entity, the configuration information of the target deployment entity is the first configuration information, the target configuration information is the third configuration information, or the target deployment entity is the second deployment entity, the configuration information of the target deployment entity is the second configuration information, and the target configuration information is the fourth configuration information. Optionally, the target deployment entity is the first deployment entity, and the method further includes: stopping the first instance entity; and starting the updated first deployment entity to create a fifth instance of the streaming mode and a fifth computing job corresponding to the fifth instance and running according to the third configuration information. Optionally, the target deployment entity is the second deployment entity, and the method further includes: And starting the updated second deployment entity to create a sixth instance of the batch mode and a sixth computing job corresponding to the sixth instance and running according to the fourth configuration information. In a second aspect, the present application provides a method for generating a computing job, including: Creating a draft entity containing a first structured query language in response to a draft entity creation request, wherein the draft creation request comprises the first structured query language; Responding to a stream mode deployment entity creation request, and creating a first deployment entity of a stream mode based on the draft entity; responding to a first