CN-121998330-A - Digital twin workshop real-time scheduling method and device oriented to limited transportation resources and charging constraint scene

CN121998330ACN 121998330 ACN121998330 ACN 121998330ACN-121998330-A

Abstract

The invention belongs to the technical field of intelligent manufacturing, and discloses a digital twin workshop real-time scheduling method oriented to limited transportation resources and charging constraint scenes, which comprises the steps of constructing a real-time scheduling frame based on digital twin and deep reinforcement learning, and realizing real-time interaction of virtual and real data; the method comprises the steps of providing a two-stage real-time scheduling model based on deep reinforcement learning, establishing a DFJSP-LTR-C Markov decision process with minimum working time as a target, designing five key elements of interaction points, real-time scheduling flow, real-time state characteristics, action space based on improved genetic programming and a compound rewarding function for the scheduling model, and providing an IAD3QN training method based on a multi-head attention mechanism to train a scheduling agent. The invention realizes the self-adaptive collaborative optimization of production and transportation resources under the condition of considering limited transportation resources and charging constraint, effectively reduces the maximum finishing time and improves the feasibility and the robustness of the scheduling scheme in an actual workshop.

Inventors

ZHU HAIPING
WU RUITING
LIU SIQI
GUAN HUI
Shen Liezheng
LIU YANQIANG

Assignees

华中科技大学

Dates

Publication Date: 20260508
Application Date: 20260120

Claims (10)

1. The digital twin workshop real-time scheduling method for the limited transportation resource and charging constraint scene is characterized by comprising the following steps of: S1, constructing a real-time scheduling frame based on digital twin and deep reinforcement learning; S2, on the basis of the step S1, a two-stage real-time scheduling model based on deep reinforcement learning is provided; S3, based on the two-stage real-time scheduling model of deep reinforcement learning proposed in the step S2, modeling a flexible job shop dynamic scheduling problem under the constraint of limited transportation resources and charging as a Markov decision process with the aim of minimizing the working time, and sequentially designing five key elements of an interaction point, a real-time scheduling flow, a real-time state characteristic, an action space based on improved genetic programming and a compound rewarding function of the scheduling model; S4, based on the step S3, providing a training method using IAD3QN to enable the scheduling agent to have feature extraction capability and dynamic decision capability, generating training cases with different scales and different disturbance levels in a digital twin simulation model, training the provided IAD3QN network, continuously updating IAD3QN network parameters, improving scheduling performance and generalization capability of the IAD3QN network, and obtaining the scheduling agent applicable to actual production decision; And S5, after training in the step S4 is completed, selecting an online application model of the scheduling agent, and when a new order arrives at the workshop, verifying and outputting a scheduling scheme by using the trained IAD3QN model according to the real-time workshop state of each interaction point by the scheduling agent to guide the physical workshop to complete the cooperative allocation of workpieces, AGVs and machines.
2. The method for real-time scheduling of digital twin workshops for limited transportation resources and charging constraints as defined in claim 1, wherein the real-time scheduling framework in step S1 comprises a physical layer, a data layer, a virtual layer and a service layer, and wherein: The physical layer is composed of all production elements of a workshop and comprises various manufacturing resources, a monitoring module and a control system; the data layer is used for completing the work of collecting, transmitting, processing and storing data; a virtual layer, which is to construct a simulation model based on Factory simulation platform and provide high-fidelity training environment and scheduling scheme verification function; The service layer comprises a real-time dynamic scheduling model and a three-dimensional model, wherein the real-time dynamic scheduling model is used for scheduling an intelligent agent, outputting scheduling instructions, and the three-dimensional model maps the production state in real time and supports visual angle switching and time axis playback.
3. The digital twin shop real-time scheduling method for limited transportation resources and charging constraints scenario according to claim 1, wherein the two-stage real-time scheduling model in step S2 comprises an offline learning stage and an online application stage, wherein: The scheduling agent and the virtual workshop perform a large amount of interaction in the offline learning stage, the performance of the training agent is continuously optimized, and the trained scheduling agent is finally obtained; The scheduling agent in the online application stage verifies and outputs a scheduling scheme based on the real-time state of the workshop, and retrains or fine-tunes model network parameters by means of a self-learning mechanism.
4. The method for real-time dispatching of digital twin workshops for limited transportation resources and charging constraint scenes according to claim 1, wherein the method for designing the interaction points in step S3 comprises the following steps: Setting a time-driven interaction point, wherein the scheduling agent scans the workshop state at fixed time steps, and the time step t meets the following two conditions simultaneously, namely the interaction point is that one or more workpieces are waiting to be transported, and at least one AGV is in an idle state and is not allocated with a charging task.
5. The digital twin shop real-time scheduling method for limited transportation resources and charging constraint scenes according to claim 1, wherein the design method of the real-time scheduling process in step S3 comprises the following steps: the method comprises the steps of dividing a real-time scheduling process into a scheduling stage and a processing stage, triggering scheduling at each interaction point by a scheduling agent, entering the scheduling stage, sequencing the priority of workpieces according to a scheduling rule, scheduling the workpieces with higher priority first, calculating the residual electric quantity of each AGV after completing a current task sequence before distributing the AGVs, inserting a charging task if the electric quantity is lower than a threshold value, and leading the AGVs to a charging pile after completing the existing transportation task; And (3) entering a processing stage, conveying the workpiece by the AGV, machining the workpiece, judging whether all the workpieces are machined and conveyed to a finished product bin, if not, repeating the steps, and if so, finishing the dispatching.
6. The method for real-time scheduling of a digital twin workshop for limited transportation resources and charging constraint scenes according to claim 1, wherein the method for designing the real-time status features in the step S3 comprises the steps of creating real-time production status features which comprehensively describe the production flow, wherein the production status features conform to (1) real-time acquisition, (2) comprehensive description of the production flow, (3) correlation with optimization targets; All state characteristic calculation formulas are as follows: Wherein, the Indicating an initial production state of the product, The balance weight representing the i-th state, Representing the normalized production state; the state information is acquired in real time through the sensor, and after the data layer is processed, a state vector is formed and is input to the dispatching intelligent body.
7. The method for real-time scheduling of digital twin workshops for limited transportation resources and charging constraint scenarios according to claim 1, wherein the method for designing an action space based on improved genetic programming in step S3 comprises the following steps: Initializing parameters; initializing a function set and a terminator set; Generating an initial population by adopting a mixing method, and calculating the fitness of each individual in the initial population; Selecting 10 individuals with the largest fitness value from the initial population to form a scheduling rule base; The method comprises the steps of selecting a population by using a method combining elite strategy and roulette strategy, selecting or not performing subtree crossing operation based on crossing rate, and updating the fitness value of each individual in the population based on node mutation operation or not performing mutation operation based on mutation rate selection; Introducing Srinivas self-adaptive genetic operators, and dynamically adjusting the crossing rate and the variation rate according to individual fitness in each iteration; finally, 10 IGP rules are generated to form an action space, and the scheduling agent selects one IGP rule to execute in the action space at each interaction point.
8. The method for real-time dispatching of digital twin workshops for limited transportation resources and charging constraint scenes according to claim 1, wherein the composite rewarding function in step S3 consists of main rewards and shaping items, and the design method comprises the following steps: The main rewards are related to minimizing the finishing time, the increment of each step of the final finishing time in the dispatching process is converted into negative rewards, and the intelligent agent is guided to learn a dispatching strategy capable of shortening the finishing time; the molding item is related to the average utilization rate of the machine, the machine is guided to keep busy, and the waiting time of the workpiece is reduced, so that the finishing time is shortened; the main rewards and the shaping items are weighted and summed by giving weight to form a single-step compound rewards function.
9. The digital twin shop real-time scheduling method for limited transportation resources and charging constraint scenes according to claim 1, wherein the training method of IAD3QN in step S4 comprises the following steps: based on D3QN, 3 extensions are introduced into ID3QN, namely, priority experience playback, soft target network updating strategy and self-adaptive exploration and utilization strategy; In the network structure of the ID3QN, a multi-head attention feature extraction layer is introduced to form an IAD3QN method, and the multi-head attention mechanism carries out self-adaptive weighting on the features in the scheduling environment and can dynamically adjust the attention degree of each feature, so that an intelligent agent pays more attention to the key features of the current decision moment; Initializing super parameters, network structures, total training times, online network and target network parameters and experience pools; initializing a digital twin simulation environment, wherein the environment comprises a machine fault and a workpiece insertion disturbance event; Performing multi-round training in a digital twin simulation environment, inputting the current workshop state into a multi-head attention feature extraction layer if the current time point is an interaction point, capturing multi-level features of input data, then selecting actions and executing, obtaining the next state and rewards from the simulation environment, and storing the state, the actions, the rewards and the next state into an experience pool; after the capacity of the experience pool is full, a priority experience playback strategy is adopted to adjust the sampling probability of the samples according to the time sequence difference error, small batches of samples are extracted to calculate a loss function, and online network parameters are updated; updating the target network parameters according to the soft target network updating strategy; Judging whether the iteration is finished or not, if so, finishing the current round of training, and if not, repeatedly executing the steps; Judging whether the training times reach the total training times, if not, updating the super parameters and starting the next training round, and if so, finishing the training; After training, a network model with better performance is saved as a scheduling agent model of online application.
10. The digital twin workshop real-time scheduling device for the limited transportation resource and charging constraint scene is characterized by comprising a processor and a storage medium; The storage medium is used for storing instructions; the processor is configured to operate in accordance with the instructions to perform the method of any one of claims 1-9.

Description

Digital twin workshop real-time scheduling method and device oriented to limited transportation resources and charging constraint scene Technical Field The invention relates to the technical field of intelligent manufacturing, in particular to a digital twin workshop real-time scheduling method and device for limited transportation resources and charging constraint scenes. Background With the increasing demand for mass customization and personalization, manufacturing industry is facing a shift from mass production of a single variety to mass customization mode, and to meet customer demands, enterprises are urgently required to increase the flexibility level and response speed of manufacturing systems. Under the background, an AGV has become a key resource for workshop material circulation, and how to realize efficient cooperation of production resources and transportation resources is a core problem to be solved in the intelligent manufacturing field. However, the AGV acts as an electrically driven device whose power consumption and charging behavior directly affect its availability, so the combined effects of transportation resource constraints, charging constraints, and disturbance events should be fully accounted for in building the dispatch model. Deep reinforcement learning provides a new method for solving the complex scheduling problem, but the scheduling agent based on DRL needs a great deal of training to be applied to a real workshop, and the training cost in a physical workshop is high and the efficiency is low, so that providing a training environment with high fidelity, safety and low cost for the agent is a key for realizing the engineering of the method. The digital twin proposal provides a new technical approach for solving the problem, and the digital twin technology can establish a virtual workshop which is highly consistent with a physical workshop, thereby providing a training platform with low cost and high fidelity for dispatching intelligent agents. Therefore, in solving the problem of dynamic real-time scheduling of the flexible job shop considering limited transportation resources and charging constraints, the construction of the real-time scheduling method based on digital twin and deep reinforcement learning has important research significance and engineering value. Disclosure of Invention Aiming at the defects or improvement demands of the prior art, the invention provides the digital twin workshop real-time scheduling method and device for the limited transportation resources and the charging constraint scene, which can realize the self-adaptive collaborative optimization of production and transportation resources under the condition of considering the limited transportation resources and the charging constraint, has good performance for flexible job workshops with different scales and different disturbance levels, effectively reduces the maximum finishing time and improves the feasibility and the robustness of a scheduling scheme in an actual workshop. To achieve the first object, according to one aspect of the present invention, there is provided a digital twin shop real-time scheduling method for limited transportation resources and charging constraint scenarios, comprising the steps of: S1, constructing a real-time scheduling frame based on digital twin and deep reinforcement learning; S2, on the basis of the step S1, a two-stage real-time scheduling model based on deep reinforcement learning is provided; S3, based on the two-stage real-time scheduling model of deep reinforcement learning proposed in the step S2, modeling a flexible job shop dynamic scheduling problem under the constraint of limited transportation resources and charging as a Markov decision process with the aim of minimizing the working time, and sequentially designing five key elements of an interaction point, a real-time scheduling flow, a real-time state characteristic, an action space based on improved genetic programming and a compound rewarding function of the scheduling model; S4, based on the step S3, providing a training method using IAD3QN to enable the scheduling agent to have feature extraction capability and dynamic decision capability, generating training cases with different scales and different disturbance levels in a digital twin simulation model, training the provided IAD3QN network, continuously updating IAD3QN network parameters, improving scheduling performance and generalization capability of the IAD3QN network, and obtaining the scheduling agent applicable to actual production decision; And S5, after training in the step S4 is completed, selecting an online application model of the scheduling agent, and when a new order arrives at the workshop, verifying and outputting a scheduling scheme by using the trained IAD3QN model according to the real-time workshop state of each interaction point by the scheduling agent to guide the physical workshop to complete the cooperative allocation of w