CN-117334060-B - Strategic gradient double-intersection traffic signal control method based on importance sampling

CN117334060BCN 117334060 BCN117334060 BCN 117334060BCN-117334060-B

Abstract

A strategic gradient double-intersection traffic signal control method based on importance sampling comprises the following steps of S1, collecting given double-intersection physical characteristic information, traffic flow information and signal lamp phase information, building a simulation model of a traffic simulation platform by using microscopic simulation software, S2, building a deep neural network model based on the built simulation model, training distribution pi θ of signal lamp phase control strategies based on strategic gradient principles of importance sampling, updating parameters theta of the neural network, and S3, obtaining road network signal lamp phase a k+1 needed to be executed at the next moment of the double-intersection according to state information of a current intersection and signal lamp phase information S k ,a k of the deep neural network obtained through training. The invention can lead the signal control to be faster and trained to obtain better performance, and can be more rapidly used for relieving traffic jam in road network control.

Inventors

Zhong Huijian
FAN XIAOHONG
LIN JING
LI ZHIQIANG
CAO HONGXIA
PENG JIYOU
FENG YUANJING
LI YONGQIANG

Assignees

航天科工广信智能技术有限公司

Dates

Publication Date: 20260508
Application Date: 20230914

Claims (2)

1. The strategic gradient double-intersection traffic signal control method based on importance sampling is characterized by comprising the following steps: S1, collecting given physical characteristic information of double intersections, traffic flow information and signal lamp phase information, and constructing a simulation model of a traffic simulation platform by using microscopic simulation software; S2, building a deep neural network model based on a built simulation model, training distribution of signal lamp phase control strategies based on strategy gradient principles of importance sampling And update parameters of the neural network ; S3, according to the deep neural network obtained through training, according to the state information of the current intersection and the signal lamp phase information Obtaining the phase of the road network signal lamp to be executed at the next moment of the double intersection The method comprises the following steps: ; In the step S2, parameters of the deep neural network are updated The steps of (a) are as follows: S21, setting the maximum iteration times Initializing deep neural network parameters The neural network structure carries out self-adaptive adjustment according to the size of the double intersections, and super parameters are set Discount factor ; S22, setting According to the initial neural network Sampling from simulation software to obtain a first intersection traffic flow state and a signal lamp action set And calculate Wherein ; Wherein the method comprises the steps of Refers to a state action pair A corresponding bonus function is provided that is based on the received signal, Is a constant of a reference line, which is a constant of a reference line, , The number of state action pairs contained in the track is represented; S23, setting Sampling through a microscopic simulation model to obtain a vehicle state action set track And calculate Wherein importance samples weights Calculated by the following formula: ; s24, calculating Updating ; S25, repeatedly executing the steps S23 and S24 until the maximum iteration number Until that is reached; s26, outputting the latest neural network parameters 。
2. The method for controlling traffic signals at strategic gradient double intersections based on importance sampling according to claim 1, wherein in the step S1, the road network vehicle traffic information data acquisition steps are as follows: s11, according to the time of data collection Acquiring current traffic flow information and signal lamp information ; S12, for a given double-intersection road network, acquiring traffic flow distribution data of an inlet according to acquired information statistics; S13, constructing a microscopic simulation model in microscopic simulation software based on traffic flow data of an entrance and physical characteristics of a double intersection, including signal lamp positions, lane distribution and lane lengths, and simulating a daily driving state of the intersection.

Description

Strategic gradient double-intersection traffic signal control method based on importance sampling Technical Field The invention belongs to the field of traffic control, and relates to a strategic gradient double-intersection traffic signal control method based on importance sampling. Background With the rapid increase of the total quantity of resident automobiles in China, the current city is generally faced with the problem of traffic jam. Meanwhile, compared with improving road network traffic facilities, the traffic timing scheme for optimizing the intersections has quicker effect and better economic benefit. The traditional signal lamp control method adopts a fixed accessory scheme, and the method cannot cope with the characteristics of strong real-time performance and rapid traffic flow change of the urban road network. The current real-time traffic timing scheme is a relatively popular research direction. Disclosure of Invention In order to overcome the defects of the prior art, the invention provides a strategic gradient double-intersection traffic signal control method based on importance sampling, a microscopic simulation model is built for training a deep neural network by collecting information of intersections, and the traffic jam condition can be effectively relieved by a neural network controller obtained by training the strategic gradient double-intersection traffic signal control method based on importance sampling. The technical scheme adopted for solving the technical problems is as follows: a strategic gradient double-intersection traffic signal control method based on importance sampling comprises the following steps: S1, collecting given physical characteristic information of double intersections, traffic flow information and signal lamp phase information, and constructing a simulation model of a traffic simulation platform by using microscopic simulation software; S2, building a deep neural network model based on a built simulation model, training distribution of signal lamp phase control strategies based on strategy gradient principles of importance sampling And update parameters of the neural network; S3, according to the deep neural network obtained by training, the state information of the current intersection and the signal lamp phase information can be obtainedObtaining the phase of the road network signal lamp to be executed at the next moment of the double intersectionThe method comprises the following steps: 。 Further, in the step S2, parameters of the deep neural network are updated The steps of (a) are as follows: S21, setting the maximum iteration times Initializing deep neural network parametersThe neural network structure carries out self-adaptive adjustment according to the size of the double intersections, and super parameters are setDiscount factor; S22, settingAccording to the initial neural networkSampling from simulation software to obtain a first intersection traffic flow state and a signal lamp action setAnd calculateWherein ; Wherein the method comprises the steps ofRefers to a state action pairA corresponding bonus function is provided that is based on the received signal,Is a constant of a reference line, which is a constant of a reference line,,The number of state action pairs contained in the track is represented; S23, setting Sampling through a microscopic simulation model to obtain a vehicle state action set trackAnd calculateWherein importance samples weightsCalculated by the following formula: ; s24, calculating Updating; S25, repeatedly executing the steps S23 and S24 until the maximum iteration numberUntil that is reached; s26, outputting the latest neural network parameters 。 Still further, in the step S1, the road network vehicle traffic information data acquiring step is as follows: s11, according to the time of data collection Acquiring current traffic flow information and signal lamp information; S12, for a given double-intersection road network, acquiring traffic flow distribution data of an inlet according to acquired information statistics; S13, constructing a microscopic simulation model in microscopic simulation software based on traffic flow data of an entrance and physical characteristics of a double intersection, including signal lamp positions, lane distribution and lane lengths, and simulating a daily driving state of the intersection. The method has the advantages that the signal control can be trained faster to obtain better performance, and the traffic jam can be relieved more quickly when the method is applied to road network control. Drawings FIG. 1 is a flow chart of the importance sampling neural network training of the present invention. Fig. 2 is a schematic diagram of a road network with double intersections according to the present invention. Detailed Description The invention is further described below with reference to the accompanying drawings. Referring to fig. 1 and 2, a strategic gradient dual intersection traffic signal