CN-121996970-A - Expressway service area inspection method and system

Abstract

The invention discloses a method and a system for inspecting a highway service area, and relates to the technical field of intelligent traffic and intelligent service area management. The method comprises the steps of obtaining multi-mode real-time sensing data of a highway service area, fusing the multi-mode real-time sensing data of video, internet of things equipment and the like, extracting semantic, density and track time sequence characteristics, constructing a service area dynamic space-time heterogeneous map taking areas, facilities and targets as nodes and taking space, logic and behavior relations as connecting edges, constructing a multi-mode time sequence map convolution network model, respectively extracting time evolution characteristics and space coupling characteristics of the node characteristics through a time convolution layer, a space map attention layer and a space-time interaction layer cascading structure, fusing the time evolution characteristics and the space coupling characteristics to generate space-time fusion characteristics, and carrying out abnormal event identification, facility fault prediction and risk propagation simulation based on the characteristics to generate an inspection path. The invention realizes the perception and deduction of the full-element and multidimensional situation of the service area, and effectively improves the inspection efficiency and the intelligent level.

Inventors

• YANG YI
• WANG WEIYI
• ZHANG YANG
• HU DI
• ZHANG JIANPENG
• ZHANG LIN
• QI KAI
• LIN CHANG
• MIN ZEYU
• DU DONGFANG
• SUN QUAN
• DAI LEI
• Qiao Shanjie
• MA JIANHUA
• WANG HUAZE
• LI LIN
• SUN FEIYA
• LIU XUN
• WANG KEHAN
• ZHANG YAXING
• WANG SHI
• LI YING
• LI WEIFENG
• SONG JIALI
• KONG SHANSHAN
• ZHAO WEI

Assignees

• 吉林省吉高服务区管理有限公司

Dates

Publication Date

20260508

Application Date

20260129

Claims (10)

1. 1.A method for patrolling a highway service area, comprising: The method comprises the steps of obtaining multi-mode real-time sensing data of a highway service area, wherein the multi-mode real-time sensing data comprise video monitoring data, vehicle detection data, personnel density data, equipment state data of the Internet of things and environment sensing data; The method comprises the steps of respectively extracting video semantic features, personnel and vehicle density features, internet of things equipment operation features and target track features of an expressway service area from multimode real-time data and splicing the video semantic features, personnel and vehicle density features, internet of things equipment operation features and target track features of the expressway service area to obtain multimode time sequence feature vectors; Constructing a multi-modal time sequence diagram convolution network model based on a space-time diagram convolution network, wherein the multi-modal time sequence diagram convolution network model comprises a cascaded time convolution layer, a space diagram attention layer and a space-time interaction layer; Extracting time evolution features from node features of dynamic space-time heterogeneous images in a highway service area in a time convolution layer, extracting space coupling features among nodes of the dynamic space-time heterogeneous images in a space dimension based on the time evolution features in a space image attention layer, and carrying out weighted fusion on the time evolution features and the space coupling features in a space-time interaction layer to obtain space-time fusion features; And according to the time evolution characteristics, the space coupling characteristics and the space-time fusion characteristics, sequentially carrying out abnormal event judgment, facility fault prediction and risk propagation simulation on the expressway service area to determine an inspection path of the expressway service area so as to inspect the expressway service area.
2. 2. The method for inspecting a highway service area according to claim 1, wherein said obtaining multi-modal real-time perception data of the highway service area comprises: Video monitoring data, vehicle detection data, personnel density data, internet of things equipment state data and environment sensing data of a service area are collected in real time through video monitoring equipment, a vehicle behavior detection device, a personnel statistics camera, an Internet of things equipment sensor and environment sensing equipment in an expressway service area; And sequentially formatting, cleaning and time synchronization processing are carried out on the video monitoring data, the vehicle detection data, the personnel density data, the equipment state data of the Internet of things and the environment sensing data to obtain four-in-one multi-mode real-time sensing data of personnel, vehicles, facilities and environments.
3. 3. The method for inspecting a service area of an expressway according to claim 1, wherein the method for inspecting the service area of the expressway, the characteristics of density of people and vehicles, the characteristics of operation of equipment of the internet of things and the characteristics of target tracks are extracted from multi-mode real-time data respectively, comprises the following steps: Cutting the picture of each region in the video monitoring data to obtain a video monitoring picture of each region; extracting video semantic features corresponding to video monitoring pictures of each area by adopting a CNN model, wherein the video semantic features reflect the congestion, density and environmental change conditions of each area in a highway service area; Inputting the vehicle detection data and the personnel density data into a density model, and outputting the personnel density of each area, wherein the formula is as follows: ; in the formula, For the personnel density of each zone within the highway service area, Is a vehicle density map obtained by statistics of a target detection model, H, W represents pixel sizes of the density map D (x, y) in the vertical direction and the horizontal direction respectively, As the number of rows of the density map, Column number as density map; Carrying out standardized processing on the state data of the equipment of the Internet of things to obtain the operation characteristics of the equipment of the Internet of things; And respectively extracting the appearance and the historical displacement sequence of the person and the vehicle in the video monitoring data through the target detection model, respectively obtaining dynamic target appearance characteristics and target displacement characteristics, and splicing to obtain target track characteristics, wherein the dynamic target appearance characteristics are used for identifying the same person or the same vehicle across the camera, and the target displacement characteristics are used for representing the movement trend of the target person or the vehicle.
4. 4. The method for inspecting a highway service area according to claim 1, wherein the determining node characteristics of each node of the highway service area based on the multi-mode time sequence characteristic vector comprises: according to the personnel and vehicle density characteristics and the video semantic characteristics in the multi-mode time sequence characteristic vector, determining regional node characteristics, wherein the expression is as follows: ; in the formula, I is the index number of the regional node, And The personnel density characteristic and the vehicle density characteristic of the t-th time step in the multi-mode time sequence characteristic vector are respectively, As the average velocity of the region, In order to be a region congestion rate, Is a video semantic feature; according to the operation characteristics of the Internet of things equipment in the multi-mode time sequence characteristic vector, determining the characteristics of facility nodes, wherein the expression is as follows: ; in the formula, Is a facility node of the expressway service area, As a matter of the type of device, In order to be a location of the facility, Is a device state including full load rate and fault flags; the current, voltage, power, liquid level and temperature and humidity real-time observation values are obtained in the time step t; according to the target track characteristics in the multi-mode time sequence characteristic vector, determining dynamic target node characteristics, wherein the expression is as follows: ; in the formula, Is a dynamic target node of the highway service area, Is a target track feature in a multi-modal timing feature vector, wherein, Representing the dynamic target appearance characteristic of the t-th time step, Is a historical displacement characteristic of a plurality of time steps.
5. 5. The method for inspecting a service area of an expressway according to claim 1, wherein the constructing a connection edge for connecting nodes by a spatial proximity relationship, a logical subordinate relationship, a visual reachability relationship, and a movement behavior relationship between the nodes comprises: through the space adjacent relation among all nodes of the expressway service area, a distance attenuation function is constructed to determine a space adjacent edge, and the calculation formula of the edge weight of the space adjacent edge is as follows: ; in the formula, Is a node Sum node Edge weights of the spatial adjacent edges between, And Separate nodes Sum node Is defined by the spatial coordinates of (a), Is a control parameter of a spatial scale and is used for adjusting the distance attenuation speed; Through the logical subordinate relations among all facility nodes in the expressway service area, constructing logical subordinate edges, wherein the expression of the edge weight of the logical subordinate edges is as follows: ; in the formula, Edge weights for logical dependent edges; Through the migration relationship between the personnel and the vehicle among different cameras, namely the visual reachable relationship, a visual reachable edge is constructed, and the formula of the edge weight of the visual reachable edge is as follows: ; in the formula, For the edge weight of the visually reachable edge, Is a node Sum node The orientation included angle formed between the two is used for indicating whether the two are observed by the camera in the same direction; is an indication function; Is a node Sum node The distance between the two plates is set to be equal, Is the maximum visible distance of the camera; Through the movement behavior relation in the high-speed kilometer service area, namely the flow displacement track formed by the historical displacement of the vehicle and the personnel, a movement relation edge is constructed, and the calculation formula of the edge weight of the movement relation edge is as follows: ; in the formula, To move the edge weights of the edges of the relationship, Representing personnel or vehicle slave nodes To the node Is used for controlling the flow rate of the water, For personnel or vehicles To all nodes Is a sum of the flow amounts of (a); And carrying out weighted fusion on the edge weights of the space adjacent edge, the logic subordinate edge, the visual reachable edge and the movement relation edge to obtain the connecting edge for connecting each node and the edge weight of the connecting edge, wherein the formula is as follows: ; in the formula, In the case of the type of edge, Is a collection of types of edges, including all spatially contiguous edges, logically dependent edges, visually reachable edges and move-relation edges, Learning weights for various types of edges.
6. 6. The method for inspecting a service area of an expressway according to claim 1, wherein the constructing a dynamic space-time heterogeneous map of the service area of the expressway based on the node characteristics and the connection edges specifically comprises: At any time step, according to node characteristics and connecting edges, a dynamic space-time heterogeneous diagram of the expressway service area is constructed, and the formula is as follows: ; in the formula, The node characteristics of facility nodes, regional nodes and dynamic target nodes of the expressway service area are included as a node set; Is an edge set; mapping for node types, which is used for defining the category of each node; mapping for edge types, which is used for defining semantic types of edges; As a set of side weights, step by step in time Dynamically changing.
7. 7. The method for inspecting a service area of an expressway according to claim 1, wherein the extracting the time evolution feature from the node features of the dynamic space-time heterogeneous map of the service area of the expressway specifically comprises: carrying out space-time convolution on the node characteristics to obtain time evolution characteristics of the expressway service area, wherein the formula is as follows: ; in the formula, Is the time evolution characteristic of the ith node of the expressway service area at the t-th time step, The activation function is represented as a function of the activation, 、 Is a convolution kernel of two groups of one-dimensional time sequences, For the multiplication on an element-by-element basis, Is a node feature.
8. 8. The method for inspecting a service area of an expressway according to claim 1, wherein the extracting spatial coupling features between nodes of the dynamic space-time heterogeneous map from a spatial dimension based on the time evolution features specifically comprises: The attention score between every two nodes is calculated, and the formula is: ; in the formula, To the node at the t-th time step Sum node An attention score of the person in between, Representing a transposed form of a trainable weight vector in an attention mechanism for weighting the spliced multidimensional feature; The characteristic linear transformation matrix is represented by a matrix, Is the time evolution characteristic of the ith node of the expressway service area at the t-th time step, Is the time evolution characteristic of the jth node of the expressway service area at the t time step, Is a node And node The relation embedding vector is used for describing the relation type or the relation attribute between the two nodes; According to the attention score between the two nodes, determining the attention weight between the two nodes, wherein the formula is as follows: ; in the formula, To the node at the t-th time step Sum node The weight of the attention between them, To the node at the t-th time step Sum node An attention score of the person in between, To the node at the t-th time step Sum node An attention score of the person in between, A node set of dynamic space-time heterogeneous diagram of the expressway service area; according to the attention weight between every two nodes, carrying out neighbor aggregation on each node to obtain the spatial coupling characteristic of each node, wherein the formula is as follows: ; in the formula, To the node at the t-th time step Is provided with a spatial coupling characteristic of (a), The linear transformation matrix of the space features is represented and is a trainable parameter and is used for carrying out linear mapping on the time evolution features of the neighbor nodes.
9. 9. The method for inspecting a service area of an expressway according to claim 1, wherein the time evolution feature and the spatial coupling feature are weighted and fused to obtain a space-time fusion feature, and the method specifically comprises: According to the time evolution characteristics of different nodes and different time steps in the expressway service area, the cross-time cross-node attention is determined, and the formula is as follows: ; in the formula, Serving area node for expressway Sum node The cross-time cross-node attention from the kth time step to the kth time step, In a transposed form of trainable weight vectors in a cross-time attention mechanism, for weighted mapping of spliced feature vectors, A linear transformation matrix that characterizes the current time step node, A linear transformation matrix that characterizes the current time step node, Is the time evolution characteristic of the ith node of the expressway service area at the t-th time step, The time evolution characteristic of the jth node in the t-k time step is provided, wherein t is greater than k; embedding vectors for representing time spans; Normalizing the attention of the cross-time cross-node, wherein the formula is as follows: ; in the formula, The normalized cross-time cross-node attention; the method comprises the steps of setting the time-span and node-span attention of nodes in expressway service areas from the kth time step to the t time step; And weighting and fusing the time evolution characteristics of all the time steps of the expressway service area with the space coupling characteristics of the current time step to obtain space-time fusion characteristics, wherein the formula is as follows: ; in the formula, Serving area node for expressway The spatiotemporal fusion feature at time step t, Is a node Spatial coupling features at the t-th time step, In order to output the characteristic linear transformation matrix, And the fusion weight coefficient is the fusion weight coefficient of the spatial characteristic and the temporal characteristic.
10. 10. The method for inspecting a service area of an expressway according to claim 1, wherein the method for inspecting a service area of an expressway based on space-time fusion features sequentially comprises the steps of: according to the time evolution characteristics of the expressway service area, carrying out abnormal time identification on the expressway service area to obtain an abnormal probability formula as follows: ; in the formula, A predictive probability vector representing an abnormal event occurring in the highway service area at ttt time step, A weight matrix of classifiers is identified for the anomaly event, Is a time evolution characteristic of the expressway service area, Identifying a weight matrix of a classifier for the abnormal event; () Representing a Softmax normalization function for mapping the model output to a probability distribution form; according to the space-time coupling characteristics of the expressway service area, the Internet of things equipment in the expressway service area is subjected to fault prediction to obtain the future fault probability, wherein the formula is as follows: ; in the formula, For the future The state of the equipment of the Internet of things at the moment, Serving area node for expressway The spatiotemporal fusion feature at time step t, For future time span The predictive decoding function of (2) is used for mapping the current space-time fusion characteristic into a future equipment state; According to the space-time coupling characteristics of different nodes in the expressway service area and the weights of connecting edges among different nodes, determining a propagation coefficient for risk propagation simulation, wherein the formula is as follows: ; in the formula, As a propagation coefficient for the risk propagation simulation, In order to activate the function, Is as the parameter of For comprehensively considering node characteristics and relationships between nodes, And Respectively nodes Sum node Is characterized by the space-time coupling characteristics of (a), Is a node Sum node Weights of the connecting edges; according to the propagation coefficient for risk propagation simulation, determining a risk evolution equation, and performing risk propagation simulation on the expressway service area to obtain risk diffusion probability, wherein the equation is as follows: ; ; in the formula, Is the first The next time step of each node Is a function of the risk exposure probability of (a), Is the first The next time step of each node Is a function of the risk exposure probability of (a), Serving the expressway with the first The individual node is at the first Probability values that the individual time steps are in a risk exposure state, To represent nodes In the first place Probability values for the time steps in the risk activated state, To represent nodes In the first place Probability values for the time steps in the risk activated state, To represent nodes In the first place The probability value of each time step in the risk susceptibility state is used for describing the sensitivity degree of the node to the risk; To represent the transition coefficient of risk from the exposed state to the activated state, is a non-negative parameter, To express the first The next time step of each node Is a function of the risk exposure probability of (a), To express the first in the expressway service area The individual node is at the first The probability value that the individual time steps are in a risk activated or risk occurring state, For attenuation or recovery coefficients representing risk activation states; determining node risk scores of the expressway service areas according to the anomaly probability, the future fault probability and the risk diffusion probability, wherein the formula is as follows: ; in the formula, Serving area node for expressway The risk score at time t is calculated, 、 And The weight coefficients of the abnormal event risk, the facility fault risk and the risk propagation result in the comprehensive risk score are used for adjusting the relative influence degrees of different risk sources; According to the risk scores of all nodes in the expressway service area, determining key nodes for inspection so as to determine an inspection path, wherein the formula is as follows: ; in the formula, For the entire set of nodes of the highway service area, Representing a patrol path P Is used for the inspection of the cost function of (a), And constraining a threshold value for the maximum patrol cost allowed for limiting the resource consumption of the patrol path in actual execution.

Description

Expressway service area inspection method and system Technical Field The application relates to the technical field of intelligent traffic and intelligent service area management, in particular to a method and a system for inspecting a highway service area. Background With the continuous increase of the traffic flow of the expressway, the service area is changed from the traditional vehicle supply facility to a comprehensive traffic service node integrating multiple functions of parking, refueling (charging), catering, shopping, resting, life service and the like. The personnel density, the vehicle density, the equipment types and the running states in the service area present high dynamics and complexity, so that the safety control and the running management face a plurality of problems. In the prior art, with the development of technologies such as deep learning, graphic neural network, video analysis and IoT perception, the space-time diagram convolutional network processes multi-modal heterogeneous data with a unified diagram structure, so that effective prediction can be performed on scenes such as traffic flow and pedestrian track. However, due to the fact that the expressway service area is distributed in structure and strong in element heterogeneity, the service area comprises regional nodes (parking areas, business areas, dining areas, sanitary areas and the like), facility nodes (charging piles, garbage cans, lighting equipment and the like) and dynamic target nodes (vehicles and pedestrians), the elements are different in types, interaction relations change with time, the prior art adopts a unified graph structure modeling mode, the complex space-time structure cannot be accurately described, real-time prediction of multiple scenes and multiple elements cannot be achieved in the service area, and potential safety hazards or equipment abnormality under the coupling of the multiple scenes and elements cannot be timely found by inspection staff, so that the safe operation of the expressway service area is affected. Disclosure of Invention Accordingly, it is necessary to provide a method and a system for inspecting a service area of an expressway in order to solve the above-mentioned problems. The invention adopts the following technical scheme: the invention provides a highway service area inspection method, which comprises the following steps: The invention provides a computer readable storage medium, wherein the storage medium stores a computer program which is executed by a processor to realize the method for inspecting the expressway service area. The invention provides a computer device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the method for inspecting the service area of the expressway when executing the program. The at least one technical scheme adopted by the invention can achieve the following beneficial effects: according to the method for inspecting the expressway service area, disclosed by the invention, the multi-mode real-time data such as video monitoring, vehicle detection, personnel density, equipment state of the Internet of things and environment perception are integrated, the dynamic space-time heterogeneous diagram taking the area, facilities and dynamic targets as nodes and taking the space proximity, logic dependence and visual accessibility and movement behaviors as connecting sides is constructed, the structural representation of the complex personnel-vehicle facility coupling scene of the service area is realized, the multi-mode time sequence diagram convolution network model is constructed on the basis, the time evolution rule of the node characteristics is extracted through the time convolution layer, the spatial dependency relationship among the nodes is captured through the space diagram attention layer, the cross-time and cross-node characteristic fusion is realized through the space-time interaction layer, a solid foundation is provided for accurate reasoning and advanced prediction of congestion tendency, personnel aggregation, vehicle abnormality and facility fault, the technical problem of insufficient analysis of the multi-element coupling scene in the prior art is overcome, the occurrence rate of safety events and equipment burst fault frequency are remarkably reduced, and the safety management efficiency and intelligent operation level of the expressway service area are effectively improved. In addition, the invention dynamically generates the inspection priority and the optimal path based on the results of anomaly identification, fault prediction and risk propagation simulation, realizes the transition from passive response to active prediction and from experience inspection to intelligent scheduling, and remarkably improves the real-time performance of service area safety management. Drawings The accompanying drawings, which are included to provide