Search

CN-116468987-B - Robust escalator passenger flow statistics system adapting to different hardware multiple scenes

CN116468987BCN 116468987 BCN116468987 BCN 116468987BCN-116468987-B

Abstract

The invention discloses a robust escalator passenger flow statistical system adapting to different hardware multiple scenes, which can adaptively select an optimal detection model and a tracking model which are suitable for the hardware platform and an application scene according to the calculation power of the different hardware platforms and the pedestrian motion information under the different escalator scenes, and uses an improved tracking model to promote the tracking effect, uses a software acceleration algorithm to accelerate the detection model, uses an improved quantization algorithm to promote the model reasoning speed and reduce the parameter quantity of the model, then adaptively determines the position and the size of an ROI (region of interest) for counting passenger flow, counts the number of people passing through the ROI of an escalator entrance, and proposes algorithms such as pedestrian detection weight removal, pedestrian number correction and the like to promote the accuracy of the number of people counting. The method can be suitable for hardware platforms with different calculation forces, is suitable for different staircase scenes, has the effect of improving model tracking, improves model detection speed, and has good robustness for different hardware and application scenes.

Inventors

  • DU QILIANG
  • Huang Guangji
  • TIAN LIANFANG

Assignees

  • 华南理工大学

Dates

Publication Date
20260505
Application Date
20230322

Claims (7)

  1. 1. The robust escalator passenger flow statistical system adapting to different hardware multiple scenes is characterized by comprising: The hardware detection and performance test module can acquire CPU information, memory information and GPU information of the application platform, test the calculation power of the hardware and the reasoning speed of the deep learning model, share the information and the tested data acquired by the hardware detection and performance test module by other modules in the system, and improve and optimize the corresponding algorithm according to the acquired information and data; The pedestrian information detection module is used for detecting pedestrian information of different scenes, including the intensity of the pedestrian flow, the motion direction of the pedestrian, the motion track of the pedestrian and the motion speed of the pedestrian in a video frame, adopting CrowdNet to estimate the crowd density so as to quickly locate a preliminary optimized ROI passenger flow statistical area, accurately detecting and tracking the pedestrian based on YOLOv and a tracking model, and acquiring the motion speed of the pedestrian according to the motion direction and the motion track; For the condition of the used tracking model, in order to overcome the problem that the original tracking model only uses IoU as a weight and cannot judge two disjoint problems and does not consider the shape of a detection frame and a tracking frame, an improved matrix weight calculation method for matching the detection frame and the tracking frame is provided, the method considers the overlapping degree of the detection frame and the tracking frame, the distance between the two frames and the size difference of the frames, and provides a scaling factor for adapting the attention degree of the shape of the detection frame and the tracking frame under multiple scenes, so that the tracking accuracy is improved; The detection model self-adaptive module adaptively selects an optimal model according to hardware information obtained by the hardware detection and performance test module, the inference speed of the hardware and the tracking model type determined by the tracking model self-adaptive module, and improves the existing corresponding model to match hardware equipment and improve the utilization rate of the hardware; the ROI area self-adaption module is used for determining a region with dense people flow as a preliminary passenger flow statistics ROI area according to the crowd density estimation of CrowdNet, determining the direction of the ROI area according to the walking direction of pedestrians, and determining the length and the width of the ROI area according to the reasoning speed of a detection model in the detection model self-adaption module; the pedestrian number counting module is used for detecting and tracking pedestrians passing through the ROI area according to the detection model and the tracking model after the ROI area is determined by the ROI area self-adaption module, and then counting the passenger flow, and for the situation that the tracking algorithm is not used in the tracking model self-adaption module, in order to increase the accuracy of the passenger flow statistics, a gray histogram comparison algorithm is used for comparing the similarity of the heads of the areas at the entrance of the ROI and the exit of the ROI so as to judge whether the same person is repeatedly counted, and in order to solve the problem of missed detection caused by shielding of the detection model when the passenger flow is crowded, a passenger number correction algorithm is provided for improving the accuracy of the passenger flow statistics.
  2. 2. The robust escalator passenger flow statistics system adapting to different hardware multiple scenarios according to claim 1, characterized in that the hardware detection and performance testing module comprises the following functions: a. the module integrates a series of hardware detection functions, can acquire CPU information of hardware, including the performance and the kernel number of the CPU, can detect whether the GPU exists, and acquires the model, architecture and video memory size information of the GPU; b. The method comprises the steps of respectively testing the operation speed of a CPU and the operation speed of a GPU based on a deep learning target detection method, wherein a detection model is YOLOv, for equipment containing the GPU, the convolution operation process runs on the GPU, otherwise, the convolution operation runs on the CPU, firstly, loading trained weight parameters, then inputting videos in different practical application scenes into the detection model, carrying out reasoning detection on each frame in the videos by the detection model, recording the detection time used by each frame, summing, and then solving the average detection time t det used by each frame.
  3. 3. The robust escalator passenger flow statistics system adapting to different hardware multiple scenes according to claim 2, wherein the pedestrian information detection module adapting to multiple scenes comprises the following functions: a. estimating crowd density in the image by using CrowdNet, and recording a region with the largest crowd density in the image, which is used as a preliminary selected region of passenger flow statistics and is marked as ROI pre ; b. the method comprises the steps of acquiring pedestrian motion information, detecting and tracking all pedestrians in a crowd-intensive region ROI pre selected through CrowdNet by using a detection model and a tracking model, acquiring a motion track and a motion direction of each person, calculating the motion speed of the pedestrians according to the motion track and the motion direction, recording a starting video frame in and a position (x in ,y in ) of each person just entering a floor plate, wherein x in and y in are respectively the coordinates in the height and width directions in the video frame in , recording the video frame out and the position (x out ,y out ) of each person appearing on the floor plate finally, wherein x out and y out are respectively the coordinates in the height and width directions in the video frame out , and then solving the linear distance dis of (x in ,y in ) and (x out ,y out ), wherein Then find the movement speed of the ith pedestrian Wherein i is E [1, pn ], pn represents the total number of pedestrians, and then the motion speed v 1 ,v 2 ,…,v pn of all pedestrians is calculated to obtain the average speed According to the average moving speed of the pedestrian detected by the pedestrian information detection module Θ i represents the running direction of the i-th pedestrian, the running directions of all pedestrians are obtained, θ 1 ,θ 2 ,...,θ pn , and the average running direction of the pedestrians is obtained
  4. 4. A robust escalator passenger flow statistics system adapted to different hardware multi-scenarios according to claim 3, characterized in that the tracking model adaptation module comprises the following functions: a. The model selection function of the tracking model comprises three different tracking models, wherein the tracking model comprises a TRACK no which indicates that any tracking model is not used, a TRACK re which indicates that the tracking model is based on correlation filtering, a TRACK kam which indicates that the tracking model is based on Kalman filtering, and the tracking model self-adaption process comprises the steps of firstly obtaining average detection time t det used by each frame obtained by a hardware detection and performance test module, setting a detection time threshold t thres according to different application scenes, and determining the type TRACKTYPE of the finally selected tracking model according to the following formula: b. In order to overcome the problem that the original algorithm only uses IoU as a weight and cannot judge two disjoint problems and does not consider the shapes of a detection frame and a tracking frame, an improved matrix weight calculation method for matching the detection frame and the tracking frame is provided, the method considers the overlapping degree of the detection frame and the tracking frame, the distance between the two frames and the size difference of the frames, and provides a scaling factor for adapting to the attention degree of the detection frame and the tracking frame in multiple scenes, and the matching degree of the detection frame D and the tracking frame T is defined as MATCH (D, T), and the calculation method is as follows: Wherein S D∩T and S D∪T are the areas of the intersection and union areas of D and T respectively, w D 、h D is the width and the height of the detection frame D, w T 、h T is the width and the height of the tracking frame T, L c is the straight line distance between the center points of the D and the T, L l is the diagonal length of the circumscribed rectangle of the D and the T, alpha is an adjustable factor, the value range interval is set to be 0,0.5, fine adjustment is carried out according to the actual application scene so as to reflect the importance degree of the scale in matching, and the value range of MATCH (D, T) is (-2, 1), and a formula is used for facilitating the processing of the subsequent matrix Normalization is performed to a value between (0, 1), Represents the matching weight of the detection frame D and the tracking frame T after normalization, and is finally used As a matching weight between the judgment detection frame D and the tracking frame T.
  5. 5. The robust escalator passenger flow statistics system adapted to different hardware multi-scenarios according to claim 4, wherein the detection model adaptation module comprises the following functions: a. The method has the function of self-adaptive type selection of a detection algorithm, and comprises two types of detection models which can be selected, namely a first-stage detection model DET one and a second-stage detection model DET two , wherein the first-stage detection model DET one is a YOLO series, CENTERNET or SSD, the second-stage detection model DET two is a fast R-CNN or a Cascade R-CNN, the detection model is selected in a self-adaptive mode according to the type of a tracking model, the detection model is selected if the tracking model is selected as TRACK no , the detection model is selected DET two , otherwise the detection model is selected as DET one , after the detection model is selected, a given video is tested by using the selected detection model, the detection time of each frame is obtained, and the final average detection time is calculated as t det_select ; b. the method for accelerating the software aiming at the detection model is provided, wherein the operation of the original non-maximum suppression process is executed on a CPU on a GPU hardware device, the operation is improved to be realized by cuda programming, the operation of the non-maximum suppression process is put on the GPU for execution, the effect of accelerating the detection is achieved; c. Firstly, carrying out asymmetric truncated mapping on weight parameters, and mapping an original 32-bit floating point precision weight parameter W o to an 8-bit integer weight parameter W p , wherein the asymmetric truncated mapping has the following formula: Wherein T n satisfies min (W o )≤-T n <0, T p satisfies 0<T p ≤max(W o , represents a intercept point of a negative value interval; W is an element in a weight parameter W o tensor before quantization, f (W) mapping operation is carried out on each element in W o to obtain each element in W p , f (W) is a function value after mapping operation, values of T n and T p are traversed under the constraint range of T n and T p are satisfied, activation values before network quantization and after asymmetric intercept mapping quantization are obtained, difference between activation value distribution is obtained by using KL divergence according to the following formula, and T n and T p corresponding to the smallest KL divergence value are found as final mapping intercept points; Where x represents discrete activation values before and after quantization, a o (x) and a p (x) represent probability distributions of network layer output activation values of weight parameters corresponding to before and after the asymmetric truncation mapping, respectively, and D KL (A p ||A o ) represents KL divergences of a o (x) and a p (x) to represent similarities of distributions therebetween.
  6. 6. The robust escalator passenger flow statistics system adapted to different hardware multi-scenarios according to claim 5, characterized in that the ROI area adaptation module comprises the following functions: the method comprises the steps of determining the direction and the size of a region of interest (ROI), determining the direction of the ROI finally used for passenger flow statistics along the width as theta according to the optimal region of a preliminary ROI pre , the average moving speed v of pedestrians and the average moving direction theta of pedestrians obtained by a pedestrian information detection module adapting to multiple scenes, wherein the width ROI w is the width of the ROI pre , and the height ROI h of the ROI is determined by the average moving speed v of pedestrians and the average detecting time t det_select obtained by a detection algorithm self-adapting module and is ROI h ＝0.9*v*t det_select .
  7. 7. The robust escalator passenger flow statistics system adapted to different hardware multi-scenarios according to claim 6, wherein the pedestrian number counting module comprises the following functions: a. The system comprises a tracking model self-adapting module, a detection model and a passenger flow statistics module, wherein the tracking model self-adapting module is used for providing two different improved passenger flow statistics algorithms according to whether the tracking model is used in the tracking model self-adapting module so as to adapt to the requirements of performances or tasks of different hardware; b. For the passenger flow statistical method without using a tracking model, a new algorithm for avoiding repeated detection of the same person is provided, wherein a detection frame near the region of the ROI, which is marked as B and is most likely to be the same as a detection frame near the region of the ROI, which is marked as A, is the same target, the algorithm determines whether the detection frame is the same target or not by comparing the gray level histogram between the two detection frames and the similarity of the shape, d (A, B) represents the difference between the two detection frames, the closer the difference is to 0, the more similar the two targets are, the more likely the two targets are the same target, and a specific calculation formula is as follows: wherein H A (i) and H B (i) are normalized histograms of corresponding images of detection frames A and B respectively, N represents the number of bar columns of the histogram, i epsilon [1, N ] represents one bar column of the histogram, w A and H A are the width and the height of the detection frame A respectively, w B and H B are the width and the height of the detection frame B respectively, alpha is the weight of the difference of the width of the two frames, beta is the weight of the difference of the height of the two frames, then the calculation result of the above formula is compared with a set threshold d thres , if d thres is larger than d thres , the detection frame B and the detection frame A of the previous frame are not repeatedly counted, the detection frame B is included in the passenger flow statistics, and if d thres is smaller than d thres , the detection frame B of the current frame B and the detection frame A of the previous frame are the same target, so the detection frame B is not included in the passenger flow statistics; c. For the passenger flow statistical method without using a tracking model, in order to correct the problem of missed detection of the model caused by congestion shielding, a passenger flow statistical correction algorithm based on the passenger flow density is provided, the thought of the passenger flow statistical correction algorithm is that the denser the passenger flow is, the more serious the shielding condition is, the larger the probability of model missed detection is, so that the number of people counted by the model is less than the number of real people, the algorithm defines the dense representation of the passenger flow in the ROI region by the method of the area occupation ratio of a detection frame, and then a correction algorithm is provided according to the dense condition, and the specific formula is as follows: Wherein R is used for representing the dense condition of the human flow in the ROI area, S bbox (j) is used for representing the area of the j-th detection frame, N p is used for representing the total number of people detected in the video frame, j epsilon [1, N p ] is used for representing any one person detected in the video frame, S ROI is used for representing the area of the ROI area, f (R) is used for representing the number of people needing to be corrected and increased, and the final statistical number detected by the current video frame is N p +f (R).

Description

Robust escalator passenger flow statistics system adapting to different hardware multiple scenes Technical Field The invention relates to the technical field of escalator security, in particular to a robust escalator passenger flow statistical system suitable for different hardware multiple scenes. Background In modern society, various high-rise buildings are continuously appeared, and in order to facilitate people to reach different floors, the escalator becomes an indispensable device, and compared with a vertical elevator, the escalator has the advantage of large passenger capacity, and is widely applied to various occasions, in particular to places with large passenger capacity such as markets, subways and the like. The traffic flow of the mall can reflect the operation quality of the mall, and has a certain commercial value reference, and the traffic flow statistics of the entrance and exit of the escalator at the subway entrance can effectively realize the monitoring of public transportation and the prevention of safety problems, so the traffic flow statistics in the scene of the escalator has important significance. The passenger flow statistics method is based on calculation vision, the passenger on the escalator floor plate is detected and tracked, the track position information of the passenger is obtained, the passenger flow statistics is carried out, and the detection and tracking combination method is an accurate passenger flow statistics method. However, in order to achieve the tracking of pedestrians, the detection speed must be faster, so that the matching between the detected target and the tracked target can be achieved, if the detection speed is too slow, the tracking of the target fails, so that the track position information of the passengers cannot be obtained effectively, and the corresponding effect cannot be obtained by the corresponding passenger flow statistical algorithm. In practical application, the AI edge products of different manufacturers have different computing power of AI chips, and the AI hardware platform with higher computing power can obtain better effect, but has relatively high price. The requirements on the accuracy of passenger flow statistics are different for different application scenes, and the selected AI hardware platforms are also different. Therefore, aiming at different hardware platforms, different application scenes and different task precision requirements, different detection algorithms or tracking algorithms need to be selected, and the problem that the same algorithm has poor effect or even cannot be used on different hardware platforms is often caused, so that a great challenge is brought to the application of AI projects on the floor. For passenger flow statistics of the escalator, only a floor area of an entrance and an exit of the escalator is needed to be selected, and the size of the floor area is generally selected by manual framing. However, the size of the ROI manually framed by the operator is not necessarily a preferred choice, and how to adaptively select the most suitable ROI region according to the computational power of the hardware is also a problem to be solved. In addition, for hardware with poor hardware performance or complicated staircase scenes, the detection speed and the accuracy of people counting are required to be improved, so that a robust passenger flow counting system capable of adapting to different hardware multiple scenes is required. Disclosure of Invention The invention aims to overcome the defects and shortcomings of the prior art, and provides a robust escalator passenger flow statistics system adapting to different hardware multiple scenes, which can adaptively select an optimal detection model and a tracking model which are suitable for the hardware platform and an application scene according to calculation power of the different hardware platforms and pedestrian motion information under the different escalator scenes, and use an improved tracking model to promote the tracking effect, and use a software acceleration algorithm to accelerate the detection model, and use an improved quantization algorithm to promote the model reasoning speed and reduce the parameter quantity of the model, then adaptively determine the position and the size of an ROI (region of) for counting passenger flow, then count the number of people passing through the ROI region of a escalator entrance, and propose algorithms such as pedestrian detection weight removal, pedestrian number correction and the like to promote the accuracy of people counting. In order to achieve the purpose, the technical scheme provided by the invention is that the robust escalator passenger flow statistical system adapting to different hardware multiple scenes comprises: The hardware detection and performance test module can acquire CPU information, memory information and GPU information of the application platform, test the calculation power of the hardware a