CN-116740659-B - 360-Degree monitoring vehicle re-identification method based on infrared imaging

CN116740659BCN 116740659 BCN116740659 BCN 116740659BCN-116740659-B

Abstract

The invention belongs to the technical field of vehicle re-identification, and relates to a 360-degree monitoring vehicle re-identification method based on infrared imaging, which comprises the steps of firstly constructing a city 360-degree infrared monitoring vehicle re-identification video data set, extracting features, then carrying out 360-degree vehicle detection based on spherical convolution, carrying out secondary projection on a detection result, then carrying out double-flow vehicle re-detection feature extraction based on pseudo-infrared monitoring images generated by adaptation to a reactive domain, finally, the two vehicle characteristics are weighted by adopting a mode of an attention mechanism, and a vehicle re-identification result is output, so that the intelligent re-identification of the vehicle can be realized, the vehicle re-identification monitoring range is widened, the infrared data can be introduced in a multi-mode form, the accuracy of the vehicle re-identification is ensured, and meanwhile, the problem of the accuracy reduction of a vehicle re-identification algorithm in a complex scene is solved.

Inventors

LIU HANSONG
WANG YONG
WANG GUOQIANG
LIU RUI
Tan Liansheng

Assignees

松立控股集团股份有限公司

Dates

Publication Date: 20260505
Application Date: 20230703

Claims (8)

1. A360-degree monitoring vehicle re-identification method based on infrared imaging is characterized by comprising the following specific steps of: (1) Collecting a 360-degree vehicle detection data set, and generating pseudo infrared monitoring data as a 360-degree infrared monitoring vehicle re-identification video data set by adopting a domain self-adaptive mode from the 360-degree vehicle detection data set; (2) Carrying out ERP projection on the video data set in the step (1) to obtain an ERP image, sampling spherical pixels into plane convolution based on a sphere sampling strategy of distortion invariance, and extracting 360-degree infrared monitoring vehicle re-identification video features by adopting a CNN network; (3) The processing mode of 360-degree video based on spherical convolution obtains vehicle position information and width and height of the vehicle, and the specific process is as follows: (3-1) inputting the ERP image monitored by 360 degrees into a convolution layer adopted by the CNN network in the step (2), enhancing vehicle characteristic constraint by introducing ConvBlock, outputting spherical coordinate position information after a plurality of ConvBlock, projecting based on the spherical coordinate position information, and restricting a peripheral boundary frame generated by projection by manual marking, wherein the method is in detail operated as follows, , Wherein, the Represents a 360-degree infrared monitoring index ERP picture, Representative will be Packaging in a form similar to ResBlock; Representative of Is the number of (3); Representing the position coordinates of the vehicle on the sphere; representing the width and height of a peripheral bounding box of the vehicle after projection; (3-2) after obtaining the position coordinates and the width and height information of the vehicle, calculating the loss between the position of the vehicle position on the spherical surface and the manual annotation by using a position loss function, , Wherein, the A position loss function representing spherical coordinates, Representing predicted spherical coordinate position information, Is the position coordinate information of the manual annotation, The loss of width and height of the peripheral bounding box is based on The loss between the peripheral bounding boxes w and h of the 2D vehicle image generated after projection and the artificially noted w and h, , Wherein, the Representing the loss function after projection; representing the predicted width and height information, Representing the manually marked width and height information; (4) According to the vehicle position information obtained in the step (3) and the width and height of the vehicle, the vehicle monitored by the spherical surface is projected into a 2D vehicle image in a projection mode, and the vehicle is cut through the width and the height to obtain a refined vehicle block; (5) Generating a pseudo infrared monitoring image by adopting a pseudo infrared monitoring image generation mode based on the adaptation to the anti-domain based on the refined vehicle block obtained in the step (4); (6) Inputting the refined vehicle block obtained in the step (4) and the pseudo infrared monitoring image obtained in the step (5) into a double-flow ResNet network for feature extraction to obtain vehicle features based on RGB and NIR data respectively, and introducing The loss function improves the generation quality of the pseudo infrared monitoring image; (7) Weighting the vehicle characteristics based on the RGB and NIR data obtained in the step (6) by adopting a mode of an attention mechanism, and outputting a vehicle re-identification result; (8) The network is trained and tested, wherein in the testing stage of the network, after the vehicle position and the size of the clipping area are positioned, the similarity measure between different vehicle characteristics based on RGB and NIR is calculated, so as to judge whether the current vehicle is the searched vehicle or not.
2. The method for identifying the 360-degree monitoring vehicle weight based on infrared imaging according to claim 1, wherein the 360-degree vehicle detection data set in the step (1) adopts VeRi-776, vehicleID and VERI-Wild data sets, vehicle position information is marked on images in the data sets in a manual marking point mode, projection data based on the position points are generated based on the vehicle position information, and then a peripheral boundary box is marked based on the projection data of the position points.
3. The infrared imaging-based 360 degree monitoring vehicle re-identification method of claim 1, wherein the operation of the 2D convolution in the CNN network in step (2) is: , The operation of the 3D convolution is: , Wherein, the Representative of Using 2D for position Convolving; Representative of Is used in the neighborhood of (a), Representing the weight parameter of the object to be weighted, Representing the characteristic layer of the object, Representative of Using 3D for position The convolution is performed with the result that, Representing a conversion function of the sphere to the tangential plane projection; Representing the projected tangential plane to sphere conversion function.
4. The method for identifying the 360-degree monitoring vehicle weight based on the infrared imaging according to claim 3, wherein, The specific process of the step (4) is as follows: , wherein, Representing vehicle location information generated based on the index frame ERP image; Representative is based on Generating an image comprising the vehicle; Representing the refined vehicle block in the index.
5. The method for re-identifying 360 degree monitoring vehicle based on infrared imaging of claim 4, wherein the specific process of step (5) is training based on existing infrared monitoring data set Obtaining trained A network for inputting the refined vehicle block obtained in the step (4) Generating infrared monitoring data I.e. 。
6. The infrared imaging-based 360 degree monitoring vehicle re-identification method of claim 5, wherein the vehicle features based on RGB and NIR data in step (6) are respectively: , , the loss function is: , Wherein, the Representing a global average pooling layer, Representing a feature mapping layer, and mapping feature dimensions into single-dimension features; Representing a matrix 1, denoted as positive samples; represents a matrix of 0, represented as a negative sample.
7. The method for identifying the 360-degree monitoring vehicle weight based on infrared imaging according to claim 6, wherein the weighting process of the method for identifying the 360-degree monitoring vehicle weight based on the attention mechanism in the step (7) is as follows: , the output vehicle re-identification result is: , Wherein, the Representing the final output result of the device, Representing a global averaging pooling layer, converting features into high-dimensional semantic information, And representing the linear mapping layer, and mapping the high-dimensional information to the low latitude so as to output a vehicle re-identification result.
8. The method for identifying the vehicle re-through 360 degrees based on infrared imaging according to claim 7, wherein the network training in the step (8) comprises three parts, the first part is vehicle detection network training, the purpose of the network training is to obtain the position information of the vehicle through 360 degrees based on a spherical convolution mode and the size to be cut, the second part is pseudo-infrared monitoring image generation network training which is adaptive to an anti-domain and is used for enabling the generated pseudo-infrared monitoring image to be higher in quality, and the third part is vehicle re-identification network training based on RGB and NIR data and is used for achieving a final vehicle re-identification network, and whether the vehicle to be searched currently is output or not is judged by judging the similarity between the feature dimensions of the vehicle.

Description

360-Degree monitoring vehicle re-identification method based on infrared imaging Technical Field The invention belongs to the technical field of vehicle re-identification, and relates to a 360-degree monitoring vehicle re-identification method based on infrared imaging. Background Vehicle re-recognition is similar to pedestrian re-recognition in nature by searching the same vehicle from massive video data by giving an image of one vehicle, but the situation of the vehicle re-recognition is more complex compared with the pedestrian re-recognition, and the actual situation to be considered is more. At present, the vehicle re-identification is mainly used for searching and tracking suspected vehicles for criminal investigation, analyzing traffic big data, charging vehicles in a parking lot, counting vehicles and the like. With the development of deep learning technology, a large data set is proposed, and vehicle re-identification is called a very hot research direction in the current computing vision and multimedia research fields. With the rapid development of smart cities, the full coverage of monitoring equipment, especially a sky-eye system, is basically realized in China, and cameras in the whole range are introduced into a unified system for management. How to use these monitoring cameras is a very well studied direction. Currently, devices for 360 degree monitoring are increasingly deployed where wide range monitoring is required, as they can provide a wider range of monitoring, and 360 degrees are dead-free. However, there is little research on vehicle detection based on infrared imaging 360 degree monitoring, mainly because of its higher complexity relative to single angle monitoring. Disclosure of Invention In order to solve the problems, the invention provides a 360-degree monitoring vehicle re-identification method based on infrared imaging, which solves the problem of failure of dim light detection in 360-degree monitoring of infrared imaging and also solves the problem of small coverage range in traditional fixed-view angle 2D prediction vehicle re-identification. In order to achieve the above purpose, the specific process of the invention for realizing the vehicle re-identification is as follows: (1) Collecting a 360-degree vehicle detection data set, and generating pseudo infrared monitoring data as a 360-degree infrared monitoring vehicle re-identification video data set by adopting a domain self-adaptive mode from the 360-degree vehicle detection data set; (2) Carrying out ERP projection on the video data set in the step (1) to obtain an ERP image, sampling spherical pixels into plane convolution based on a sphere sampling strategy of distortion invariance, and extracting 360-degree infrared monitoring vehicle re-identification video features by adopting a CNN network; (3) The method comprises the steps that vehicle position information and width and height of a vehicle are obtained based on a processing mode of 360-degree video of spherical convolution; (4) According to the vehicle position information obtained in the step (3) and the width and height of the vehicle, the vehicle monitored by the spherical surface is projected into a 2D vehicle image in a projection mode, and the vehicle is cut through the width and the height to obtain a refined vehicle block; (5) Generating a pseudo infrared monitoring image by adopting a pseudo infrared monitoring image generation mode based on the adaptation to the anti-domain based on the refined vehicle block obtained in the step (4); (6) Inputting the refined vehicle block obtained in the step (4) and the pseudo infrared monitoring image obtained in the step (5) into a double-flow ResNet network for feature extraction to respectively obtain vehicle features based on RGB and NIR data, and introducing a loss function The generation quality of the pseudo infrared monitoring image is improved; (7) Weighting the vehicle characteristics based on the RGB and NIR data obtained in the step (6) by adopting a mode of an attention mechanism, and outputting a vehicle re-identification result; (8) The network is trained and tested, wherein in the testing stage of the network, after the vehicle position and the size of the clipping area are positioned, the similarity measure between different vehicle characteristics based on RGB and NIR is calculated, so as to judge whether the current vehicle is the searched vehicle or not. As a further technical scheme of the invention, the 360-degree vehicle detection dataset in the step (1) adopts VeRi-776, vehicleID and VERI-Wild datasets, vehicle position information is marked on images in the datasets in a manual point marking mode, projection data based on the position points are generated based on the vehicle position information, and peripheral bounding boxes are marked based on the projection data of the position points. As a further technical solution of the present invention, the operation of the 2D convolution in the CN