CN-116778530-B - Cross-appearance pedestrian re-identification detection method based on generation model

CN116778530BCN 116778530 BCN116778530 BCN 116778530BCN-116778530-B

Abstract

The invention provides a cross-appearance pedestrian re-recognition detection method based on a generation model, which belongs to the technical field of pedestrian re-recognition and comprises the following steps of generating new pedestrian images by exchanging body type characteristics and appearance characteristics among different pedestrian images in the generation model, obtaining a pedestrian profile sketch and a pedestrian analysis drawing by an RGB image of the pedestrian images through a pre-trained edge detection network and a pedestrian semantic segmentation network, inputting the same backbone network extraction characteristics into the pedestrian profile sketch, the RGB image and the pedestrian analysis drawing of the pedestrian images of a pedestrian dataset, and carrying out reasoning training after characteristic fusion. The invention uses the generated model to strengthen the appearance dimension of the pedestrian image and introduces the generated pedestrian image into the training stage of the model, and the three modes use the same backbone network to extract the characteristics, fuse and infer, guide the model to learn the key characteristics of different appearances of the same pedestrian, and the model has more robust performance under the scene crossing the appearance.

Inventors

LIU MENGMENG
Xie Xueshuo
LI TAO

Assignees

先进计算与关键软件(信创)海河实验室
南开大学

Dates

Publication Date: 20260505
Application Date: 20230721

Claims (4)

1. The cross-appearance pedestrian re-identification detection method based on the generated model is characterized by comprising the following steps of: Step 1, inputting pedestrian images of a pedestrian data set into a generation model, respectively obtaining body type characteristics and appearance characteristics of the pedestrian images by different encoders of the generation model, exchanging the body type characteristics and the appearance characteristics among different pedestrian images, generating a new pedestrian image through a generator, identifying tag information of the new pedestrian image by matching with a discriminator, and storing the new pedestrian image and the tag information thereof into the pedestrian data set; Step 2, taking RGB images of all pedestrian images of the pedestrian data set as input, respectively extracting contour information and semantic segmentation information of pedestrians through a pre-trained edge detection network and a pre-trained pedestrian semantic segmentation network to obtain a pedestrian contour sketch and a pedestrian analysis drawing, and storing the pedestrian contour sketch and the pedestrian analysis drawing in the pedestrian data set; Step 3, inputting the pedestrian contour sketch, the RGB image and the pedestrian analytic image of the pedestrian data set into the same backbone network to extract the characteristics, thereby obtaining 、 And Feature fusion to obtain Then carrying out reasoning training, and carrying out cross-appearance pedestrian re-identification detection on the backbone network after training; In the step 1, after the pedestrian image is input into the generation model, the appearance characteristic and the body type characteristic are respectively obtained through an appearance encoder and a body type encoder, the appearance characteristic and the body type characteristic of the same pedestrian image are input into a generator to obtain a self-reconstruction pedestrian image, and a discriminator discriminates the label information of the self-reconstruction pedestrian image; In the step 1, the appearance encoder is a characteristic extraction network of a pedestrian re-identification network, and the body type encoder is a network which introduces pyramid structures focused on different scales; When the generated model is trained, the appearance encoder uses an identity loss function to conduct constraint, and the body type encoder uses an identity tag loss function to guide training of the model.
2. The method for detecting the cross-appearance pedestrian re-recognition based on the generated model as set forth in claim 1, wherein in the step 2, the edge detection network utilizes an edge detection data set to complete pre-training, and the pedestrian semantic segmentation network utilizes an LIP data set to complete pre-training.
3. The method for detecting cross-appearance pedestrian re-recognition based on a generated model as claimed in claim 1, wherein in step 3, the method comprises the following steps of 、 And The fusion mode of (a) is weighted splicing.
4. The method for detecting cross-appearance pedestrian re-recognition based on a generated model as claimed in claim 1, wherein in the step 3, densenet is adopted as the backbone network.

Description

Cross-appearance pedestrian re-identification detection method based on generation model Technical Field The invention belongs to the technical field of pedestrian re-recognition, and particularly relates to a cross-appearance pedestrian re-recognition detection method based on a generation model. Background At present, the application scene of the computer vision technology is very wide. The face recognition technology is superior to human beings, is widely applied to industries such as industry, medical treatment, education and the like, and the eyes in academic circles and industry gradually turn to a subject with scientific research significance and application value, namely pedestrian re-recognition. In practical scenes such as transportation, industrial manufacturing and the like, most cases are the case of blurring a human face and even not containing the human face, so that the human face recognition effect is very limited. The Re-identification (Re-ID) aims at giving a monitoring pedestrian image to search other images of the pedestrian under cross equipment, and the pedestrian Re-identification technology can make up for the failure of face recognition and the visual limitation of a fixed camera and is applied to the fields of video monitoring, intelligent security, intelligent life and the like. Because Re-ID needs to find objects in images and videos across devices, the resolution and the positions of the devices are different, the coverage areas of the devices are not overlapped, so that the lack of coherent information is caused, the illumination, the background and the shielding of scenes are different, the posture and the appearance of the objects are changed, and the Re-recognition technology of pedestrians is greatly challenged. A pedestrian re-recognition method based on deep learning feature learning. The global features mainly utilize global images of the whole body to perform feature learning, common improvement ideas include an Attention mechanism, multi-scale fusion and the like, the local features utilize local image areas such as partial structures of pedestrians or simple vertical area division to perform feature learning, the local features are finally aggregated into final pedestrian features to be identified, the auxiliary features utilize auxiliary information to enhance the effect of feature learning such as semantic information, visual angle information, domain information, information generated by GAN, data enhancement and the like, and the specific network design utilizes the characteristics of Re-ID tasks to design a plurality of network structures with fine granularity, multi-scale and the like so that the network structure is more suitable for Re-ID scenes. The pedestrian re-identification method based on the deep learning metric learning mainly comprises the design of different types of loss functions and the improvement of sampling strategies. Identity loss, namely taking the training process of Re-ID as an image classification problem, taking different pictures of the same pedestrian as a category, commonly having a Softmax cross entropy loss function, verification loss, namely taking the training of Re-ID as an image matching problem, whether the training of Re-ID belongs to the same pedestrian to perform classification learning, commonly having a contrast loss function and a classification loss function, and triple loss, namely taking the training of Re-ID as an image retrieval problem, wherein the characteristic distance of the picture of the same pedestrian is smaller than the characteristic distance of different pedestrians, and various improvements thereof, and the improvement of training strategies, namely a self-adaptive sampling mode and different weight allocation strategies. Cross-appearance pedestrian re-recognition methods researchers have proposed cross-appearance pedestrian re-recognition methods applied to these datasets with the release of cross-appearance pedestrian re-recognition datasets, most of which abandoned clothing-related appearance features. The RF-Reid is an end-to-end model for pedestrian re-identification, and takes a radio frequency track as input, and features are extracted from the track, so that the model can obtain accurate human body contours for identification. In PRCC, researchers use Angle Specific Extractors (ASE) to extract fine-grained angle specific discrimination features by varying the sampling range of the SPT and to contribute a multi-stream network to aggregate multi-granularity features. The BC-Net uses the clothing template to search the candidate pedestrian image by using the double-branch network, so that the biological characteristics and clothing characteristics are effectively fused. CASE-Net utilizes a cross-appearance, antagonistic learning strategy to extract body type features and structure body type through image generation of posture changes. FSAM proposes a dual stream framework that learns fine-grained b