US-12625953-B2 - Adaptive meta-attack system and method for target tracker under autonomous driving scenarios

US12625953B2US 12625953 B2US12625953 B2US 12625953B2US-12625953-B2

Abstract

An adaptive meta-attack system for target trackers under autonomous driving scenarios including an initialization module, a meta-training iteration module, a meta-testing module, a perturbation generator, and an inference module. The initialization module includes a model initialization sub-module and a parameter initialization sub-module. The meta-training iteration module includes a video input sub-module, a training-validation model division sub-module, and a meta-training sub-module. The meta-testing module includes a performance validation and evaluation sub-module and a parameter output sub-module. An adaptive meta-attack method for a target tracker under autonomous driving scenarios applied in the adaptive meta-attack system is further provided.

Inventors

Yuanfang CHEN
Sihang MA
Xing Fang
Xiaohan Chen
Jie Xiong
Xiajun HE

Assignees

HANGZHOU DIANZI UNIVERSITY

Dates

Publication Date: 20260512
Application Date: 20240905
Priority Date: 20240329

Claims (7)

1 . An adaptive meta-attack method for a target tracker under an autonomous driving scenario using an adaptive meta-attack system, the adaptive meta-attack system comprising: an initialization module; a meta-training iteration module; a meta-testing module; a perturbation generator; and an inference module; wherein the initialization module comprises a model initialization sub-module and a parameter initialization sub-module; the model initialization sub-module is configured to perform model initialization, and the parameter initialization sub-module is configured to perform parameter initialization; the meta-training iteration module comprises a video input sub-module, a training-validation model division sub-module, and a meta-training sub-module; the meta-testing module comprises a performance validation and evaluation sub-module and a parameter output sub-module; the meta-training iteration module is configured to train the perturbation generator through multiple iterations by using a video dataset input by the video input sub-module to obtain a trained perturbation generator; the meta-testing module is configured to perform performance validation and evaluation on the trained perturbation generator; and the inference module is configured to attack a video by using the trained perturbation generator; the adaptive meta-attack method comprising: (a) constructing, by the model initialization sub-module, a tracker model pool M={M1, M2 . . . , Mn} with a diversified network structure; initializing, by the model initialization sub-module, the tracker model pool; and performing, by the parameter initialization sub-module, parameter initialization of the perturbation generator; wherein the tracker model pool Mis configured to construct a meta-training model pool N and a meta-testing model E to make the perturbation generator adaptive to different trackers; and a parameter of the perturbation generator is 0 output from a previous training process; and random parameters are used in a first training; (b) extracting, by the video input sub-module, a video from the video dataset for sampling, and extracting 1 frame from every 10 frames to constitute a training image for a current round; (c) randomly selecting, by the training-validation model division sub-module, one tracker model from the tracker model pool M to construct the meta-testing model E; wherein remaining tracker models in the tracker model pool M are configured as the meta-training model pool N; (d) performing, by the meta-training sub-module, random sampling in the meta-training model pool N to construct a tracker model group S; inputting a search region into the perturbation generator to generate a first adversarial example; and computing a dual-confidence balance optimization loss function L c and a peripheral attack regression loss function L r generated by each of tracker models in the tracker model group S; and performing weighted ensemble on the dual-confidence balance optimization loss function L c and the peripheral attack regression loss function L r , and performing model optimization on the perturbation generator by back propagation; (e) repeating step (d) φ times, and outputting a parameter θ 1 ; (f) performing performance test, by the performance validation and evaluation sub-module, on the perturbation generator trained each time by the meta-training sub-module, wherein a tracker model used by the performance validation and evaluation sub-module is the meta-testing model E; (g) outputting, by the parameter output sub-module, a parameter θ 2 of the perturbation generator in the current round; (h) executing step (b), and repeating training for t times; and taking a parameter of the perturbation generator after last training as a final parameter; and (i) loading, by the inference module, the final parameter of the perturbation generator to generate perturbation and superimpose the perturbation onto an original search image to generate a second adversarial example, thereby deceiving the target tracker.
2 . The adaptive meta-attack method of claim 1 , wherein in the step (d), a meta-training algorithm is performed by the meta-training sub-module through the following steps: (d1) performing the random sampling in the meta-training model pool N to construct the tracker model group S; (d2) performing a multi-task training on the tracker model group S; and transmitting the original search image frame by frame to the perturbation generator to generate the first adversarial example; (d3) inputting the first adversarial example and an original example into the tracker model group S to obtain a feature map of each of the tracker models in the tracker model group S, and analyzing a difference between the first adversarial example and the original example and deceiving the tracker models in the tracker model group S; and (d4) taking the first adversarial example and the original example respectively as input images of the tracker models in the tracker model group S to obtain a regression feature map generated by each of the tracker models; calculating, based on the regression feature map, a classification feature map and a probability feature map created by softmax, the dual-confidence balance optimization loss function L c and the peripheral attack regression loss function L r ; and weighting and summing the dual-confidence balance optimization loss function L c and the peripheral attack regression loss function L r to obtain a combined loss, and updating model parameters of the perturbation generator by back propagation.
3 . The adaptive meta-attack method of claim 1 , wherein in the step (d), a training algorithm for the perturbation generator in the meta-training sub-module is performed through the following steps: (d-1) randomly selecting the one tracker model from the tracker model pool M as the meta-testing model E, and taking the remaining tracker models in the tracker model pool M as the meta-training model pool N; and performing the random sampling in the meta-training model pool N to construct the tracker model group S; (d-2) producing, by a perturbation generator G, a perturbation image P; generating the first adversarial example corresponding to the search region; based on the first adversarial example and an original example, generating regression feature maps, classification feature maps and probability feature maps respectively of each of the tracker models; computing the dual-confidence balance optimization loss function L c and the peripheral attack regression loss function L r , and weighting and summing the dual-confidence balance optimization loss function L c and the peripheral attack regression loss function L r ; and (d-3) performing parameter updating on the perturbation generator.
4 . The adaptive meta-attack method of claim 1 , wherein in the step (d), the dual-confidence balance optimization loss function L c is expressed as: L C high ( P m a , P m h , C m a ) = ( 1 ) 1 H ⁢ ∑ H = P m h > δ ( λ 1 ⁢ BCE ⁡ ( P m a [ A ] , zero ) ± λ 2 ( C m a [ H ] [ : 0 ] - C m a [ H ] [ : 1 ] ) ) * ( P m h ) 2 L C blow ( P m a , P m h , C m a ) = 1 L ⁢ ∑ L = Sort ⁡ ( ϑ < P m h < ζ ) ( λ 3 ⁢ BCE ⁡ ( P m a [ L [ : 1.5 H ] ] , one ) ± λ 4 ( C m a [ L [ : 1.5 H ] ] [ : 0 ] - C m a [ L : 1.5 H ] [ : 1 ] ) ) * ( P m h ) 2 ; ( 2 ) L c = λ 5 * L C high + λ 6 * L C blow ; ( 3 ) wherein δ represents a high confidence threshold; ζ represents a medium confidence threshold; Θ represents a low confidence threshold; m represents a certain tracker; h represents an original example; a represents the adversarial example; P represents a probability feature map generated by softmax on a classification feature map; C represents the classification feature map; BCE represents a binary cross-entropy formula; H represents an index of a confidence region with P m h > δ ; λ 1 ,λ 2 , λ 3 , λ 4 , λ 5 , and λ 6 represent weights of individual weight coefficients; P m h represents a probability feature map generated by an original example transfer tracker m; P m a represents a probability feature map generated by an adversarial example transfer tracker m; C m a represents a classification feature map generated by the adversarial example transfer tracker m; L C high represents a loss function of a high confidence region which is greater than the δ; and L C blow represents a loss function of a low confidence region which is lower than the δ.
5 . The adaptive meta-attack method of claim 1 , wherein in the step (d), the peripheral attack regression loss function L r is expressed as: L r ( R m h , R m a , P m h ) = 1 H ⁢ ∑ H = P m h > δ λ 7 ⁢ giou ( R m h [ H ] , R m a [ H ] ) ; ( 1 ) wherein L r represents the peripheral attack regression loss function; R represents generation of a prediction box boundary; R m a represents a prediction box boundary generated by an adversarial example transfer tracker m; R m h represents a prediction box boundary generated by an original example transfer tracker m; m represents a certain tracker; h represents an original example; δ represents a high confidence threshold; H represents an index of a confidence region with P m h > δ ; represents a weight coefficient; P m h represents a probability feature map generated by the original example transfer tracker m; R m a [ H ] represents a regression feature map generated by a tracker model in the tracker model pool M in a meta-training phase after experiencing an adversarial attack within a confidence range of [δ, 1], which reflects a position of a prediction box; and R m h [ H ] represents a regression feature map corresponding to a highest confidence determined by the tracker model in the tracker model pool M within an unattacked search region.
6 . The adaptive meta-attack method of claim 1 , wherein in the step (f), an algorithm for performance validation and evaluation is performed through the following steps: applying a meta-training strategy in a meta-training phase to the performance validation and evaluation sub-module; inputting the first adversarial example and the original example to the meta-testing model E to obtain a regression feature map, a classification feature map, and a probability feature map to test generalization ability of the perturbation generator for tracker models in the meta-testing model E, and to evaluate and guide an optimization direction of the perturbation generator.
7 . The adaptive meta-attack method of claim 1 , wherein in the step (d), an algorithm for the perturbation generator in the meta-training module is performed through the following steps: generating a perturbation image by the perturbation generator G; generating the first adversarial example corresponding to the search region; inputting the first adversarial example into the meta-testing model E to generate a classification feature map and a probability feature map; calculating the dual-confidence balance optimization loss function L c and the peripheral attack regression loss function L r ; and performing parameter updating on the perturbation generator.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of priority from Chinese Patent Application No. 202410373108.9, filed on Mar. 29, 2024. The content of the aforementioned application, including any intervening amendments thereto, is incorporated herein by reference in its entirety. TECHNICAL FIELD This application relates to autonomous driving and machine vision-based target tracking, and more particularly to an adaptive meta-attack system and method for target trackers under autonomous driving scenarios. BACKGROUND In the digital age, the intelligent driving systems are challenged by adversarial attacks. These adversarial attacks may cause the autonomous driving system to make wrong decisions, increasing the risk of traffic accidents. As a key part of the intelligent driving systems, the visual target tracking technique can help the intelligent driving systems recognize and track various objects on the road. However, the adversarial attacks can interfere with these tracking systems to lead to incorrect target tracking or tracking failures, thus threatening the safety and stability of intelligent vehicles. To address this challenge, researchers have proposed adversarial attack algorithms to promote the understanding and improve the attack effectiveness and generalization performance of adversarial examples. The current researches mainly focus on white-box attack, black-box attack, and semi-white-box attack. Among them, the semi-white-box attack is considered as an effective method due to its good adaptability and balance in designing attack strategies in the case of limited model information. Target tracking algorithms mainly rely on deep learning. With the popularization of the Transformer model, it will inevitably play a dominant role in the automatic driving, and the current visual target tracking is mainly based on the Transformer framework instead of the Siamese network. However, different algorithms have different structures and principles, which lead to some limitations in the study of attacks against specific trackers. To address the shortcomings of the existing attack methods, a method for enhancing adversarial attacks against target tracking systems in the autonomous driving has been developed, which can improve the attack effectiveness and generalization performance of adversarial examples in complex autonomous driving scenarios by generating adversarial examples with high efficiency and effectiveness and strong generalization performance through deep analysis. This research contributes to the enhancement of the defense capability of target tracking systems against adversarial attacks and the development of adversarial defense techniques. SUMMARY This application aims to address the adversarial attack problems faced by visual tracking in autonomous driving systems, including the emergence of adversarial examples, insufficient generalization capability, and balance between effectiveness and speed. With the development of the autonomous driving technology, the threat suffered by the visual tracking system from adversarial examples is increasingly serious. The adversarial examples are specially-designed input data that can deceive the tracking system, and thus can cause the system to produce incorrect recognition and tracking results, thus affecting the performance and safety of the entire autonomous driving system. The traditional adversarial attack methods have insufficient generalization capability, such that the attack strategy may be effective only for a specific tracker model, and fails to adapt to various tracker structures and characteristics. The adversarial attack methods need to ensure the attack effectiveness while minimizing the impact on the performance and speed of the tracking system, so as to ensure the real-time performance and stability of the autonomous driving system. The technical solutions of the present application are as follows. This application provides an adaptive meta-attack system for a target tracker under an autonomous driving scenario, comprising an initialization module, a meta-training iteration module, a meta-testing module, a perturbation generator, and an inference module; wherein the initialization module comprises a model initialization sub-module and a parameter initialization sub-module; the model initialization sub-module is configured to perform model initialization, and the parameter initialization sub-module is configured to perform parameter initialization;the meta-training iteration module comprises a video input sub-module, a training-validation model division sub-module, and a meta-training sub-module;the meta-testing module comprises a performance validation and evaluation sub-module and a parameter output sub-module;the meta-training iteration module is configured to train the perturbation generator through multiple iterations by using a video dataset input by the video input sub-module to obtain a trained perturbation ge