CN-116522127-B - Black box countermeasure sample generation method, device, equipment and medium

CN116522127BCN 116522127 BCN116522127 BCN 116522127BCN-116522127-B

Abstract

The application discloses a black box countermeasure sample generation method, a device, equipment and a medium, which relate to the field of deep neural networks and comprise the steps of obtaining a reference deformation image, processing the reference deformation image to obtain dislocation shadows, and generating a black box countermeasure sample based on the dislocation shadows; detecting the black box countermeasure sample, and if the detection does not pass, performing text image semantic segmentation on the black box countermeasure sample to obtain a background image and text position information; and performing background color filling and text pixel filling operation on the dislocation shadow based on the background image and the text position information so as to obtain a target black box countermeasure sample. By the technical scheme, the stability of the black box countermeasure sample generation can be increased, and the universality of the black box countermeasure sample generation is improved.

Inventors

XIA HUI
ZHANG RUI
KANG ZI
JIANG SHULIANG
XU SHUO

Assignees

中国海洋大学

Dates

Publication Date: 20260512
Application Date: 20230116

Claims (8)

1. A black box challenge sample generation method, comprising: Obtaining a reference deformation image, processing the reference deformation image to obtain a dislocation shadow, and generating a black box countermeasure sample based on the dislocation shadow; detecting the black box countermeasure sample, and if the detection does not pass, performing text image semantic segmentation on the black box countermeasure sample to obtain a background image and text position information; Performing background color filling and text pixel filling operations on the dislocation shadows based on the background image and the text position information to obtain a target black box countermeasure sample; The method comprises the steps of obtaining a reference deformation image, carrying out channel translation transformation processing on the reference deformation image by utilizing single-channel dislocation so as to obtain dislocation shadows; The black box countermeasure sample generation method based on the dislocation shadows comprises the steps of obtaining a preset countermeasure disturbance value, generating a sample based on the dislocation shadows, and adding the countermeasure disturbance value to the sample to obtain the black box countermeasure sample.
2. The black box challenge sample generation method of claim 1, wherein the detecting the black box challenge sample comprises: Generating sample detection conditions based on visual perception business requirements; and judging whether the black box countermeasure sample meets the sample detection condition.
3. The black box countermeasure sample generation method according to claim 1, wherein the text image semantic segmentation is performed on the black box countermeasure sample to obtain a background image and text position information, including: performing image binarization processing on the black box countermeasure sample to obtain a processed black box countermeasure sample; And carrying out text image semantic segmentation on the processed black box countermeasure sample to obtain a background image and text position information.
4. The black box countermeasure sample generation method of claim 1, wherein the background color filling and text pixel filling operation of the dislocation shadows based on the background image and the text position information includes: Determining the pixel quantity of the dislocation shadow, and filling the dislocation shadow with background color based on the background image and the pixel quantity; And determining a dislocation text based on the text position information and the dislocation shadow, and performing text pixel filling operation on the dislocation text.
5. The black box countermeasure sample generation method according to claim 1, wherein the generating a sample based on the dislocation shadows and adding the countermeasure disturbance value to the sample to obtain a black box countermeasure sample includes: performing distortion transformation, image noise processing and geometric transformation on the reference deformed image to obtain a reference deformed image after the image deformation processing, and performing channel dislocation transformation on the reference deformed image after the image deformation processing to obtain an image after the channel dislocation transformation processing; generating a sample based on the dislocation shadows and adding the counterdisturbance value to the sample; And calling a preset function, and generating the black box countermeasure sample based on the reference deformed image after the image deformation processing, the image after the channel dislocation transformation processing and the sample.
6.A black box challenge sample generating device, comprising: the image acquisition module is used for acquiring a reference deformation image, processing the reference deformation image to obtain dislocation shadows, and generating a black box countermeasure sample based on the dislocation shadows; the detection module is used for detecting the black box countermeasure sample, and if the detection does not pass, text image semantic segmentation is carried out on the black box countermeasure sample so as to obtain a background image and text position information; The target black box countermeasure sample generation module is used for carrying out background color filling and text pixel filling operation on the dislocation shadow based on the background image and the text position information so as to obtain a target black box countermeasure sample; The image acquisition module is specifically used for acquiring a reference deformation image, and carrying out channel translation transformation processing on the reference deformation image by utilizing single-channel dislocation so as to obtain dislocation shadows; the image acquisition module is specifically used for acquiring a preset disturbance countermeasure value, generating a sample based on the dislocation shadow, and adding the disturbance countermeasure value to the sample to obtain a black box disturbance countermeasure sample.
7. An electronic device, comprising: A memory for storing a computer program; A processor for executing the computer program to implement the black box challenge sample generating method of any of claims 1 to 5.
8. A computer-readable storage medium for storing a computer program, wherein the computer program when executed by a processor implements the black box challenge sample generation method of any of claims 1 to 5.

Description

Black box countermeasure sample generation method, device, equipment and medium Technical Field The invention relates to the field of deep neural networks, in particular to a black box countermeasure sample generation method, a black box countermeasure sample generation device, black box countermeasure sample generation equipment and black box countermeasure sample generation medium. Background The scene text recognition based on the deep neural network overcomes the defects that the traditional scene text recognition is difficult to detect irregular texts and severely depends on hand-made features, is widely applied to various text recognition tasks such as license plate recognition, document automation, road sign recognition, optical character recognition and the like, but inherits the vulnerability of the deep neural network to attack resistance, namely, a scene text recognition model based on the deep neural network can be misled to give an erroneous recognition result by adding fine disturbance on a benign sample. Although there has been a great deal of research on combating attacks at present, they have focused mainly on the fields of image classification, object monitoring, and instance segmentation. Because the scene text image is irregular in shape, an attack scheme aiming at victim models such as an image classifier and a target detector cannot be directly applied to the scene text identifier based on the deep neural network, and the scene text image contains a large number of tiny objects, compared with classical image classification and target detection, the attack scheme aiming at the scene text identifier based on the neural network has much greater challenges. Song et al first developed a study of the text detector against attacks, and then developed some attack schemes for scene text recognizers, detectors, but these schemes still remained in the white-box attack study phase of constructing a challenge sample based on victim model gradients. In fact, in practical applications, the adversary cannot obtain confidential information such as network structure, model gradient and the like of the victim model. Furthermore, existing attack schemes still employ a construction pattern of adding an anti-disturbance on a benign image when generating an anti-sample, which results in the generated anti-sample being unnatural, easily perceived by the human eye, and difficult to evade inspection of defenses. From the above, it is a problem to be solved in the art how to increase the stability of the black box against the generation of the sample and to increase the versatility of the black box against the generation of the sample. Disclosure of Invention In view of the above, the present invention aims to provide a method, a device, an apparatus and a medium for generating a black box countermeasure sample, which can increase the stability of the generation of the black box countermeasure sample and increase the versatility of the generation of the black box countermeasure sample. The specific scheme is as follows: in a first aspect, the application discloses a black box challenge sample generation method, comprising: Obtaining a reference deformation image, processing the reference deformation image to obtain a dislocation shadow, and generating a black box countermeasure sample based on the dislocation shadow; detecting the black box countermeasure sample, and if the detection does not pass, performing text image semantic segmentation on the black box countermeasure sample to obtain a background image and text position information; and performing background color filling and text pixel filling operation on the dislocation shadow based on the background image and the text position information so as to obtain a target black box countermeasure sample. Optionally, the acquiring a reference deformed image, and processing the reference deformed image to obtain a dislocation shadow, includes: acquiring a reference deformation image; And carrying out channel translation transformation processing on the reference deformed image by utilizing single-channel dislocation so as to obtain dislocation shadows. Optionally, the detecting the black box challenge sample includes: Generating sample detection conditions based on visual perception business requirements; and judging whether the black box countermeasure sample meets the sample detection condition. Optionally, the performing text image semantic segmentation on the black box countermeasure sample to obtain a background image and text position information includes: performing image binarization processing on the black box countermeasure sample to obtain a processed black box countermeasure sample; And carrying out text image semantic segmentation on the processed black box countermeasure sample to obtain a background image and text position information. Optionally, the performing background color filling and text pixel filling operations on the dislocation shadows b