Search

CN-117011648-B - Haptic image dataset expansion method and device based on single real sample

CN117011648BCN 117011648 BCN117011648 BCN 117011648BCN-117011648-B

Abstract

The invention belongs to the technical field of network training, and discloses a method and equipment for expanding a haptic image data set based on a single real sample, wherein the method comprises the steps of (1) constructing a simulation scene corresponding to a real environment, and then constructing a simulation haptic data set; the method comprises the steps of (1) obtaining a single real touch image sample, training a multi-scale generation countermeasure network model by adopting the single real touch image sample, (3) fixing the weight of a generator after training each scale generator in the multi-scale generation countermeasure network model, adding countermeasure disturbance to a discriminator, and (4) inputting a simulated touch data set into a trained multi-scale countermeasure generation neural network to obtain a data set after preliminary expansion, and carrying out various image transformations on the data set after preliminary expansion to realize further expansion. The invention can construct a large-scale tactile data set based on a single real tactile sample collected by any high-resolution tactile sensor, and reduce the collection cost of the large-scale tactile data set.

Inventors

  • GONG ZEYU
  • TANG JING
  • TAO BO
  • WU LI
  • ZHAO ZITONG

Assignees

  • 华中科技大学

Dates

Publication Date
20260505
Application Date
20230823

Claims (9)

  1. 1. A method for augmenting a haptic image dataset based on a single real sample, the method comprising the steps of: (1) Constructing a simulation scene corresponding to the real environment, and constructing a simulation touch data set under the simulation scene; (2) Obtaining a single real touch image sample, training a multiscale generation countermeasure network model by adopting the single real touch image sample, directly inputting a noise image with a corresponding size at a first scale in the multiscale generation countermeasure network model in the training process, inputting the real touch image sample scaled to the corresponding scale and the noise image with the same scale for each subsequent scale, fixing the weight of a scale discriminator, and adding countermeasure disturbance to a generator; (3) After training of each scale generator in the multi-scale generation countermeasure network model is completed, the weight of the generator is fixed, and countermeasure disturbance is added to the discriminator; (4) Inputting the simulated haptic data set into the trained multi-scale countermeasure generation neural network to obtain a data set after preliminary expansion, and further expanding the data set after preliminary expansion; the step of further expanding the preliminarily expanded data set by adopting elastic transformation is as follows: First, based on each pixel in the haptic image Constructing two random values, called And The values of which are in the range of-1 to 1, respectively expressed in And Distance moved in the direction; Then, a Gaussian kernel of size n is constructed As a kernel function of the convolution operation, the mean value is 0, and the standard deviation is ; Then, according to And The shifted image is convolved to obtain a final deformed image.
  2. 2. A single real sample based haptic image dataset expansion method as recited in claim 1 wherein the network generator and discriminator of the multi-scale generation countermeasure network are structurally identical and consist of 5 fully connected layers, the input scale of the fully connected layers of each scale generation countermeasure network model is the last scale 。
  3. 3. The haptic image dataset augmentation method based on a single real sample of claim 2, wherein the network penalty of generator training is: Wherein the method comprises the steps of Is a generator that generates an antagonism network, Is to generate the antagonism network A discriminator of the individual dimensions of the sample, Generating an reactance network Loss of contrast of individual scale for computing original image Counterfeit samples generated by a generator Differences between; Is a reconstruction penalty for ensuring that each generator is able to generate and compare An image is similarly generated.
  4. 4. A method for augmenting a tactile image dataset based on a single real sample as recited in claim 1, wherein the identifier comprises The WGAN-GP loss is selected and used, The addition of the challenge disturbance to the discriminator involves the following sub-steps: first, based on generating a sample image Computer generator network Training loss And calculate the gradient of training loss ; Then, generating a sample image at the corresponding scale input Adding disturbance thereto Disturbance is satisfied The concrete calculation mode is as follows: Wherein the method comprises the steps of ; Then, continuously adding T times of disturbance on the image input by the corresponding scale, wherein the disturbance is calculated in the following way: Wherein: , 。
  5. 5. The method for augmenting a single real sample-based haptic image dataset of claim 1-4, further augmenting the initially augmented dataset with affine or rotational transformations.
  6. 6. The single true sample based haptic image dataset augmentation method of claim 1, wherein: the values are achieved by minimizing the combined objective function: Wherein the method comprises the steps of For the image structure similarity function, calculating the image similarity between the single real tactile image and the corresponding simulated tactile image; is a root mean square error measurement function for calculating the image variability between a single real haptic image and a corresponding simulated haptic image.
  7. 7. The method for augmenting a single real sample-based haptic image dataset of any one of claims 1-4, wherein adding a tamper-resistant means to the generator comprises the sub-steps of: first, based on a real sample image Computer generator network Training loss And calculate the gradient of training loss ; Then, the real sample image is input at the corresponding scale Adding disturbance thereto Disturbance is satisfied The concrete calculation mode is as follows: Wherein the method comprises the steps of ; Then, continuously adding T times of disturbance on the image input by the corresponding scale, wherein the disturbance is calculated in the following way: Wherein: , 。
  8. 8. a haptic image dataset expansion system based on a single real sample, characterized in that the system comprises a memory and a processor, the memory storing a computer program, the processor executing the method for haptic image dataset expansion based on a single real sample as claimed in any one of claims 1-7 when executing the computer program.
  9. 9. A computer readable storage medium, characterized in that it stores machine executable instructions that, when invoked and executed by a processor, cause the processor to implement the single real sample based haptic image dataset augmentation method of any one of claims 1-7.

Description

Haptic image dataset expansion method and device based on single real sample Technical Field The invention belongs to the technical field of network training, and particularly relates to a method and equipment for expanding a tactile image data set based on a single real sample. Background The tactile image data is a type of tactile information collected by a high resolution Vision-based tactile sensor, which can reflect distribution of contact surface force, surface texture in an image manner. However, since these high resolution tactile sensors capture tactile images through deformation of the camera capture contact gel face, the gel face is very vulnerable to breakage, which presents a significant challenge in constructing large-scale tactile image datasets. In recent years, with the development of deep learning technology, many object classification models based on haptic image data, vision-haptic reconstruction models, and robot gripping models based on haptic image data depend on a large-scale data set. Therefore, the haptic image data expansion based on a single real haptic sample can help to improve the performance of the deep learning task based on the haptic image data. China patent 201811270865.4 proposes a mobile robot feasible region training dataset expansion method. The invention uses the binocular camera to collect the original image which is rated and contains the topography, then the image is standardized, so that the subsequent data set expansion and transformation are convenient, and then the image transformation and expansion are carried out. On the basis, image samples which can be acquired only under a plurality of special conditions are converted by image synthesis, adding rain and snow marks and simulating infrared to expand the image samples obtained under different weather and shooting conditions. The invention effectively expands the coverage range of the data set, increases more training samples for subsequent machine learning, obviously shortens the period of constructing the data set, reduces the cost of constructing the data set, assists in improving the training effect of the mobile robot and improves the recognition rate of the robot to a feasible region under various special conditions. Chinese patent 201911056394.1 proposes an expansion method, a training method and a related device for training images, which provide an expansion method, a training method and a related device for training images, and relate to the field of pedestrian recognition in machine learning. The expansion method of the training image comprises the steps of obtaining a plurality of images to be converted of pedestrians, wherein the plurality of images to be converted comprise at least two pedestrian color data of the pedestrians, obtaining a color data generation model according to the plurality of images to be converted, wherein the color data generation model is a model obtained by training at least two pedestrian color data under a generation contrast network, inputting the plurality of images to be converted into the color data generation model, and obtaining a plurality of target color images, wherein the images to be converted and the plurality of target color images are used as a color training set. By using the data expansion method provided by the application, more specific color data can be obtained on the basis of real color data, so that the training data requirement of training a color recognition model is met, and the accuracy of color recognition is improved. In general, a data set expansion method for haptic image data has not been proposed yet, and a data set expansion method for a single sample is also to be studied urgently, so that a haptic image data set expansion method based on a single real sample is to be proposed urgently in the art to realize a high-performance deep learning task based on a single real haptic sample. Disclosure of Invention In order to meet the above-mentioned defects or improvement demands of the prior art, the invention provides a haptic image dataset expansion method and device based on a single real sample, which can construct a large-scale haptic dataset based on a single real haptic sample collected by any high-resolution haptic sensor, can greatly reduce the collection cost of the large-scale haptic dataset, and meanwhile, the quality of the generated haptic image is higher due to the introduction of a countermeasure training method. To achieve the above object, according to one aspect of the present invention, there is provided a haptic image dataset expansion method based on a single real sample, the method comprising the steps of: (1) Constructing a simulation scene corresponding to the real environment, and further constructing a simulation touch data set; (2) Obtaining a single real touch image sample, training a multiscale generation countermeasure network model by adopting the single real touch image sample, directly inputting a noise image