CN-116403091-B - Haptic object identification method based on gradient self-adaptive sampling and 3D neural network

CN116403091BCN 116403091 BCN116403091 BCN 116403091BCN-116403091-B

Abstract

The invention provides a haptic object recognition method based on gradient self-adaptive sampling and a 3D neural network, which is used for solving the technical problems of information redundancy/loss caused by the use of a uniform sampling strategy in the conventional haptic object recognition model and the technical problem that generalization capability is insufficient for processing haptic data at different grabbing speeds. The method comprises the steps of sending an original touch frame into a gradient self-adaptive sampling strategy to carry out self-adaptive selection of the touch frame, obtaining a touch frame set with rapid gradient change, carrying out multi-time scale downsampling on the touch frame set, carrying out feature extraction on the downsampled touch frame by using an MR3D-18 network to obtain features in different time scales, fusing the features in different time scales, and identifying the category of an object according to the fused features to obtain a prediction classification result. The method is based on a gradient self-adaptive sampling strategy and a multi-time scale 3D convolutional neural network, and can effectively improve the recognition accuracy of the haptic object recognition task.

Inventors

WANG WEI
LIU YUCUI
DENG WEI
QIAN XIAOLIANG
MENG JIA
ZENG LI
YUE WEICHAO
Ren Hangli
LIU XIANGLONG
WANG FANG

Assignees

郑州轻大产业技术研究院有限公司
郑州轻工业大学

Dates

Publication Date: 20260508
Application Date: 20230418
Priority Date: 20230320

Claims (6)

1. A haptic object identification method based on gradient self-adaptive sampling and 3D neural network is characterized by comprising the following steps: The method comprises the steps of firstly, adaptively selecting original touch frames by using a gradient adaptive sampling strategy, and obtaining a touch frame set with quicker gradient change; Step two, sampling the touch frame set obtained in the step one in a multi-time scale manner; Respectively extracting features of the downsampled haptic frame set by using an MR3D-18 network to obtain features in different time scales, fusing the features in different time scales, and identifying the category of the object according to the fused features to obtain a prediction classification result; The method for adaptively selecting the touch frame by the gradient adaptive sampling strategy comprises the following steps: S1, performing matrix subtraction on adjacent original haptic frames in input haptic data to obtain gradient absolute value matrixes of two adjacent haptic frames at the moment t, and performing matrix subtraction on an initial haptic frame to obtain gradient absolute value matrixes of the haptic frames at the initial moment; s2, normalizing the absolute value matrix of the gradient in the time dimension, and calculating the normalized gradient cumulative distribution; s3, dividing the sampling time of the touch frame into N sections according to the accumulated gradient distribution, randomly taking out one frame from the N sections as the touch frame, and forming a touch frame set by the N touch frames; The implementation manner of the step S1 is as follows: ; where D represents a set of original haptic frames, H, W and T represent the height, width and number of original haptic frames, respectively, Representing the original haptic frame at time t, Representing the original haptic frame at time t-1, Representing the absolute value matrix of the gradients of two adjacent original haptic frames at time t, A gradient absolute value matrix of the initial frame; the implementation manner of the step S2 is as follows: ; Wherein, the Representing absolute value matrix of gradient At time t the values of x rows and y columns, Representing absolute value matrix of gradient Is used for the normalization matrix of the (c), Represents the gradient accumulation at time t, an , 。
2. The method for identifying the tactile object based on the gradient adaptive sampling and the 3D neural network according to claim 1, wherein the method for acquiring the tactile frame set is as follows: Gradient accumulation Is divided into N subintervals, i.e According to gradient accumulation The function curve of (2) obtains N corresponding subintervals on the time axis, namely Finally, randomly selecting one sampling point from N subintervals of the time axis to obtain N sampling points to form a sampling point set According to the sampling point set Sampling from the original haptic frame set D, marking the sampled haptic frame set as Wherein T represents the number of original haptic frames, Points represented on the vertical axis gradient accumulation Based on gradient accumulation Mapping points of the function curve of (c) on the time axis, Representing the selected N sampling points.
3. The haptic object recognition method based on gradient adaptive sampling and 3D neural network according to claim 2, wherein the multi-time scale downsampling is implemented by: ; Wherein, the Representing the downsampling rate of the mth time scale, A downsampling operation is indicated and is indicated, Representing haptic frame sets A set of haptic frames downsampled at the mth time scale, Representing haptic frame sets Including the number of haptic frames.
4. A method of tactile object recognition based on gradient adaptive sampling and 3D neural networks according to claim 1 or 3, wherein the MR3D-18 network removes a pooling layer from ResNet D-18 network, and the MR3D-18 network adds a Dropout layer behind Res 2 layer of ResNet D-18 network.
5. The method for recognizing a tactile object based on gradient adaptive sampling and 3D neural network according to claim 4, wherein the implementation method of the third step is as follows: Downsampling M time scales to obtain a haptic frame set Respectively sending the extracted multi-time scale characteristics into an MR3D-18 network for characteristic extraction, wherein the extracted multi-time scale characteristics are as follows ; Multi-time scale features The fusion is carried out in a summation mode, and the fused features sequentially pass through an FC layer and a softmax classifier to obtain a prediction classification result as follows: ; Wherein, the Representing a feature fusion operation, which is a sum operation; a full-join convolution operation is represented, Representing a softmax sort operation, Representing haptic frame sets And C represents the number of categories of the target, and the category corresponding to the highest scoring element in S is regarded as the predicted target category.
6. The haptic object recognition method based on gradient adaptive sampling and 3D neural network according to claim 5, wherein the training of the whole multi-time scale 3D convolutional neural network is completed by supervising the predicted classification result through the artificially marked classification result, and the loss function used in the training is a traditional binary cross entropy function.

Description

Haptic object identification method based on gradient self-adaptive sampling and 3D neural network Technical Field The invention relates to the technical field of deep learning, in particular to a haptic object recognition method based on gradient self-adaptive sampling and a 3D neural network. Background Vision and touch are two major ways for robots to perceive the world. Visual perception can only provide the robot with the appearance of objects, physical characteristics of which must be obtained by tactile perception, such as hardness, roughness, texture, etc. The technology development of the robot industry in China is faster, and the tactile object identification is one of key technologies of robot perception, and is also one of core problems of robot participation in automatic production, intelligent driving, virtual reality, intelligent artificial limbs, remote medical treatment, garbage classification and other applications, so that the method has important application value. The robot touch sensing technology has important significance in promoting the development of the intelligent robot and solving the practical problems. Haptic object recognition can be roughly divided into two parts, namely, obtaining haptic data and recognizing object categories based on the haptic data. First, a tactile sensor on the manipulator is used to acquire tactile data (typically pressure data) of an object. Thereafter, a CPU equipped on the robot is used to identify the class of the object from the haptic data, which is also the subject of the present invention. Currently, the dominant methods of haptic object recognition include methods based on feature extraction, methods based on pattern recognition, and methods based on deep learning. Among them, the haptic object recognition method based on deep learning has higher recognition accuracy and robustness, and thus gradually becomes a research hotspot. Currently, haptic object recognition methods based on deep learning are mostly methods that do not use time information. Such methods use single frame haptic data as input, with less computational expense and better real-time. However, the appearance of most objects is often similar in daily life and industrial production, which may reduce the accuracy of identifying objects using such models that model objects using a single frame. Another type of haptic object recognition method using time information uses multi-frame time sequence haptic data to model an object, and has higher recognition accuracy and robustness because more haptic information on the surface of the object is covered. Disclosure of Invention Aiming at the technical problems of information redundancy/loss caused by the use of a uniform sampling strategy in the conventional tactile object recognition model and the technical problem that generalization capability is insufficient for processing tactile data at different grabbing speeds, the invention provides a tactile object recognition method based on gradient self-adaptive sampling and a 3D neural network. In order to achieve the purpose, the technical scheme of the invention is realized by a haptic object identification method based on gradient self-adaptive sampling and a 3D neural network, which comprises the following steps: the method comprises the steps of carrying out self-adaptive selection on original touch frames by using a gradient self-adaptive sampling strategy to obtain a touch frame set with rapid gradient change, and enabling most of sampled touch frames to contain more abundant touch information, so that the problem of data redundancy or data missing of the traditional sampling strategy is solved. Step two, sampling the touch frame set obtained in the step one in a multi-time scale manner; And thirdly, respectively extracting the characteristics of the downsampled haptic frame set by using an MR3D-18 network to obtain the characteristics in different time scales, fusing the characteristics in different time scales, and identifying the category of the object according to the fused characteristics to obtain a prediction classification result. The multi-time scale 3D convolutional neural network model can improve the generalization capability of the existing tactile object recognition model on the tactile data generated at different grabbing speeds, and effectively improve the recognition precision of the grabbed objects. Preferably, the method for adaptively selecting the haptic frames by the gradient adaptive sampling strategy comprises the following steps: S1, performing matrix subtraction on adjacent original haptic frames in input haptic data to obtain gradient absolute value matrixes of two adjacent haptic frames at the moment t, and performing matrix subtraction on an initial haptic frame to obtain gradient absolute value matrixes of the haptic frames at the initial moment; s2, normalizing the absolute value matrix of the gradient in the time dimension, and calculating the normal