CN-122016093-A - Stress detection and analysis method for visual touch sensor

CN122016093ACN 122016093 ACN122016093 ACN 122016093ACN-122016093-A

Abstract

The invention discloses a stress detection and analysis method for a visual touch sensor, relates to the field of sensors, and is used for realizing comprehensive high-precision stress detection and analysis on the top surface and the side surface of the visual touch sensor. The sensor comprises a magnetic film covering the top surface and the side surface, wherein the magnetic film is made by mixing magnetic powder and silicon rubber, a bracket supports the magnetic film, a magnetic display card and a light source are arranged in the bracket, magnetic induction data and development data of stress of the magnetic film are collected by a visual touch collector, and the stress position and/or the stress size are analyzed by a multi-mode pressure detection model. The invention has high stress detection sensitivity, can realize comprehensive detection of the top surface and the side surface, and has high detection precision.

Inventors

ZHANG RUIRUI
MA XUESI
LV TONG
ZHENG KAIYUAN
HUANG TAO

Assignees

成都人形机器人创新中心有限公司

Dates

Publication Date: 20260512
Application Date: 20251217

Claims (10)

1. A method for detecting and analyzing stress of a visual touch sensor is characterized in that, The visual tactile sensor includes: a magnetic thin film covering the top and side regions; a support for supporting the magnetic thin film; The magnetic display card is arranged on the bracket and responds to the stress of the magnetic film to develop; The light source is arranged in the bracket; the visual touch collector is used for collecting magnetic induction data of the stress of the magnetic film and development data of the magnetic display card; The stress detection and analysis method comprises the following steps: and inputting the magnetic induction data and the development data into a trained multi-mode pressure detection model, and outputting a stress position and/or stress magnitude by the multi-mode pressure detection model.
2. The visual touch sensor force detection analysis method of claim 1, wherein the multi-modal pressure detection model is configured to: Preprocessing the input magnetic induction data and development data respectively; Extracting magnetic induction characteristics from the preprocessed magnetic induction data, and extracting development characteristics from the preprocessed development data; Fusing the magnetic induction characteristic and the development characteristic to obtain a fused characteristic; And deducing the stress position and/or the stress size according to the fusion characteristics.
3. The method of claim 2, wherein preprocessing the input magnetic induction data and the development data respectively comprises: performing standardization processing on the magnetic induction data to convert the magnetic induction data into a range which can be processed by the multi-mode pressure detection model; and controlling the resolution ratio of the development data to a preset size, carrying out gray processing on the development data, and finally normalizing the pixel value of the development data to be within the range of [0,1 ].
4. A method of visual tactile sensor stress detection analysis according to claim 3, wherein said method of normalization processing comprises: The magnetic induction data is controlled to be within a desired range by subtracting the mean value or dividing the magnetic induction data by the standard deviation.
5. The visual touch sensor force detection analysis method of claim 2, wherein extracting magnetic induction characteristics from the preprocessed magnetic induction data comprises: Performing feature extraction on the preprocessed magnetic induction data layer by adopting a plurality of full-connection layers to obtain magnetic induction features; Extracting development features from the preprocessed development data, comprising: performing feature extraction on the preprocessed development data layer by adopting a plurality of convolution layers, wherein each convolution layer performs dimension expansion on the input data; and performing activation treatment and pooling treatment on the features after the convolution treatment to obtain the development features.
6. The visual touch sensor force detection analysis method of claim 2, wherein fusing the magnetic induction feature and the development feature to obtain a fused feature comprises: And fusing the magnetic induction characteristic and the development characteristic by adopting a cross-modal fusion layer, wherein: The cross-mode fusion layer is used for splicing the magnetic induction characteristics and the development characteristics, then the SE module is used for carrying out attention fusion on the spliced characteristics, and finally the full-connection layer is used for carrying out dimension compression on the characteristics output by the SE module to obtain the fusion characteristics.
7. The visual touch sensor force detection analysis method of claim 2, wherein inferring a force location and/or a force magnitude from the fusion features comprises: The method comprises the steps of adopting an output layer to infer the fusion characteristics, adopting two full-connection layers to conduct dimension compression on the fusion characteristics by the output layer, compressing characteristic dimensions to the number of output channels, enabling the number of the output channels to be 1 when only the stress magnitude is needed to be inferred, outputting the stress magnitude, enabling the number of the output channels to be 3 when the stress position and the stress magnitude are needed to be inferred at the same time and are represented through two-dimensional coordinates, and respectively outputting an x coordinate, a y coordinate and the stress magnitude.
8. The visual touch sensor force detection analysis method of any one of claims 1-7, wherein the multi-modal pressure detection model training method comprises: Acquiring a training sample set; the training sample set is divided into a training set and a verification set according to a preset proportion, and the training set is sequentially input into an initialized multi-mode pressure detection model for iterative training; And after the stopping condition of model training is reached, saving model parameters.
9. The method of claim 8, wherein obtaining a training sample set comprises: Pressing on the magnetic film of the visual touch sensor to respectively acquire magnetic induction data and developing data during pressing and pressing stress positions and/or stress sizes, and associating the magnetic induction data, the developing data and the stress positions and/or stress sizes into a group of training sample data, wherein the stress positions and/or stress sizes are tag values of the magnetic induction data and the developing data; and merging all the training sample data to obtain a training sample set.
10. The visual touch sensor force detection analysis method of claim 9 wherein the loss function of the multi-modal pressure detection model is trained as a weighted MSE loss function designed to, when the final detection objective is to detect both force location and force magnitude: ; wherein, loss represents the model Loss, The weight of the ith channel is represented, and the total stress position x coordinate, the stress position y coordinate and the stress size are 3 channels; And Respectively representing the predicted value and the label value of the jth training sample data in the ith channel, and m represents the number of training sample data.

Description

Stress detection and analysis method for visual touch sensor The application relates to a split application of Chinese application patent application with the application number 202511906273.7, the application date 2025, the 12 th month and 17 th day and the application name of 'a visual touch sensor, electronic skin and robot'. Technical Field The invention relates to the field of sensors, in particular to a stress detection and analysis method for a visual touch sensor. Background The electronic skin is a flexible electronic device imitating human skin sensing function, the core working principle is that external stimulus signals are captured through a sensor, converted and optimized through a signal processing module, and finally identifiable signals are output, so that accurate detection of physical quantities such as pressure, temperature, humidity, strain and the like is realized, and part of the electronic skin even has a stimulus feedback function. In recent years, with the development of artificial intelligence and robotics, electronic skin capable of accurately detecting physical quantities such as pressure, temperature, humidity, strain and the like is becoming more and more important, and has a broad application prospect. Especially, based on the perception mode of combining the visual touch sensor with magnetic induction, the problems of missing 3D position information receiving of a marked point, limited camera frequency and response speed and missing state information perception capability of an operation object under the condition of a monocular camera of the current visual touch sensor are solved, and the high-precision force measurement, high response and proximity function of the visual touch sensor are realized. However, in the sensor structure, the surface coating is arranged on the surface of the transparent silica gel to isolate an external light source, reconstruct the surface texture of a perceived object, and the deformation information of the surface coating is received by a camera to realize information acquisition. The presence of the surface coating increases the overall thickness of the magnetic film and affects the sensitivity of the force detection. Furthermore, the magnetic films of known electronic skin structures are generally planar structures on which forces cannot be detected when contact occurs on the sides of the electronic skin. Disclosure of Invention The invention aims to provide a method for detecting and analyzing the stress of a visual touch sensor, aiming at all or part of the problems, so as to realize comprehensive high-precision stress detection and analysis on the top surface and the side surface of the visual touch sensor. The technical scheme adopted by the invention is as follows: A method of visual tactile sensor force detection analysis, the visual tactile sensor comprising: a magnetic thin film covering the top and side regions; a support for supporting the magnetic thin film; The magnetic display card is arranged on the bracket and responds to the stress of the magnetic film to develop; The light source is arranged in the bracket; the visual touch collector is used for collecting magnetic induction data of the stress of the magnetic film and development data of the magnetic display card; The stress detection and analysis method comprises the following steps: and inputting the magnetic induction data and the development data into a trained multi-mode pressure detection model, and outputting a stress position and/or stress magnitude by the multi-mode pressure detection model. Optionally, the multi-modal pressure detection model is configured to: Preprocessing the input magnetic induction data and development data respectively; Extracting magnetic induction characteristics from the preprocessed magnetic induction data, and extracting development characteristics from the preprocessed development data; Fusing the magnetic induction characteristic and the development characteristic to obtain a fused characteristic; And deducing the stress position and/or the stress size according to the fusion characteristics. Optionally, the preprocessing the input magnetic induction data and the development data respectively includes: performing standardization processing on the magnetic induction data to convert the magnetic induction data into a range which can be processed by the multi-mode pressure detection model; and controlling the resolution ratio of the development data to a preset size, carrying out gray processing on the development data, and finally normalizing the pixel value of the development data to be within the range of [0,1 ]. Optionally, the method of normalizing includes: The magnetic induction data is controlled to be within a desired range by subtracting the mean value or dividing the magnetic induction data by the standard deviation. Optionally, extracting magnetic induction characteristics from the preprocessed magnetic induction data includes: