Search

CN-121973235-A - Multi-mode tactile sensing and object recognition system and method for manipulator

CN121973235ACN 121973235 ACN121973235 ACN 121973235ACN-121973235-A

Abstract

The invention relates to a system and a method for multi-mode touch sensing and object identification of a manipulator, wherein the system comprises a manipulator actuator, a multi-channel flexible touch sensor array, a signal acquisition module and a processing terminal, wherein a flexible touch sensor in the multi-channel flexible touch sensor array comprises a packaging layer, an electrode layer, a conductive sensitive layer and an elastomer layer, the multi-channel flexible touch sensor array contacts a target object, a microstructure array is arranged on one side of the packaging layer and/or the elastomer layer, which faces the conductive sensitive layer, and the microstructure array and the conductive sensitive layer form an interlocking/embedding interface. Compared with the prior art, the method has the advantages of improving the recognition robustness under the complex surface condition, reducing the weight prediction error and the like.

Inventors

  • ZHANG YAN
  • CHEN KAI
  • YANG DONGYE
  • LI FANGJIE
  • Peng Binxiang

Assignees

  • 上海工程技术大学

Dates

Publication Date
20260505
Application Date
20260330

Claims (10)

  1. 1. A manipulator multi-mode touch sensing and object recognition system is characterized by comprising a manipulator actuator (6), a multi-channel flexible touch sensor array, a signal acquisition module (10) and a processing terminal (11), wherein the flexible touch sensor in the multi-channel flexible touch sensor array comprises a packaging layer (1), an electrode layer (2), a conductive sensitive layer (3) and an elastomer layer (4), and the multi-channel flexible touch sensor array contacts a target object (5); And a microstructure array (13) is arranged on one side of the packaging layer (1) and/or the elastomer layer (4) facing the conductive sensitive layer (3), and the microstructure array (13) and the conductive sensitive layer (3) form an interlocking/jogged interface.
  2. 2. The manipulator multi-modal haptic sensing and object recognition system of claim 1, wherein the multi-channel flexible haptic sensor array further comprises a set of strain channels (8) and a set of pressure channels (7), the flexible haptic sensor being connected to the set of strain channels (8) and the set of pressure channels (7).
  3. 3. The multi-mode tactile sensing and object recognition system of claim 2, wherein the strain channel group (8) and the pressure channel group (7) are connected with a Flexible Printed Circuit (FPC) (9), the Flexible Printed Circuit (FPC) (9) is connected with a signal acquisition module (10), the signal acquisition module (10) is connected with a processing terminal (11), and a multi-task time sequence model (12) is arranged in the processing terminal (11).
  4. 4. A manipulator multimodal tactile sensation and object recognition system according to claim 1, wherein the microstructure array (13) is a periodic array structure.
  5. 5. The system of claim 4, wherein the micro-structured array (13) is any one of a micro-pyramid array, a micro-needle array, a micro-pillar array, a hemispherical array, or a bionic texture array.
  6. 6. The multi-modal tactile sensing and object recognition system of claim 4 wherein the conductive sensitive layer (3) is a conductive hydrogel sensitive layer.
  7. 7. A method for multi-modal haptic perception and object recognition of a manipulator, characterized in that a system according to any one of claims 1-6 is used, the method comprising the steps of: S1, controlling a manipulator executor (6) to execute a grabbing action sequence comprising initial homing, closed grabbing, stable holding and release resetting, and acquiring a multichannel original resistance time sequence signal in the grabbing process based on a signal acquisition module (10); s2, preprocessing the multichannel original resistance time sequence signal in a processing terminal (11) to obtain a preprocessed resistance time sequence signal; S3, extracting pressure characteristics, strain characteristics and texture related characteristics from the preprocessed resistance time sequence signals, and fusing to form a fused characteristic set; S4, inputting the fusion feature set into a pre-trained multi-task time sequence model (12) to output an object type recognition result and a weight prediction value.
  8. 8. The method of claim 7, wherein the texture-related features include one or more of high frequency fluctuation features, frequency domain energy features, and time-frequency domain features of the pressure signal.
  9. 9. The method for multi-modal tactile sensation and object recognition by a manipulator according to claim 7, wherein the specific steps of fusing to form a fused feature set are: and aligning and fusing the pressure characteristic, the strain characteristic and the texture related characteristic according to a unified time window to obtain a fused characteristic set.
  10. 10. The method of claim 7, wherein the preprocessing includes synchronous alignment, acquisition of a baseline of a resting stage before the start of grabbing, and performing baseline correction, normalization processing, filtering noise reduction, outlier rejection, and drift correction.

Description

Multi-mode tactile sensing and object recognition system and method for manipulator Technical Field The invention relates to the technical field of intelligent robot touch sensing and flexible sensing, in particular to a system and a method for multi-mode touch sensing and object identification of a manipulator. Background The manipulator needs to sense the information such as contact pressure, joint deformation, friction/micro slip generated by contact with the surface of an object in the grabbing and operating tasks so as to realize stable grabbing, grabbing state evaluation and object attribute sensing. The existing grabbing sensing system mostly adopts a pressure sensor or a bending/strain sensor to collect single or small quantity of modal signals, and then combines machine learning to perform object identification or weight estimation. When surface texture differentiation is involved, conventional practice often relies on additional texture-specific sensors or the addition of active sweeping/scanning actions to excite texture signals, followed by frequency/time-frequency analysis of the high frequency components to accomplish identification. However, the surface texture of the actual gripping object has differences of smoothness, roughness, fabric texture and the like, and the texture differences can cause contact friction, micro-slippage and signal fluctuation characteristics to be obviously different. Depending on the pressure or joint gesture signals, it is often difficult to stably distinguish different objects under complex surface conditions, and weight prediction is susceptible to changes in gripping contact conditions. Conventional texture-aware schemes typically rely on additional texture-specific sensors or active sweeping actions to excite texture signals, increasing system complexity, which is detrimental to deployment in conventional grabbing actions. Disclosure of Invention The invention aims to provide a system and a method for multi-mode touch perception and object identification of a manipulator, which can enable a sensor to simultaneously obtain pressure response, deformation strain response and texture related fluctuation signals caused by natural micro friction/micro slip without an additional texture sensor in a conventional grabbing process, and realize synchronous output of object type identification results and weight prediction values under single grabbing time sequence input through multi-mode feature fusion and multi-task time sequence models, so that identification robustness under complex surface conditions is improved, and weight prediction errors are reduced. The aim of the invention can be achieved by the following technical scheme: the system comprises a manipulator actuator, a multichannel flexible touch sensor array, a signal acquisition module and a processing terminal, wherein the flexible touch sensor in the multichannel flexible touch sensor array comprises a packaging layer, an electrode layer, a conductive sensitive layer and an elastomer layer, and the multichannel flexible touch sensor array contacts a target object; And the packaging layer and/or the elastomer layer is/are provided with a microstructure array on one side of the conductive sensitive layer, and the microstructure array and the conductive sensitive layer form an interlocking/jogging interface. Further, the multi-channel flexible touch sensor array further comprises a strain channel group and a pressure channel group, and the flexible touch sensor is connected with the strain channel group and the pressure channel group. Further, the strain channel group and the pressure channel group are connected with a flexible printed circuit board FPC, the flexible printed circuit board FPC is connected with a signal acquisition module, the signal acquisition module is connected with a processing terminal, and a multitasking time sequence model is arranged in the processing terminal. Further, the microstructure array is a periodic array structure. Further, the microstructure array is any one of a micro pyramid array, a micro needle array, a micro column array, a hemispherical array or a bionic texture array. Further, the conductive sensitive layer is a conductive hydrogel sensitive layer. A multi-mode touch sensing and object identification method of a manipulator adopts the system, and the method comprises the following steps: S1, controlling a manipulator executor to execute a grabbing action sequence comprising initial homing, closed grabbing, stable holding and release resetting, and acquiring a multichannel original resistance time sequence signal in the grabbing process based on a signal acquisition module; S2, preprocessing the multichannel original resistance time sequence signal in a processing terminal to obtain a preprocessed resistance time sequence signal; S3, extracting pressure characteristics, strain characteristics and texture related characteristics from the preprocessed resistance time se