CN-122008242-A - Self-adaptive bow-transporting control method for violin playing robot

CN122008242ACN 122008242 ACN122008242 ACN 122008242ACN-122008242-A

Abstract

A self-adaptive bow-transporting control method for a violin playing robot relates to the field of intelligent robot control. The invention aims to solve the problem of poor control accuracy of the existing bow-carrying control method of the violin playing robot. And acquiring three-axis horizontal force, three-axis rotation moment, arm end position, arm end gesture, arm joint angle and visual information in the process of carrying out bow in real time in the robot performance, predicting a plurality of robot bow carrying action sequences at continuous moments in the future or carrying out weighted fusion on bow carrying actions at the same moment in the plurality of predicted robot bow carrying action sequences at continuous moments in the future, generating a control instruction, and adjusting the actions of the robot performance at the corresponding moments. The invention is used for adjusting the motion of the bow in real time.

Inventors

LI JIANRONG
LI MAOGUO
LI SHIYU
ZHANG NING
WEI CHENG
Qiao Shunan
SUN CHENGXIN
WANG ZIXUAN
WANG LIJIAN
ZHENG HAOTIAN
Yin Xiuyang
Ma Zichun

Assignees

哈尔滨工业大学

Dates

Publication Date: 20260512
Application Date: 20260402

Claims (9)

1. The self-adaptive bow-transporting control method of the violin playing robot is characterized in that the violin playing robot is in an anthropomorphic configuration and is used for simulating a human to clamp and play the violin; The control method is characterized by comprising the following steps: Step 1, collecting historical action samples played by experts, extracting visual information, triaxial horizontal force in the process of carrying out bow, triaxial rotating moment, arm joint angle, arm tail end position and arm tail end gesture in the process of carrying out bow in the samples at the same moment, obtaining hidden characteristics of a contact state according to the triaxial horizontal force and the triaxial rotating moment at each moment, and labeling string type and bow method type labels at each moment; Taking the triaxial horizontal force and triaxial rotation moment in the bow-transporting process at each moment as input data of 1 number one sample, and taking string type and bow method type labels at the moment as output data of 1 number one sample; The contact state hidden characteristic, the triaxial horizontal force, the triaxial rotation moment, the arm end position, the arm end gesture, the arm joint angle and the visual information at each moment are taken as input data of 1 second sample, and the arm joint angle at a plurality of continuous moments in the future is taken as output data of 1 second sample; Step 2, training the contact state encoder by using the input data and the output data of the first sample to obtain a pre-trained contact state encoder, Training a diffusion strategy network by using input data and output data of a second sample to obtain a pre-trained diffusion strategy network; Step 3, at the first At moment, acquiring three-axis horizontal force, three-axis rotation moment, arm tail end position, arm tail end gesture, arm joint angle and visual information in the process of carrying out bow in the performance of the robot; Inputting the collected triaxial horizontal force and triaxial rotation moment to a pre-trained contact state encoder to predict the string type and bow method type, extracting the contact state hidden characteristic generated by the contact state encoder in the prediction process, An initial value of 1; hidden features for contact status, and in the first The three-axis horizontal force, the three-axis rotation moment, the arm tail end position, the arm tail end gesture, the arm joint angle and the visual information which are acquired at the moment are input into a pre-trained diffusion strategy network, and a plurality of robot bow-moving action sequences at continuous moments in the future are predicted, wherein the bow-moving action sequences consist of a plurality of robot arm joint angles at the continuous moments; Step 4, generating a control instruction according to a predicted robot bow-moving action sequence at the future moment, and adjusting the action of the robot performance at the corresponding moment; Step 5, order = +1, Collecting three-axis horizontal force, three-axis rotation moment, arm tail end position, arm tail end gesture, arm joint angle and visual information in the process of carrying out bow in the performance of the robot; Inputting the collected triaxial horizontal force and triaxial rotation moment to a pre-trained contact state encoder to predict the string type and bow method type, and extracting contact state hidden features generated by the contact state encoder in the prediction process; hidden features for contact status, and in the first The three-axis horizontal force, the three-axis rotation moment, the arm tail end position, the arm tail end gesture, the arm joint angle and the visual information which are acquired at the moment are input into a pre-trained diffusion strategy network, and a plurality of robot bow-moving action sequences at continuous moments in the future are predicted, wherein the bow-moving action sequences consist of a plurality of robot arm joint angles at the continuous moments; Extraction of the first Motion of moving bow at +1 moment, and extracting the first from motion sequence of moving bow predicted at the preface moment The motion of the bow at time +1, all the first ones to be extracted The motion of the bow at the moment +1 is weighted and fused to obtain the fused first segment A bowing motion at +1, generating a control command according to the bowing motion, and adjusting the robot playing motion at the moment; Step 6, judging Whether equal to the preset moment, and if so, and if not, executing the step 5.
2. The adaptive bow-handling control method of violin playing robot according to claim 1, wherein the fused first The arching motion at time +1 is expressed as: , In the formula, For the time after fusion The motion of the bow is +1, To the first pair The number of predictions that contributed to time +1, Is the first The weight of the secondary prediction result is calculated, Is the first Correspondence in secondary predictions The action of the moment in time is that, , Is a smoothing coefficient.
3. The adaptive bow-handling control method of a violin playing robot of claim 1, wherein the contact state encoder comprises a convolution feature extraction layer, a time sequence attention layer, a global averaging pooling layer, a full connection mapping layer, a full connection layer and an activation function layer; The convolution feature extraction layer is used for extracting local time domain features of horizontal force in the bow conveying process and rotation moment in the bow conveying process and sending the local time domain features to the time sequence attention layer; the time sequence attention layer is used for giving different weights to the extracted local time domain features and sending the weighted local time domain features to the global average pooling; Global average pooling is used for carrying out dimension reduction and feature fusion on the local time domain features endowed with weight and sending the local time domain features to a full-connection mapping layer; the full-connection mapping layer is used for mapping the fused features into the low-dimensional contact state hidden features and sending the low-dimensional contact state hidden features to the full-connection layer; The full-connection layer is used for mapping the contact state hidden characteristics into multidimensional vectors and sending the multidimensional vectors to the activation function layer; and the activation function layer is used for normalizing the multidimensional vector into probability distribution and outputting the prediction results of the string type and the bow method type.
4. The adaptive bow-handling control method of a violin playing robot according to claim 1, wherein the contact state hidden feature is expressed as: , In the formula, Representing the parameters as Is provided with a bowstring contact state encoder, Indicating time of day The corresponding contact state hidden feature vector is used, Representing the dimensions of the hidden feature of the contact state, To take time of The triaxial horizontal force and triaxial rotational moment in the end timing observation window.
5. The adaptive bow-handling control method of a violin playing robot according to claim 1, wherein the step 1 of taking the arm joint angles at a plurality of successive moments in the future as output data of a number two sample and the step 2 of training a diffusion strategy network by using input data and output data of the number two sample comprise adding gaussian noise to the arm joint angles at a plurality of successive moments in the future to obtain a noisy action sequence.
6. The adaptive bowing control method of a violin playing robot according to claim 5, wherein after predicting the robot bowing motion sequence of a plurality of successive moments in the future in step 3 and step 5, includes denoising the robot bowing motion sequence of a plurality of successive moments in the future to obtain a clean motion sequence.
7. The adaptive bow-handling control method of violin playing robot of claim 5, wherein the noisy motion sequence Expressed as: , In the formula, The number of diffusion steps is indicated, Represent the first The step-wise corresponding cumulative noise dispatch coefficient, Representing standard gaussian noise in the same dimension as the action sequence, For a sequence of arm joint angles at a number of successive moments in the future, Is an identity matrix.
8. The adaptive bow-handling control method of a violin playing robot according to claim 1, wherein three-axis horizontal force in the bow-handling process and three-axis rotational moment in the bow-handling process are acquired by using six-dimensional force/moment sensors.
9. The adaptive bow-handling control method of a violin playing robot according to claim 1, wherein the visual information is collected by using a depth camera.

Description

Self-adaptive bow-transporting control method for violin playing robot Technical Field The invention relates to the field of intelligent control of robots. In particular to a self-adaptive bow-transporting control method of a violin playing robot based on a diffusion strategy and bowstring contact coding. Background As the robot technology continues to expand from industrial fields to scenes of services, education, art, etc., performance robots with artistic expression become the foregrounds of interdisciplinary research. The violin has extremely high requirements on the flexible operation and environment interaction capability of a robot due to the fact that complicated bow-transporting skills and accurate force control of human arms are required to be simulated, and the violin is an ideal platform for verifying the fine operation capability. The key operation of violin playing, namely the bow carrying, is essentially a typical contact rich and flexible operation task. In the playing process, continuous and changeable contact force interaction exists between the string and the string, and the accurate matching of the bow pressure, the bow speed and the contact point directly determines the tone quality. Different strings (G, D, A, E strings) show significantly different mechanical properties due to the differences of string diameters, tensions and materials, and different bow methods (bow dividing, connecting, bowing and the like) correspond to distinct force-speed-displacement coupling modes. Therefore, the bow-handling control strategy needs to be able to sense the current bowstring contact state and adaptively adjust the bow-handling trajectory and force accordingly. However, the existing bow-carrying control method of the violin playing robot has obvious technical defects that actual playing requirements are difficult to meet, the traditional PID control is used for correcting joint moment in real time by setting target bow pressure/bow speed and utilizing a proportional-integral-differential feedback link to enable actual contact force to track a target value, and the traditional impedance control strategy is used for enabling the tail end of the mechanical arm to present flexible response characteristics when contacting strings by setting expected rigidity and damping parameters. However, the two traditional controls all need to manually preset control parameters including PID gain, target bow pressure, expected rigidity, damping and the like, and the parameters need to be manually set according to specific string types, bow method types and force conditions. Therefore, a group of parameters can only be adapted to a specific playing condition (including different string diameters, tensions and materials of the strings G/D/A/E), bow method types (different requirements of bow pressure and bow speed by dividing, connecting, bowing, jumping and the like), force levels (different requirements of weak playing and strong playing on bow pressure) and the like), when the string types or the bow method types are changed, the original parameters are not applicable any more, manual resetting is needed to realize accurate control, so that the fixed parameter combination cannot be adapted in real time during playing, however, in actual playing, the switching of the strings and the bow method is frequent and continuous, the traditional method cannot automatically adapt to the changes during playing, and setting parameters by means of a modeling mode is extremely difficult, and parameters tend to be not accurate enough due to factors such as nonlinear friction, bow Mao Rouxing deformation and the like in the bow string contact process. In recent years, imitation learning has made remarkable progress in the field of robot operations. The Diffusion Policy (Diffusion Policy) models probability distribution of the action sequence through a conditional Diffusion model, and strong multi-mode action distribution modeling capability and generalization performance are shown in the operation tasks with rich contact. However, the diffusion strategy has not been applied in fine operation scenes such as music performance robots, which require force sense sensing and compliant control. In summary, the existing bow-carrying control method of the violin playing robot cannot adapt to the dynamic change of the bowstring contact state due to the problems of difficult modeling, fixed parameters, lack of self-adapting capability and the like, so that the control accuracy is poor, high-quality continuous playing under multiple working conditions is difficult to realize, and a brand-new self-adapting bow-carrying control method is needed to solve the technical pain. Disclosure of Invention The invention aims to solve the problem of poor control accuracy of the existing violin playing robot bow-carrying control method, and provides a self-adaptive bow-carrying control method of the violin playing robot. The self-adaptive bow-transporting