Search

CN-121980142-A - Remote operation data end side self-screening method for industrial robot with body based on shadow reasoning

CN121980142ACN 121980142 ACN121980142 ACN 121980142ACN-121980142-A

Abstract

The invention discloses a remote operation data end side self-screening method of an industrial robot with a body based on shadow reasoning. The method comprises the steps of 1, constructing a cross-modal data tuple flow of visual frame-instruction-action based on a visual frame time stamp, discretely mapping continuous action vectors into real action Token vectors, 2, constructing a shadow reasoning model, generating a prediction probability distribution matrix based on the cross-modal data tuples, 3, calculating a prediction deviation factor and an endogenous uncertainty factor according to the real action Token vectors and the prediction probability distribution matrix, fusing to obtain frame-level value, packaging the data tuples with the frame-level value continuously meeting preset conditions into a preliminary screening sample fragment, and 4, calculating feature differences between the preliminary screening sample fragment and reserved fragments, and determining reservation or discarding of the preliminary screening sample fragment through double-threshold judgment. The invention can realize low-cost and high-efficiency remote control data screening at the end side and improve the data quality of the intelligent model training.

Inventors

  • WANG YUEFEI
  • Zhou Yaman
  • YUAN YICHEN
  • ZHAO RUIHUA
  • Yi Zhejin
  • YIN QUANSHENG

Assignees

  • 合肥工业大学

Dates

Publication Date
20260505
Application Date
20260409

Claims (7)

  1. 1. The remote operation data end side self-screening method of the industrial robot based on shadow reasoning is characterized by comprising the following steps of: step 1, taking a visual frame time stamp as a unified time reference, and collecting visual observation of a robot Instruction embedding Normalized motion vector Constructing a unified cross-modal data tuple stream containing vision, instructions, and actions ; Step 2, constructing a mapping rule of motion vector to discrete motion Token vectorization, and continuously normalizing the motion vector Mapping to corresponding real action Token vector Real action Token sequence { is generated in time sequence }; Step 3, constructing a shadow reasoning model based on a history context window As input, the current time step is output via a dual layer matrix mapping process The following predictive probability distribution matrix ; Step 4, constructing an evaluation model and inputting a time step Corresponding real action Token vector And predictive probability distribution matrix Calculating a prediction bias factor And endogenous uncertainty factor ; Step 5, utilizing the prediction deviation factor And endogenous uncertainty factor Building frame-level valuation And in the cross-modal data tuple flow Packaging fragments to be saved by upper combined continuous hit mechanism Calculating the fragments Feature difference factor relative to a set of retained segments Determining whether to treat the fragment by dual threshold decision And its characterization vector writes into the end side memory space.
  2. 2. The method for remote operation data end-side self-screening of an industrial personal robot based on shadow reasoning according to claim 1, wherein the step1 comprises: step 1.1, taking a visual frame time stamp output by a visual sensor as a unified time reference, and setting visual frame frequency as Visual frame index Corresponding time steps are Wherein , Time steps in remote operation data acquisition; Step 1.2, obtaining a high-frequency original absolute pose state flow of a robot tail end controller , wherein, Representing an end position vector; Representing an end gesture quaternion and satisfying ; Indicating the end jaw state; sampling a time stamp for the controller; step 1.3, constructing a visual frame sequence under a unified time reference And instruction stream ; Step 1.3.1, acquiring a visual frame sequence output by an airborne visual sensor under a unified time reference , wherein, To be in a time step The obtained visual observation; step 1.3.2, acquiring instruction stream Taking the current time step The last updated instruction is embedded as the current instruction , The time stamp is updated for the instruction, Embedding a representation for a corresponding instruction vector; Step 1.4, visual observation under unified time reference Instruction embedding And normalized motion vector Index by time Performing tuple association to construct a unified cross-modal data tuple stream: 。
  3. 3. the method for remote operation data end-side self-screening of an industrial personal robot based on shadow reasoning as set forth in claim 2, wherein the step 1.2 comprises the steps of: step 1.2.1 streaming in raw State The search of the middle part satisfies Using equation (1) to calculate each time step Is increased by: (1) in the formula (1), the components are as follows, In order to normalize the temporal interpolation coefficient, Respectively an end position increment vector, an end gesture quaternion increment and a clamping jaw state increment, , , ; Is quaternion multiplication; spherical linear interpolation representing unit quaternion Conversion to Euler angle increment to obtain rotation increment vector ; Step 1.2.2 defining the original motion vector Normalizing by using a truncated mapping function to generate a normalized motion vector The calculation formula is as follows: (2) In the formula (2), the amino acid sequence of the compound, Representing time steps Lower normalized motion vector Is the first of (2) A dimension component; Representing a truncation function; representing an original motion vector Is the first of (2) A dimensional physical quantity component; And (3) with Respectively represent the first Maintaining a minimum cutoff threshold and a maximum cutoff threshold of the physical quantity; For the total dimension number of the motion vector, the components of each dimension are combined according to the dimension sequence to obtain a standardized motion vector , wherein, 。
  4. 4. The remote operation data end-side self-screening method of the industrial personal robot based on shadow reasoning as set forth in claim 3, wherein the step 2 comprises the following steps: Step 2.1 giving the motion vector Total dimension Number of discrete symbols per dimension Defining a discrete action domain: Wherein, the Representing a discrete action Token vector; represent the first A discrete Token index for each action dimension; Step 2.2, for the first Defining quantitative binning boundary sequences for individual action dimensions Wherein, the method comprises the steps of, , , The boundary sequence The mapping relation between the physical numerical value interval of the dimension and the discrete Token index is determined; Is the first The discrete Token indexes after the sub-boxes are quantized by the action dimensionality; Step 2.3 for time step Is the first of (2) Dimensional action component At the binning boundary sequence The corresponding real discrete Token index is determined according to the formula (3) by interval retrieval And combining the index of each dimension to obtain time step Is a true action Token vector of (a) Real action Token sequence { is constructed according to time sequence }, (3)。
  5. 5. The method for remote control data end-side self-screening of an industrial personal robot based on shadow reasoning as set forth in claim 4, wherein the step 3 comprises the steps of: Step 3.1 for each time step under the unified time reference Reading current observation and instruction from the cross-modal tuple flow to obtain Construction length is History context window of (a) Wherein the first is in the window Each time sequence element , Constructed according to formula (4): (4) In the formula (4), the amino acid sequence of the compound, Is a preset filling item, is configured as a zero filling item Or first item ; Step 3.2, the deployment parameters are Light-weight shadow inference model of (3) Output space and Maintain consistency for any effective time Definition of Defining window splice fusion vectors Wherein The dimensions of the visual features are represented, The representation represents the instruction embedding dimension, For the time-series splicing operator, Extracting a network for visual characterization, outputting a predictive probability distribution matrix using equation (5): (5) in the formula (5), the amino acid sequence of the compound, Representing a normalized exponential mapping function in each action dimension, Is a preset activation function; and cross-modal fusion matrix Hidden layer bias vector Motion distribution projection matrix Output offset vector , Representing hidden layer feature dimensions; Representing rearrangement as A matrix; step 3.3, predicting probability distribution matrix Confidence degree distribution for representing discrete Token values of each action dimension of the model in the current state: (6) In the formula (6), the amino acid sequence of the compound, Expressed in time steps Under the condition of Dimension action Token takes value And satisfies the row normalization constraint: 。
  6. 6. the method for remote control data end-side self-screening of an industrial personal robot based on shadow reasoning according to claim 5, wherein the step 4 comprises the following steps: step 4.1, input time step Under the true action Token vector Dimension-by-dimension probability distribution matrix output by shadow reasoning model ; Step 4.2, the evaluation model calls a prediction deviation evaluation operator to construct a prediction deviation factor (7) (7) In the formula (7), the amino acid sequence of the compound, Representing time steps Lower (th) The value of the dimension true action Token is taken, Expressed in time steps Under the condition of the first The dimension action Token takes the value of the predicted probability of the real action corresponding to the Token, Is a preset minimum positive number; step 4.3, the evaluation model calls an endophytic uncertainty evaluation operator to construct an endophytic uncertainty factor (8) (8) In the formula (8), the amino acid sequence of the compound, Represent the first Dimension action Token takes value Is used for the prediction probability of (1).
  7. 7. The method for remote control data end-side self-screening of an industrial personal robot based on shadow reasoning as set forth in claim 6, wherein the step 5 comprises the steps of: step 5.1, based on the prediction bias factor And endogenous uncertainty factor Building frame-level valuation (9) In the formula (9), the amino acid sequence of the compound, In order to predict the weight of the bias factor, For the endogenous uncertainty factor weights, 、 , ; Step 5.2, performing continuous hit detection based on the frame-level value, packaging the data fragment to be saved and defining the fragment value; Step 5.2.1, setting a frame level scoring threshold And continuous hit Length parameter If the current time is Satisfy the condition of continuity Frame-level value of a frame exceeds a threshold And at the previous time Not meeting the above-mentioned continuous conditions Frame condition, then the tuple stream is generated using equation (10) Upper mark encapsulation segment ; (10) In the formula (10), the amino acid sequence of the compound, , And (3) with Is fixed at front and back Wen Changdu; Step 5.2.2 definition of fragments The segment value of (2) is the maximum value of the intra-segment frame level scores: ; step 5.3, deploying fixed-parameter lightweight observation characterization extraction network on end-side computing equipment Visual observation of each frame in a segment Extracting low-dimensional token vectors , Is that Is a segment of the segment by the formula (11) Generating a segment characterization vector: (11) step 5.4, the end side computing device maintains a set of saved fragments Representing vectors for corresponding segments Fragment to be persisted Calculating the minimum cosine difference degree of the residual set relative to the residual set, and constructing the characteristic difference factor of the fragment by using the formula (12): (12) In the formula (12), the amino acid sequence of the compound, Is a preset minimum positive number, when When the end-side computing device will Setting the preset maximum value; step 5.5, the end side computing device sets a feature difference threshold And value threshold For each triggered fragment using equation (13) Performing a binary retention decision function: (13) in the formula (13), the amino acid sequence of the compound, To indicate the function, 1 is taken when the condition in brackets is satisfied, otherwise 0 is taken, when When the end-side computing device generates the structured data object Writing to a persisted set When (when) Discarding at the time The end side computing device gathers the fragments Configured to have a capacity of Annular retention buffer of (1) when When new fragments need to be written, the end side computing equipment covers the earliest written fragments according to the writing sequence, and maintains 。

Description

Remote operation data end side self-screening method for industrial robot with body based on shadow reasoning Technical Field The invention belongs to the technical field of industrial tool robots, and mainly relates to a remote operation data end side self-screening method of an industrial tool robot based on shadow reasoning. Background In recent years, with the rapid development of artificial intelligence and robotics, the technology of robotics has evolved from an early laboratory research stage to a current industrial application attempt and a partial scene landing stage. Many institutions are actively developing and testing body-of-intelligence technologies including cross-modal awareness, decision-making, intelligent computing, etc. to achieve autonomous decision-making and real-time environmental interactions. The acquisition and screening of high-quality remote control data are one of the core elements of the intelligent model training, and the robot needs to learn complex action logic through massive teaching data so as to safely and effectively execute tasks in an industrial environment. However, the existing industrial body robot has the defects that the traditional full-quantity data acquisition mode can generate huge data redundancy in the long-time remote operation process, so that storage and bandwidth resources are seriously wasted, in a complex industrial operation scene, the quality of part of acquired data is uneven due to operation fluctuation or environmental interference of a demonstrator, the training value of the data is difficult to evaluate in real time at the end side of the existing system, and when the body model is relied on for prediction, the model cannot timely capture and screen out difficult-case data with model prediction failure or low confidence degree due to limited generalization capability of the model when the model faces an unseen scene, so that the iterative optimization efficiency of a subsequent model is directly affected. Therefore, how to utilize a lightweight model to perform real-time quality evaluation and uncertainty analysis on cross-modal data in a teleoperation process under the condition of limited end-side computing force, and accurately reserve difficult-case segments with high feature difference degree so as to reduce redundancy in the data construction process and improve model training quality has become a current urgent problem to be solved. Disclosure of Invention In order to overcome the defects in the prior art, the invention provides the remote operation data end side self-screening method of the industrial robot based on shadow reasoning, so that the quality of remote operation data of the industrial robot can be improved, the mining capacity of the industrial robot on difficult-to-be-detected data is enhanced, the redundant ratio of end side storage and bandwidth resources is reduced, and a new thought is provided for intelligent data engineering. The invention adopts the following technical scheme for solving the problems: step 1, taking a visual frame time stamp as a unified time reference, and collecting visual observation of a robot Instruction embeddingNormalized motion vectorConstructing a unified cross-modal data tuple stream containing vision, instructions, and actions; Step 2, constructing a mapping rule of motion vector to discrete motion Token vectorization, and continuously normalizing the motion vectorMapping to corresponding real action Token vectorReal action Token sequence { is generated in time sequence}; Step 3, constructing a shadow reasoning model based on a history context windowAs input, the current time step is output via a dual layer matrix mapping processThe following predictive probability distribution matrix; Step 4, constructing an evaluation model and inputting a time stepCorresponding real action Token vectorAnd predictive probability distribution matrixCalculating a prediction bias factorAnd endogenous uncertainty factor。 Step 5, utilizing the prediction deviation factorAnd endogenous uncertainty factorBuilding frame-level valuationAnd in the cross-modal data tuple flowPackaging fragments to be saved by upper combined continuous hit mechanismCalculating the fragmentsFeature difference factor relative to a set of retained segmentsDetermining whether to treat the fragment by dual threshold decisionAnd its characterization vector writes into the end side memory space. Compared with the prior art, the invention has the beneficial effects that: 1. The invention provides a remote operation data end side self-screening method of an industrial robot with a body based on shadow reasoning, which carries out real-time accurate evaluation on the value of cross-mode data in the remote operation process, is based on the support degree and the distribution uncertainty of a real action Token in the predictive probability distribution, the data value is quantitatively evaluated, difficult data with low shadow inference model