CN-122008229-A - Double-mechanical-arm cooperative allocation decision method and system based on combination constraint

CN122008229ACN 122008229 ACN122008229 ACN 122008229ACN-122008229-A

Abstract

The invention discloses a double-mechanical-arm cooperative allocation decision-making method and a system based on combination constraint, which belong to the technical field of intelligent control of robots; the method comprises the steps of combining kinematic reachability constraint, task dependence constraint and combined cost function, carrying out double-arm distribution on the subtasks, further verifying parallel execution feasibility through double-arm joint track planning, determining a task execution mode, finally guiding a visual language model to generate an executable control script through structural prompt engineering, and completing closed-loop task execution through a robot control system.

Inventors

SUN QIYU
XU JING
TANG YANG
LIN JIALING

Assignees

华东理工大学

Dates

Publication Date: 20260512
Application Date: 20260324

Claims (10)

1. The double-mechanical-arm cooperative allocation decision method based on the combination constraint is characterized by comprising the following steps of: S1, receiving a natural language task instruction and environment visual information, carrying out joint analysis through a multi-mode large model, identifying a task related target and generating a task semantic representation; S2, task decomposition is carried out on task semantic representation, a plurality of atomic operation subtasks are generated, and time sequence dependency relations among the atomic operation subtasks are established; S3, aiming at each atomic operation subtask, evaluating the candidate execution mechanical arm by combining mechanical arm kinematic reachability constraint, time sequence dependency relationship and combined cost function to determine the mechanical arm for executing the subtask and an execution strategy; S4, for an atomic operation subtask combination with parallel execution potential, performing double-arm joint track planning in a complete configuration space of the robot, searching joint motion tracks which meet the collision constraint of the mechanical arm and are related to the avoidance of environmental static and dynamic obstacles by mutual dryness between the double arms, and verifying whether the subtask combination can be executed in parallel; s5, determining a task execution mode according to the verification result, wherein if the joint motion trail is successfully planned, the parallel execution mode is adopted, otherwise, the sequential execution mode is adopted; s6, inputting task structure information, action constraint conditions and a track planning result into a visual language model, and generating a structured task execution script through a structured prompt project; s7, calling a robot control system to execute corresponding atomic operations according to the task execution script, and acquiring environment feedback information in real time through the visual perception module to update the task state and generate a subsequent task to form a closed-loop execution process.
2. The method for collaborative allocation decision-making of dual robotic arms based on a combined constraint of claim 1 wherein the kinematic reachability constraint is determined by an inverse kinematic solution function that determines that the robotic arm is capable of performing a corresponding operation if there is a valid inverse kinematic solution to the target pose.
3. The method for determining the allocation of the double mechanical arms in cooperation based on the combination constraint according to claim 1, wherein the time sequence dependency relationship is used for describing the sequence of execution among different atomic operations and guaranteeing the correctness of task execution logic through the dependency constraint.
4. The dual-mechanical-arm cooperative allocation decision method based on combination constraint according to claim 1, wherein the combination cost function is: Wherein, the Representation mechanical arm Executing an action The single step cost of time is recorded as The method is specifically expressed as follows: Wherein, the For the operation degree index calculated based on the jacobian matrix, For a weighted euclidean distance between the target location and the robot base, The cost of the change for the joint space configuration, Is a normalized weight parameter.
5. The combined constraint-based double-mechanical-arm collaborative distribution decision-making method is characterized in that the method for planning the double-arm collaborative trajectory is characterized in that the combined motion trajectory meeting the collision constraint and the environment constraint is searched in a complete configuration space of a robot, and the collision constraint of the mechanical arm, the mutual interference between the double arms and the static and dynamic obstacles of the environment are simultaneously considered in the planning process.
6. The method for collaborative allocation decision-making of double mechanical arms based on combined constraint according to claim 1 is characterized in that the structured prompt engineering adopts a preset task prompt template, the reachability constraint of the mechanical arms, the action dependency relationship and the path safety condition are embedded into a prompt structure, a visual language model is guided to output a task execution script containing condition judgment logic in a programming-like format, the task execution script contains concurrent execution instructions in a parallel execution mode and is used for triggering a plurality of mechanical arm actions at the same time, and atomic operations are sequentially triggered in a sequence execution mode according to a preset sequence.
7. A system employing the combined constraint-based two-arm robot collaborative decision-making method of any of claims 1-6, comprising: the task analysis module is used for receiving the natural language task instruction and the environment visual information, generating task semantic representation through the multi-mode large model, and decomposing the task to obtain an atomic operation subtask and a time sequence dependency relationship thereof; The double-arm cooperation distribution module is used for determining an execution mechanical arm and an execution strategy of each atomic operation subtask according to the kinematic reachability constraint, the time sequence dependency relationship and the combined cost function; The collaborative feasibility verification module is used for carrying out double-arm joint track planning on atomic operation subtask combinations with parallel execution potential, verifying the parallel execution feasibility of the atomic operation subtask combinations and outputting an execution mode; the execution script generation module is used for inputting task structure information, action constraint conditions and a track planning result into the visual language model, and generating a structured task execution script through a structured prompt project; and the robot control module is used for calling the robot control system to execute corresponding atomic operations according to the task execution script, acquiring environment feedback through the visual perception module to update the task state and generate a subsequent task, and forming a closed-loop execution process.
8. The system of claim 7, wherein the dual-arm cooperative allocation module is further configured to evaluate a total cost of each of the robotic arms to perform the candidate actions based on the combined cost function during the task decomposition process, and to preferentially select a robotic arm with a lower total cost to perform the corresponding sub-task, thereby avoiding unnecessary dual-arm cooperation.
9. An electronic device comprising a memory and a processor, characterized in that the memory is configured to store a program supporting the processor to execute the dual-robot cooperative allocation decision method based on combined constraints according to any of claims 1 to 6, the processor being configured to execute the program stored in the memory.
10. A storage medium having stored thereon a computer program, which when executed by a processor performs the steps of the combined constraint based two-robot collaborative allocation decision method of any of claims 1 to 6.

Description

Double-mechanical-arm cooperative allocation decision method and system based on combination constraint Technical Field The invention belongs to the technical field of intelligent control and man-machine interaction of robots, and relates to a double-mechanical-arm cooperative allocation decision method and system based on combination constraint. Background With the rapid development of artificial intelligence technology and robot technology, self-intelligence is becoming an important research direction for realizing a general-purpose robot system. The intelligent emphasis of the intelligent agent is that the intelligent agent forms closed-loop interaction through perception, decision and action in the real environment, so that the complex task is completed. In recent years, the multi-modal large model and the visual language model are widely applied in the field of robots, and task understanding, planning and decision making capabilities of the robots in an open environment can be effectively improved by utilizing semantic knowledge and cross-modal alignment capability learned from massive data. In the field of robot operation, a two-arm robot has significant advantages in complex task execution due to its higher degrees of freedom and stronger operational capabilities compared to a single-arm robot. For example, in large-size object handling, two-hand cooperative assembly, trans-regional operation, and complex interactive tasks, the two-arm system can significantly improve task completion efficiency and operational stability through cooperative action. Therefore, the double-arm cooperative robot has wide application prospect in the scenes of home service, intelligent manufacturing, medical assistance, emergency rescue and the like. With the development of large model technology, part of research begins to introduce a large language model or a visual language model into a double-arm robot system so as to improve high-level task planning and collaborative decision-making capability. Such methods typically understand the context semantics through natural language instructions and generate task decomposition or action sequences to achieve robotic task execution. However, there are still significant limitations to the prior art. Firstly, part of double-arm robot systems default that all tasks need double-arm participation in a task planning stage, lack a judging mechanism for judging whether the tasks are suitable for single-arm execution, and easily introduce unnecessary double-arm cooperation in simple tasks, so that calculation overhead is increased and potential collision risks are increased. Secondly, task division is carried out by a collaboration method based on a multi-agent large model through natural language interaction, and although the method has certain flexibility, the decision process mainly depends on semantic reasoning and lacks explicit modeling on the actual physical constraint of the robot. For example, the working space accessibility, kinematic constraint or collision risk of the mechanical arm are not fully considered in the task allocation process, so that the situation that the task allocation result cannot be realized in the actual execution stage may occur. Again, existing two-arm mission planning methods often rely on a fixed role division strategy, such as presetting one arm to be responsible for clamping and another arm to be responsible for operation. The fixed division mode lacks the adaptability to environmental changes and task demands, and is difficult to realize efficient collaboration in complex scenes. Furthermore, existing methods often lack a physical feasibility pre-verification mechanism during the mission planning phase. When a task sequence is generated by high-level planning, an effective track can not be generated by a bottom-level motion planner when track planning is performed due to problems such as path conflict, joint limit or self-collision, so that the system needs to be frequently re-planned and even task execution fails. Meanwhile, when the two arms execute tasks in parallel, the conventional method is difficult to judge whether the two mechanical arms can safely execute actions simultaneously in a planning stage. Therefore, the existing double-arm robot collaborative decision-making method has a disjoint between semantic task planning and physical execution, and lacks a unified decision-making mechanism capable of simultaneously considering semantic understanding capability and physical constraint conditions. Disclosure of Invention The technical scheme of the invention is used for solving the problems of inconsistent large model decision and bottom execution, and insufficient cooperative reliability and efficiency caused by lack of physical feasibility constraint modeling in the task allocation process of the existing double-arm robot. The invention solves the technical problems through the following technical scheme: the invention provides a double-mech