CN-121722157-B - Mobile robot task scheduling method and system based on heterogeneous computing architecture

CN121722157BCN 121722157 BCN121722157 BCN 121722157BCN-121722157-B

Abstract

The application provides a mobile robot task scheduling method and system based on heterogeneous computing architecture, and relates to the technical field of intelligent scheduling, wherein the method comprises the steps of constructing a working space geometric model based on a planting row geometric structure, acquiring a motion state parameter of a mobile robot by using a first computing unit, and determining a look-ahead compensation quantity by combining a statistic value of time consumption of target recognition task reasoning by a second computing unit; and a second computing unit performs target recognition on the image data, generates a working instruction corresponding to the recognition target, and controls a lateral executing mechanism to execute the work according to the working instruction. Therefore, the problem of time-space asynchronous between the sensing result and the execution position can be relieved in the inter-traveling operation scene, the missing operation and the misplacement operation are reduced, and the communication and the calculation load of the heterogeneous calculation system are reduced.

Inventors

YANG LONG
Xiong jiao
Zang Sudong
ZHU WENFENG
LI JIAN
ZHANG YONGPENG
WANG KANGBIN

Assignees

上海恒泽辅汇智能科技有限公司
上海恒启智向智能科技有限公司

Dates

Publication Date: 20260505
Application Date: 20260225

Claims (10)

1. The mobile robot task scheduling method based on the heterogeneous computing architecture is characterized by comprising the following steps of: on a mobile robot carrying forward imaging equipment and a lateral actuating mechanism, constructing a working space geometric model based on the geometric structure of a structured planting row, wherein the working space geometric model characterizes the spatial correspondence between a visual field area of the forward imaging equipment and a physical working envelope area of the lateral actuating mechanism; Acquiring a motion state parameter of the mobile robot by using a first computing unit, and determining a look-ahead compensation amount based on the motion state parameter and a time-consuming reasoning statistic value of a second computing unit on a target recognition task, wherein the first computing unit is communicated with the forward imaging device, the second computing unit is communicated with the first computing unit, and the motion state parameter comprises an instantaneous linear speed and/or an angular speed along a planting row direction; Determining, with the first computing unit, a spatio-temporal pre-aiming slice in the visual field region based on the look-ahead compensation amount and the working space geometric model; Cutting out image data positioned in the space-time pre-aiming slice from the acquired original image data by using the first computing unit, and sending the cut image data to the second computing unit; And executing target recognition on the image data in the space-time pre-aiming slice by using the second computing unit to generate a job instruction corresponding to the recognition target, and controlling the lateral execution mechanism to execute the job according to the job instruction.
2. The mobile robot task scheduling method based on heterogeneous computing architecture according to claim 1, wherein the look-ahead compensation amount is used to characterize a line displacement of the mobile robot along a planting line within a predetermined look-ahead time, and the predetermined look-ahead time is determined based on a statistics value of time consumed by reasoning of the second computing unit on the target recognition task.
3. A mobile robot task scheduling method based on heterogeneous computing architecture according to claim 1 or 2, wherein determining a spatio-temporal pre-aiming slice comprises: In the operation space geometric model, reversely translating the physical operation envelope region along the motion direction of the mobile robot by the forward-looking compensation amount to obtain a translated operation region; And projecting the translated working area to an imaging plane of the forward imaging device, wherein the projection area is used as the space-time pre-aiming slice.
4. The mobile robot task scheduling method based on a heterogeneous computing architecture according to claim 1 or 2, wherein the first computing unit is an FPGA unit, and the second computing unit is an NPU unit.
5. The heterogeneous computing architecture based mobile robot task scheduling method of claim 1, wherein the controlling the lateral actuator to execute the job according to the job instruction comprises: determining that the recognition target is located within the physical job envelope based on the job space geometric model; and triggering the lateral actuating mechanism to execute the operation instruction in response to the determination result being located in the physical operation envelope region.
6. The mobile robot task scheduling method based on heterogeneous computing architecture of claim 2, wherein determining the spatio-temporal pre-aiming slice comprises: synchronously acquiring a driving current high-frequency sequence of a chassis motor of the mobile robot and a point cloud stream output by a 4D radar sensing unit which is associated with the forward imaging equipment and used for sensing a space structure of the operation roadway in the process that the mobile robot runs along the operation roadway defined by the multi-layer three-dimensional planting frame by utilizing the first computing unit in a time window corresponding to the preset look-ahead time; Determining an instantaneous jitter offset vector of the forward imaging device based on the driving current high-frequency sequence by using the first computing unit, and determining a slip pose compensation amount of the mobile robot along the planting row direction based on the point cloud flow and the encoder speed of the chassis motor; And correcting a visual field area corresponding to the operation roadway in the operation space geometric model based on the instantaneous jitter offset vector and the slip pose compensation quantity by using the first calculation unit, and determining the space-time pre-aiming slice subjected to jitter and slip compensation based on the corrected operation space geometric model.
7. The heterogeneous computing architecture based mobile robot task scheduling method of claim 6, wherein the determining the instantaneous jitter offset vector of the forward imaging device comprises: performing wavelet packet transformation on the driving current high-frequency sequence by using the first computing unit, and extracting current ripple components matched with a mechanical resonance frequency band; And mapping the current ripple component into an instantaneous jitter offset vector of the forward imaging device at the imaging moment based on a preset current-end jitter transfer function.
8. The method for scheduling mobile robot tasks based on heterogeneous computing architecture according to claim 6, wherein determining the slip pose compensation amount of the mobile robot along the planting row direction based on the point cloud flow and the encoder speed of the chassis motor comprises: Selecting a height zone corresponding to the height range of the multi-layer three-dimensional planting frame stand column in the vertical direction by using the first computing unit according to the pitch angle information of the point cloud flow, and extracting horizontal slice point clouds distributed along the planting row direction in the height zone; performing Doppler velocity field analysis on the horizontal slice point cloud, and constructing an environment reference velocity matrix based on upright post echo clusters distributed in a preset interval range along the row direction of an operation roadway; Performing cross-correlation operation on the environment reference speed matrix and the encoder speed to generate a nonlinear slipping coefficient sequence; And performing time domain integration on the nonlinear slipping coefficient sequence along the row direction of the planting row by using the first computing unit to obtain the slipping pose compensation quantity.
9. The mobile robot task scheduling method based on heterogeneous computing architecture according to claim 7, wherein the preset current-end jitter transfer function is obtained by: in a static calibration stage of the mobile robot stopped at a typical operation roadway position corresponding to the structured planting row, applying a step excitation signal to the lateral actuating mechanism by using the first computing unit, and acquiring a displacement attenuation curve of a calibration plate by using the forward imaging equipment; Synchronously recording a motor current response sequence caused by the step excitation signal; And establishing a mapping model between the motor current response sequence and the displacement attenuation curve by using a system identification algorithm, and storing parameters of the mapping model in on-chip storage resources of the first computing unit as the current-end jitter transfer function.
10. A mobile robotic task scheduling system based on heterogeneous computing architecture, comprising: The construction module is used for constructing a working space geometric model on the mobile robot carrying the forward imaging equipment and the lateral actuating mechanism based on the geometric structure of the structured planting row, wherein the working space geometric model characterizes the space corresponding relation between the visual field area of the forward imaging equipment and the physical working envelope area of the lateral actuating mechanism; The processing module is used for acquiring a motion state parameter of the mobile robot by using a first computing unit, and determining a look-ahead compensation amount based on the motion state parameter and a time-consuming statistic value of reasoning of a target recognition task by a second computing unit, wherein the first computing unit is communicated with the forward imaging equipment, the second computing unit is communicated with the first computing unit, and the motion state parameter comprises an instantaneous linear speed and/or an angular speed along the row direction of planting; Determining, with the first computing unit, a spatio-temporal pre-aiming slice in the visual field region based on the look-ahead compensation amount and the working space geometric model; Cutting out image data positioned in the space-time pre-aiming slice from the acquired original image data by using the first computing unit, and sending the cut image data to the second computing unit; Performing target recognition on the image data in the space-time pretightening slice by using the second computing unit, and generating a work instruction corresponding to the recognition target; And the scheduling module is used for controlling the lateral execution mechanism to execute the job according to the job instruction.

Description

Mobile robot task scheduling method and system based on heterogeneous computing architecture Technical Field The application relates to the technical field of intelligent scheduling, in particular to a mobile robot task scheduling method and system based on a heterogeneous computing architecture. Background At present, a mobile robot provided with forward imaging equipment and a lateral actuating mechanism is gradually used for fruit and vegetable picking, fixed-point spraying, pest and disease damage identification and other travelling intercropping scenes. In order to improve the accuracy and the instantaneity of target identification, a plurality of schemes introduce heterogeneous computing architectures including FPGA, NPU and the like, and image preprocessing, target detection and track planning are completed on the edge side, so that the burden of an upper computer is reduced, and the overall time delay of a perception-decision-execution link is shortened. In the prior art, a mobile robot generally continuously runs among structured planting rows, a forward imaging device acquires visual images along the row direction, an FPGA or other preprocessing units simply cut, scale-change or format-convert the images, the processed images or regions of interest are transmitted to an NPU to execute target recognition, and then a lateral execution mechanism is controlled to finish operations such as grabbing or spraying according to recognition results. Such schemes can basically meet the operation requirements under static or low-speed working conditions, but in the scene that the robot keeps a certain travelling speed and the operation objects are densely distributed, the NPU reasoning in the perception link inevitably introduces the reasoning time consumption of millisecond level or even longer. Because the mobile robot still continuously moves along the planting row during NPU reasoning, the existing scheme often directly drives the lateral actuating mechanism based on the image coordinates at the shooting time, and is difficult to timely and accurately compensate the displacement generated by the robot in the reasoning time, so that the visual position of the target and the physical operation envelope area of the lateral actuating mechanism have deviation in time and space, the recognition result is delayed from the actual execution position, and the problems of missing operation, misplacement operation and the like occur. To reduce the above risk, some schemes "wait" for the recognition result by reducing the travel speed or frequent pauses, but this can significantly affect the overall line efficiency. On the other hand, in the heterogeneous computing architecture, if the whole frame of image or the image data of a larger area is continuously transmitted to the NPU for processing, higher bus bandwidth and computing resources are occupied, it is difficult to maintain sufficient frame rate and resolution under the constraint of limited power consumption, and the communication and computing loads are heavy, so that the expansion capability of the system in a complex agricultural scene is affected. In summary, in the operation scene between mobile robots in the prior art, there is still a technical problem of how to overcome the time-space asynchronous problem that the sensing result is lagged behind the physical execution position due to the time consumption of sensing unit (NPU) reasoning, and simultaneously reduce the communication and calculation load of heterogeneous calculation systems. Disclosure of Invention Aiming at the defects of the prior art, the application provides a mobile robot task scheduling method and system based on a heterogeneous computing architecture. In a first aspect, the present application provides a mobile robot task scheduling method based on a heterogeneous computing architecture, including: on a mobile robot carrying forward imaging equipment and a lateral actuating mechanism, constructing a working space geometric model based on the geometric structure of a structured planting row, wherein the working space geometric model characterizes the spatial correspondence between a visual field area of the forward imaging equipment and a physical working envelope area of the lateral actuating mechanism; Acquiring a motion state parameter of the mobile robot by using a first computing unit, and determining a look-ahead compensation amount based on the motion state parameter and a time-consuming reasoning statistic value of a second computing unit on a target recognition task, wherein the first computing unit is communicated with the forward imaging device, the second computing unit is communicated with the first computing unit, and the motion state parameter comprises an instantaneous linear speed and/or an angular speed along a planting row direction; Determining, with the first computing unit, a spatio-temporal pre-aiming slice in the visual field region based on the look-ahead compe