CN-122022677-A - Transformer-based multi-target motion trail prediction device and real-time processing system thereof

CN122022677ACN 122022677 ACN122022677 ACN 122022677ACN-122022677-A

Abstract

The invention discloses a transform-based multi-target motion track prediction device and a real-time processing system thereof, which relate to the technical field of track prediction and comprise a light passing area construction module, a light passing area prediction module and a light receiving area prediction module, wherein the light passing area construction module is used for generating a vertical extension length based on an original gray image and generating a light passing area image based on the vertical extension length; the system comprises a geometric shearing bias calculation module, a physical centroid calculation module, a pure track generation module and a pure track generation module, wherein the geometric shearing bias calculation module is used for generating geometric shearing bias based on a light-passing area image, the physical centroid calculation module is used for calculating a local observation centroid and calculating physical centroid coordinates based on the local observation centroid and the geometric shearing bias, and the pure track generation module is used for determining a correlation result based on the physical centroid coordinates and generating a pure physical track based on the correlation result. The invention improves the operation efficiency and the safety in the narrow roadway warehouse operation.

Inventors

ZHANG LICHAO
TIAN LING
SHAO CHUNPING
Que Yongwei
HE HUA
CHEN LINXIAO

Assignees

广州航海学院
广州鹏远智能设备有限公司

Dates

Publication Date: 20260512
Application Date: 20260122

Claims (9)

1. The transform-based multi-target motion trail prediction device is characterized by comprising the following steps of: The light passing region construction module is used for generating a vertical extension length based on the original gray level image and generating a light passing region image based on the vertical extension length; the geometric shearing bias calculation module is used for generating geometric shearing bias based on the light transmission area image; the physical centroid calculation module is used for calculating a local observation centroid and calculating physical centroid coordinates based on the local observation centroid and the geometric shearing offset; the pure track generation module is used for determining a correlation result based on the physical centroid coordinates and generating a pure physical track based on the correlation result; And the motion trend prediction module is used for predicting the future motion trend of the target picker based on the pure physical track.
2. The transform-based multi-target motion trajectory prediction device of claim 1, wherein generating a vertical extension length based on an original gray scale image comprises: Acquiring an original gray image acquired by a robot vision sensor; performing edge extraction on the original gray level image to obtain a vertical edge set; coordinate positioning is carried out on the vertical central axis of the original gray level image, and a reference height is obtained; presetting a threshold value of the reference height to obtain a width threshold value; Acquiring pixel widths of each vertical edge in the vertical edge set at a reference height; And carrying out vertical connected pixel height statistics on each vertical edge in the vertical edge set to obtain the vertical extension length.
3. The transform-based multi-target motion trajectory prediction device of claim 2, wherein generating the light passing region image based on the vertical extension length comprises: Judging the vertical edge of which the pixel width is larger than a width threshold value and the vertical extension length penetrates through the original gray level image as a near-end column pixel; generating a near-end pillar mask based on the near-end pillar pixels; judging the vertical edge of which the pixel width is smaller than or equal to a width threshold value and the vertical extension length does not penetrate through the original gray level image as a far-end column pixel; generating a far-end pillar mask based on the far-end pillar pixels; and performing non-union logical operation on the near-end upright post mask and the far-end upright post mask to obtain a light-transmitting area image.
4. The transform-based multi-target motion trajectory prediction device of claim 3, wherein generating a geometric shearing bias based on the light passing region image comprises: carrying out connected domain analysis on the light passing region image to obtain a slit region; acquiring a pixel geometric center corresponding to the slit region; performing edge attribute matching on the horizontal coordinate range of the slit region according to the near-end upright post mask to obtain a near-end upright post; Performing midpoint coordinate calculation on the horizontal distance between the near-end upright posts to obtain a theoretical gap center; and calculating the difference between the geometric center of the pixel and the theoretical gap center to obtain the geometric shearing offset of the slit region.
5. The transducer-based multi-target motion trajectory prediction device of claim 4, wherein calculating a local observation centroid comprises: detecting a moving target of the original gray image to obtain an original pixel region; Performing intersection operation on a pixel coordinate set corresponding to the original pixel region and a pixel coordinate range corresponding to each slit region to obtain an overlapped pixel set; counting the number of pixels in the overlapped pixel set; judging a slit area corresponding to the pixel number with the largest value as a target occupied slit; and determining the local observation centroid of the pixel area corresponding to the target picker in the target occupying slit.
6. The Transformer-based multi-target motion trail prediction device of claim 5, wherein calculating physical centroid coordinates based on the local observation centroids and the geometric shear offsets comprises: Acquiring the pixel width of a target occupied slit; Performing perspective projection calculation on a pixel area corresponding to the target picker to obtain an average pixel width; multiplying the ratio calculation result of the pixel width of the target occupying slit and the average pixel width by the geometric shearing offset to obtain a preliminary proportional displacement; performing product operation on the preliminary proportional displacement and a preset gain coefficient to obtain a final offset compensation quantity; and subtracting the final offset compensation amount from the local observation centroid to obtain a physical centroid coordinate.
7. The transform-based multi-target motion trajectory prediction device of claim 1, wherein determining a correlation result based on physical centroid coordinates comprises: Calculating displacement vectors between two physical centroid coordinates at adjacent moments; Defining a horizontal vector vertical to the arrangement direction of the shelf upright posts as a noise sensitive vector; Calculating a projection component of the displacement vector in the noise sensitive vector direction; calculating the time-dependent rate of change of the geometric shear offset; and carrying out correlation judgment on the change trend of the projection component and the time change rate of the geometric shearing offset to obtain a correlation result.
8. The transform-based multi-target motion trajectory prediction device of claim 7, wherein generating a clean physical trajectory based on correlation results comprises: Performing numerical comparison on the projection component and a preset column width threshold value to obtain a comparison result; Carrying out zero setting treatment on components of the displacement vector on the noise sensitive vector according to the correlation result and the comparison result to obtain a pure physical displacement vector; And carrying out vector superposition operation on the pure physical displacement vectors corresponding to each moment to obtain a pure physical track.
9. A real-time processing system for multi-target motion trajectory prediction based on a transducer, wherein the system comprises the multi-target motion trajectory prediction device based on a transducer according to any one of claims 1 to 8.

Description

Transformer-based multi-target motion trail prediction device and real-time processing system thereof Technical Field The invention relates to the technical field of track prediction, in particular to a transform-based multi-target motion track prediction device and a real-time processing system thereof. Background With the continuous development of intelligent storage and logistics automation technology, mobile robots are gradually and widely applied to high-density storage operation scenes, particularly in a narrow roadway storage area, and the mobile robots are required to finish operation tasks such as carrying, inspection, auxiliary picking and the like under limited space conditions. In such a scenario, in order to improve the storage capacity of a unit area, storage shelves generally adopt a back-to-back arrangement mode to form a shelf channel which is continuously arranged, and two sides of the channel are formed by a plurality of rows of vertical upright posts, so that the space structure is regular and dense. Meanwhile, in the actual operation process, the mobile robot often needs to cooperatively operate with the manual pickers in the same roadway, the robot senses the operation environment in real time through a carried visual sensor to acquire the position change condition of the pickers, and predicts the future movement trend of the pickers based on historical position data, so that path planning and safety avoidance are realized. In the warehouse operation environment, the dense arrangement of the shelf upright posts, the narrowness of the roadway space and the continuous movement of the robot, so that the vision sensing process faces complex space shielding relation and perspective change conditions, and high requirements are provided for stable acquisition of the target position. In the prior art, when predicting the motion trail of a picker in a roadway based on a visual image, the position of a target in a visible pixel area in the image is generally calculated, and the position is directly used as motion input data of the target. When a picker is in a roadway environment formed by back-to-back goods shelf upright posts, the body outline of the picker is often periodically blocked by the upright posts with different distances from front to back, and along with the movement and viewing angle change of a robot, the visible area is asymmetrically changed in the horizontal direction, so that the target position calculated based on the visible pixels generates obvious transverse jump in a short time. The existing method generally misconsiders that the target moves truly and transversely by the position change, so that a large amount of unreal motion information is introduced in the track prediction process, the prediction result is unstable, even the robot is caused to frequently take unnecessary avoidance or braking operation, and the actual requirements on the stability and reliability of the motion track prediction in the narrow roadway storage scene are difficult to meet. Disclosure of Invention The invention aims to solve the defect that the actual requirements on the stability and reliability of motion trail prediction in a narrow roadway storage scene are difficult to meet in the prior art, and provides a transform-based multi-target motion trail prediction device and a real-time processing system thereof. In order to solve the problems existing in the prior art, the invention adopts the following technical scheme: the multi-target motion trail prediction device based on the Transformer comprises: The light passing region construction module is used for generating a vertical extension length based on the original gray level image and generating a light passing region image based on the vertical extension length; the geometric shearing bias calculation module is used for generating geometric shearing bias based on the light transmission area image; the physical centroid calculation module is used for calculating a local observation centroid and calculating physical centroid coordinates based on the local observation centroid and the geometric shearing offset; the pure track generation module is used for determining a correlation result based on the physical centroid coordinates and generating a pure physical track based on the correlation result; And the motion trend prediction module is used for predicting the future motion trend of the target picker based on the pure physical track. Preferably, generating the vertical extension length based on the original gray image includes: Acquiring an original gray image acquired by a robot vision sensor; performing edge extraction on the original gray level image to obtain a vertical edge set; coordinate positioning is carried out on the vertical central axis of the original gray level image, and a reference height is obtained; presetting a threshold value of the reference height to obtain a width threshold value; Acquiring pixel widths of each