CN-122018507-A - Intelligent storage multi-vehicle automatic driving collaborative decision-making system and method based on machine vision

CN122018507ACN 122018507 ACN122018507 ACN 122018507ACN-122018507-A

Abstract

The invention discloses an intelligent storage multi-vehicle automatic driving collaborative decision-making system and method based on machine vision, and relates to the technical field of intelligent storage and automatic logistics. The system comprises a multi-vision sensing module, a collaborative decision-making module, a vehicle-mounted execution module and a storage environment mapping module which are sequentially connected. The invention constructs a multidimensional machine vision perception system, integrates the perception resources of the vehicle-mounted binocular RGB-D camera, the shelf end vision sensor and the channel side panoramic camera, realizes the comprehensive perception of multiple elements such as vehicles, goods positions, goods, channels and the like in the storage environment, has more comprehensive perception range and stronger dynamic adaptability compared with the existing single vision or laser positioning scheme, and can effectively dynamically change the storage environment.

Inventors

ZHANG ZHANG
HU BINGYANG

Assignees

浙江科技大学
杭州沐创智能科技有限公司

Dates

Publication Date: 20260512
Application Date: 20260211

Claims (8)

1. An intelligent storage multi-vehicle automatic driving collaborative decision-making system based on machine vision is characterized by comprising a multi-vision sensing module, a collaborative decision-making module, a vehicle-mounted execution module and a storage environment mapping module which are connected in sequence, The multi-vision perception module comprises binocular RGB-D cameras, a shelf end vision sensor and a channel side panoramic camera, wherein the binocular RGB-D cameras are deployed on respective movable driving storage vehicles, the shelf end vision sensor is deployed on a shelf, the channel side panoramic camera is deployed on a channel side, the binocular RGB-D cameras are used for acquiring front scene images of vehicle driving paths, three-dimensional contour images of cargo trays and position and posture images of adjacent vehicles, the shelf end vision sensor is used for acquiring shelf goods space occupation state images and goods identification images, and the channel side panoramic camera is used for acquiring global scene images and multi-vehicle distribution images of storage channels; the storage environment mapping module is used for receiving various image data output by the multi-vision perception module and generating a storage dynamic semantic map through image splicing, feature matching and three-dimensional reconstruction; The collaborative decision-making module comprises edge computing nodes and a vehicle-vehicle communication interaction unit, wherein the edge computing nodes are used for receiving a warehouse dynamic semantic map and real-time state data of each mobile driving warehouse vehicle to generate a multi-vehicle task allocation scheme and a path collaborative strategy; The vehicle-mounted execution module is deployed on each movable driving warehouse truck and is used for receiving the decision instruction output by the collaborative decision module and controlling the truck to finish path tracking, fork lifting, pallet butt joint and obstacle avoidance actions.
2. The intelligent warehousing multi-vehicle automatic driving collaborative decision-making system based on machine vision according to claim 1, wherein the multi-vision perception module further comprises a vision data preprocessing unit, wherein the vision data preprocessing unit is used for performing noise filtering, exposure correction and distortion correction on the acquired image data, and extracting shelf upright post feature points, tray corner points, lane line features and vehicle contour features in the image through a feature extraction algorithm.
3. The intelligent warehousing multi-vehicle automatic driving collaborative decision-making system based on machine vision of claim 1, wherein the three-dimensional reconstruction process of the warehousing environment mapping module comprises the steps of constructing a cargo space three-dimensional model based on multi-view cargo space images acquired by a goods shelf end vision sensor, combining a global image acquired by a channel side panoramic camera to realize fusion of the cargo space model and the channel model, and updating three-dimensional coordinates of dynamic barriers through depth images acquired by binocular RGB-D cameras of respective mobile driving warehousing vehicles.
4. The intelligent storage multi-vehicle automatic driving collaborative decision-making method based on machine vision is characterized by applying the intelligent storage multi-vehicle automatic driving collaborative decision-making system based on machine vision as set forth in any one of claims 1-3, and comprises the following steps: S1, a multi-vision perception module starts image acquisition, binocular RGB-D cameras of respective movable driving warehouse vehicles acquire a driving path scene image and a surrounding vehicle image in real time, a goods shelf end vision sensor acquires a goods space state and a goods identification image, and a channel side panoramic camera acquires a channel global image; S2, the storage environment mapping module receives various image data, performs image splicing and three-dimensional reconstruction after preprocessing, and generates a storage dynamic semantic map containing goods positions, channels, dynamic barriers and goods information, and the storage dynamic semantic map is synchronized to the collaborative decision module in real time; S3, the collaborative decision-making module receives the storage dynamic semantic map and the position, speed and load state data of each mobile driving storage vehicle, and based on the multi-vehicle distribution characteristics and the channel occupation characteristics extracted by machine vision, a multi-vehicle task distribution scheme is generated, and the goods taking position, the goods delivering position and the running priority of each vehicle are defined; s4, aiming at the task paths of the vehicles, the cooperative decision-making module recognizes a junction and a narrow channel section in the paths through visual feature matching, calculates the time difference of the vehicles reaching the junction based on the multi-vehicle real-time position image, and generates a path cooperative strategy, wherein the path cooperative strategy comprises a junction passing sequence, a narrow channel vehicle meeting avoidance mode and a dynamic safety distance threshold; S5, the respective movable driving storage vehicles share sensing data and task execution states through the vehicle-vehicle communication interaction unit, the vehicle-mounted execution module controls the vehicles to run according to the collaborative decision-making instruction, the path deviation and the obstacle change are monitored in real time through the binocular RGB-D camera, the running parameters are dynamically adjusted, and the goods taking and placing and path running tasks are completed.
5. The intelligent warehousing multi-vehicle automatic driving collaborative decision-making method based on machine vision of claim 4, wherein the updating mode of the warehousing dynamic semantic map in S2 is that when a channel-side panoramic camera detects that an additional obstacle appears in a channel, binocular RGB-D cameras of surrounding vehicles are synchronously triggered to focus and shoot obstacle images, the three-dimensional size and the position of the obstacle are determined through multi-view image fusion, and the three-dimensional size and the position of the obstacle are updated to the warehousing dynamic semantic map.
6. The intelligent storage multi-vehicle automatic driving collaborative decision-making method based on machine vision according to claim 4, wherein the task allocation scheme in S3 is generated according to the goods space occupation state image and the goods identification image acquired by the goods shelf end vision sensor, and the goods weight and the volume characteristics are determined through image recognition, so that the load capacity of each vehicle is matched.
7. The intelligent warehouse multi-vehicle automatic driving collaborative decision-making method based on machine vision according to claim 4, wherein the determination process of the dynamic safety distance threshold in S4 is that the dynamic safety distance threshold is calculated by a visual ranging algorithm based on a front vehicle tail three-dimensional image acquired by a binocular RGB-D camera, a front vehicle outline size feature is extracted, and a vehicle running speed image and a channel illumination intensity image are combined.
8. The intelligent storage multi-vehicle automatic driving collaborative decision-making method based on machine vision according to claim 4, wherein the method is characterized in that the method also comprises a goods docking vision guiding process, namely when a vehicle runs to a target goods space, a binocular RGB-D camera collects three-dimensional contour images of a tray, extracts central coordinates and attitude angles of the tray, generates a goods fork adjusting instruction, and controls the goods fork to be accurately docked with the tray.

Description

Intelligent storage multi-vehicle automatic driving collaborative decision-making system and method based on machine vision Technical Field The invention relates to the technical field of intelligent storage and automatic logistics, in particular to an intelligent storage multi-vehicle automatic driving collaborative decision-making system and method based on machine vision. Background Along with the rapid development of the electronic commerce industry, intelligent storage is used as a core link of an automatic logistics system, and higher requirements are put forward on the efficiency and the accuracy of goods storage and transportation. The automatic driving warehouse truck is used as key execution equipment in intelligent warehouse, and the multi-truck collaborative operation capability directly determines the overall operation efficiency of the warehouse system. The existing intelligent storage multi-vehicle collaborative technology is dependent on a pre-paved magnetic stripe, two-dimensional code or laser SLAM positioning navigation scheme, and has the defects that firstly, the pre-paved navigation scheme of the magnetic stripe, the two-dimensional code and the like is high in construction cost and poor in flexibility in the early stage and cannot adapt to the requirement of dynamic adjustment of storage goods positions, secondly, the laser SLAM scheme is easily influenced by factors such as goods shelf shielding and goods reflection in a storage complex environment, so that the positioning precision is reduced, and the laser sensor cost is high, thirdly, the existing multi-vehicle collaborative decision is based on preset path planning, the sensing and response capability to dynamic change of a real-time environment is lacking, and vehicle congestion and collision risks are easy to occur, and particularly, the collaborative efficiency is low in a narrow channel intersection scene. The machine vision technology has the advantages of comprehensive information acquisition, relatively low cost, strong dynamic environment adaptation capability and the like, is gradually applied to the storage field, but the existing application is more limited to the visual navigation or goods identification of a single vehicle, and does not form a complete system of multi-vision node collaborative perception and multi-vehicle decision linkage. Specifically, the prior art does not fully integrate multidimensional visual resources such as vehicle-mounted vision, shelf vision, channel vision and the like, cannot construct a comprehensive and real-time storage dynamic environment model, and meanwhile, in the multi-vehicle collaborative decision process, dynamic features (such as real-time distribution of vehicles, channel occupation state, dynamic property of goods and the like) extracted by machine vision are not deeply integrated into a decision mechanism, so that collaborative strategies lack of accuracy and instantaneity. How to construct a perception system based on multi-dimensional machine vision, realize comprehensive dynamic perception of storage environment, and promote accuracy and efficiency of multi-vehicle automatic driving collaborative decision based on visual perception information, become the technical problem to be solved in the prior intelligent storage technology. Therefore, an intelligent storage multi-vehicle automatic driving collaborative decision-making system and method based on machine vision are provided to solve the difficulty existing in the prior art, which is a problem to be solved by the person skilled in the art. Disclosure of Invention In view of the above, the invention provides an intelligent warehousing multi-vehicle automatic driving collaborative decision-making system and method based on machine vision, which are used for constructing a warehousing dynamic semantic map by integrating multi-dimensional vision sensing resources, realizing accurate decision and efficient execution of multi-vehicle collaborative operation and improving the operation efficiency and flexibility of the intelligent warehousing system. In order to achieve the above object, the present invention provides the following technical solutions: an intelligent storage multi-vehicle automatic driving collaborative decision-making system based on machine vision comprises a multi-vision sensing module, a collaborative decision-making module, a vehicle-mounted execution module and a storage environment mapping module which are connected in sequence, The multi-vision perception module comprises binocular RGB-D cameras, a shelf end vision sensor and a channel side panoramic camera, wherein the binocular RGB-D cameras are deployed on respective movable driving storage vehicles, the shelf end vision sensor is deployed on a shelf, the channel side panoramic camera is deployed on a channel side, the binocular RGB-D cameras are used for acquiring front scene images of vehicle driving paths, three-dimensional contour images of c