US-12617426-B2 - Autonomous docking system for freight trucks

US12617426B2US 12617426 B2US12617426 B2US 12617426B2US-12617426-B2

Abstract

Autonomous docking systems and methods receive asynchronous images from a vehicle-mounted camera and a plurality of infrastructure cameras and perform object mapping to determine candidate routes leading to a docking station. Candidate routes are divided into route segments for which disparity map quality scores are estimated to select a navigation route. Information from infrastructure cameras and vehicle-mounted cameras is used to generate a disparity map to perform object detection, distance measurement, or localization along the navigation route. The disparity map is used to communicate navigation commands to an HMI or a vehicle control system.

Inventors

Subrata Kumar KUNDU
Haruki Matono
Xiaoliang Zhu
Akshay Gokhale

Assignees

HITACHI, LTD.

Dates

Publication Date: 20260505
Application Date: 20240312

Claims (19)

1 . A method for autonomous docking of a vehicle, the method comprising: in response to receiving, at a docking system from a plurality of infrastructure cameras that are mounted at different locations and orientations at a docking facility, image data indicative of a presence of a vehicle, obtaining vehicle-related information from a vehicle, which comprises a vehicle-mounted camera, to identify an available docking station; extracting features from at least some of the image data to perform an object mapping to identify one or more objects in proximity to the docking facility; using the object mapping to determine candidate routes leading to the available docking station; for each of the candidate routes, iteratively performing steps comprising: dividing the candidate route into a plurality of waypoints; determining route segments that each is defined by a distance between two waypoints among the plurality of waypoints; for each of one or more route segments among the route segments, performing steps comprising: using location information regarding at least some of the plurality of infrastructure cameras and the vehicle-mounted camera to determine an overlapping region between the vehicle-mounted camera and one infrastructure camera among the plurality of infrastructure cameras to identify a selected infrastructure camera; combining asynchronous images from the selected infrastructure camera and from the vehicle-mounted camera, to create a three-dimensional (3-D) depth map; and using the 3-D depth maps to estimate a disparity map quality score; and using the disparity map quality scores for the route segments to estimate a disparity map quality score for the candidate route; using the disparity map quality scores for the candidate routes to select a navigation route; and using information from at least some of the plurality of infrastructure cameras and the vehicle-mounted camera to generate a disparity map to perform object detection, distance measurement, and localization along the navigation route; and using the disparity map to communicate navigation commands to at least one of a human machine interface (HMI) or a vehicle control system.
2 . The method of claim 1 , further comprising communicating the estimate, via a human machine interface, to a driver.
3 . The method of claim 2 , wherein the HMI comprises a smartphone application.
4 . The method of claim 2 , further comprising establishing a secure communication between the docking system and at least one of the HMI or the vehicle control system.
5 . The method of claim 4 , further comprising in response to selecting the route and receiving, via an HMI application, a response to a query, initiating a docking process that comprises steps comprising at least one of: receiving from the vehicle control system at least one of vehicle specifications, sensor specifications, or vehicle electronic control unit (ECU) specifications; communicating a driving command to a vehicle control system, or communicating driving information that comprises the selected route to the HMI.
6 . The method of claim 5 , wherein at least one or more of the object mapping, the 3D depth map, the vehicle-related information, the driving command, or the driving information is updated in real-time.
7 . The method of claim 5 , wherein the driving information comprises a driving recommendation that comprises at least one of a speed recommendation or a steering angle recommendation.
8 . The method of claim 5 , further comprising, in response to the docking process having been initiated, performing steps comprising at least one of: using one or more cameras to determine a vehicle localization and performing a mapping; for a plurality of sampling times, combining first image data obtained from the selected infrastructure camera and second image data obtained from the vehicle camera to generate a disparity map of the route; using the disparity map to identify one or more objects along the driving route; and monitoring a driving performance.
9 . The method of claim 8 , further comprising in response to vehicle reaching the docking station, using the driving performance to generate a driver rating and communicating the driver rating to the docking system.
10 . The method of claim 7 , wherein communicating the driving information comprises communicating directions to move the vehicle along the selected route.
11 . The method of claim 1 , wherein using the disparity map quality scores comprises determining at least one of a maximum disparity map quality score or an average disparity map quality score for the candidate route.
12 . The method of claim 1 , further comprising, for at least one of the one or more route segments, dividing that route segment into a first region and a second region, and wherein the selected infrastructure camera is identified based on its proximity to the first region or the second region.
13 . The method of claim 1 , wherein the selected infrastructure camera is identified based on a vehicle location and a driving direction, wherein the vehicle-mounted camera is a rear-view camera when the vehicle is moving in a reverse direction.
14 . The method of claim 1 , wherein the selected infrastructure camera and the vehicle-mounted camera are located at different planes.
15 . The method of claim 1 , further comprising calculating for two route segments that each have different lengths a stopping site distance.
16 . The method of claim 1 , wherein identifying the available docking station comprises generating an estimate of at least one of a minimum docking time or a wait time.
17 . The method of claim 1 , wherein combining the asynchronous images to create the 3-D depth map: receiving asynchronous images from the vehicle-mounted camera and the infrastructure camera; comparing a feature of the asynchronous images that were taken with a time delay to time-synchronize the asynchronous images; using camera locations to calculate a calibration matrix; using the calibration matrix to perform an image rectification on the asynchronous images to obtain rectified images; and in response to determining a disparity range and a disparity resolution, using the rectified images to estimate the 3-D depth map.
18 . A system for autonomous vehicle docking, the system comprising: infrastructure cameras configured to be mounted at different locations and orientations at a docking facility; a docking system controller coupled to the infrastructure cameras, the docking system controller configured to perform steps comprising: in response to receiving asynchronous images from a vehicle-mounted camera and a plurality of infrastructure cameras, performing an object mapping to determine candidate routes leading to a docking station; in response to dividing each candidate route into route segments, estimating for each route segment a disparity map quality score to select a navigation route among the candidate routes; using information from at least some of the plurality of infrastructure cameras and the vehicle-mounted camera to generate a disparity map to perform at least one of object detection, distance measurement, or localization along the navigation route; and using the disparity map to communicate navigation commands to at least one of a human machine interface (HMI) or a vehicle control system, wherein estimating the disparity map quality scores comprises simulating overlapping regions between the vehicle-mounted camera and an infrastructure camera among the plurality of infrastructure cameras.
19 . A non-transitory computer-readable medium for storing instructions for executing a process, the instructions comprising: in response to receiving asynchronous images from a vehicle-mounted camera and a plurality of infrastructure cameras, performing an object mapping to determine candidate routes leading to a docking station; in response to dividing each candidate route into route segments, estimating for each route segment a disparity map quality score to select a navigation route among the candidate routes; using information from at least some of the plurality of infrastructure cameras and the vehicle-mounted camera to generate a disparity map to perform at least one of object detection, distance measurement, or localization along the navigation route; and using the disparity map to communicate navigation commands to at least one of a human machine interface (HMI) or a vehicle control system, wherein estimating the disparity map quality scores comprises simulating overlapping regions between the vehicle-mounted camera and an infrastructure camera among the plurality of infrastructure cameras.

Description

BACKGROUND Field The present disclosure is generally directed to autonomous driving, and more specifically, to systems and methods for autonomous docking of vehicles, such as freight trucks, at a docking facility, e.g., a warehouse. Related Art Connected automated trucks are gaining wide attention as recent rapid technological advancements in digitalization, artificial intelligence, robotics, and advanced computing platforms are expected to play major role in the future of connected mobility ecosystem, which is instrumental in fostering a sustainable and resilient society. Ensuring the safe and reliable operation of freight trucks is critically important for achieving the ultimate benefits for the logistics industry. However, efficient and safe autonomous docking of logistics trucks still remains a challenge. In addition, the logistics industry suffers from truck driver shortages, lack of adequate parking facilities, driver fatigue issues, and protracted waiting periods at loading or docking stations. These factors exacerbate delivery delays, increase backlogs, and add to supply chain costs. Docking a long truck proficiently at a docking station, which is critical not only to avoid delays or detention at docking facilities but also to increase the logistical efficiency, creates a challenge especially for inexperienced truck drivers. Existing methods for intelligent docking, which involve remote monitoring and autonomous vehicles guidance based on image or video data, typically ascertain a vehicle's present location and maneuver the vehicle using cameras. However, these methods oftentimes result in inaccuracies. Some truck docking techniques offer assistance once when trailer approaches a docking station, utilizing radar, camera, ultrasonic sensors, etc., at the docking bay to gauge the distance between the trailer and edge of the docking station, providing relevant guidance. However, during peak times in large warehouses or facilities, it becomes challenging for novice truck drivers to navigate from the entrance gate to the docking station and securely dock the trailer at the docking station within a limited time frame. Conventional autonomous docking of the truck at the docking station relies on vehicle-mounted front and back cameras, which fall short in achieving precise distance and localization estimation. SUMMARY In embodiments described herein, end-to-end automated docking systems and methods provide driving assistance or fully autonomously docking for trucks at a docking station. This is accomplished through the implementation of data fusion techniques that combine asynchronous information from both infrastructure cameras located at a docking facility and truck-mounted cameras. This integration significantly increases the accuracy of 3D depth mapping at docking facilities. The resulting improvements in object detection and localization increase perception and distance measurement accuracy, thereby facilitating the efficient operation of automated docking systems. Additional advantages include systems and methods for using feedback to assess driver performance and targeted generate recommendations for driver training. In some aspects of the disclosure, a docking system identifies an available docking station based on image data from infrastructure cameras and vehicle-related information. The docking system extracts features from the image data to perform an object mapping to identify objects in proximity to the docking facility to determine candidate routes leading to the available docking station. For each of the candidate routes, the docking system iteratively divides the candidate route into waypoints and determines route segments defined by a distance between two waypoints. For each route segment location information regarding the infrastructure cameras or vehicle-mounted cameras are used to determine an overlapping region between a vehicle-mounted camera and an infrastructure camera to select an infrastructure camera whose images are then combined with those from the vehicle-mounted camera to create a three-dimensional (3-D) depth map. This map is used to estimate a disparity map quality score for each route segment to estimate a disparity map quality score for the candidate routes. Estimating the disparity map quality scores may comprise simulating overlapping regions between the vehicle-mounted camera and an infrastructure camera among the plurality of infrastructure cameras. Once a candidate route is selected for navigation a disparity map is generated to perform object detection, distance measurement, and localization along the navigation route. Navigation commands are securely communicated to a human machine interface (HMI), e.g., a smartphone, or a vehicle control system. Aspects of the present disclosure can involve a system, which can involve means for performing steps comprising: in response to receiving asynchronous images from a vehicle-mounted camera and a plurality of infrastructure