Search

EP-4742121-A1 - SELECTING LOCATIONS FOR AN AUTOMATED CONSTRUCTION MACHINE TO UNLOAD CARGO

EP4742121A1EP 4742121 A1EP4742121 A1EP 4742121A1EP-4742121-A1

Abstract

A vehicle moves through an environment (e.g., a farming, construction, mining, or forestry environment) and performs one or more actions in the environment. A control system associated with the vehicle may include a mode selection module and a transportation determination module. In particular, the control system may employ a machine vision model to identify images captured for each region in the environment. The model identifies viable unloading locations in an unloading zone of the environment, and then navigates the vehicle to one or more unloading locations based on pixels of the image representing terrain features and existing cargo dump piles. Control signals for loading or unloading cargo are determined in alignment with a transportation protocol set by a site manager. A loading or unloading mechanism is then actuated using these control signals to unload cargo in the appropriate location.

Inventors

  • WARDEN, Grant

Assignees

  • Deere & Company

Dates

Publication Date
20260513
Application Date
20251110

Claims (15)

  1. A method for autonomously controlling a machine (100) to manage cargo loads at a plurality of locations of a jobsite, the method comprising: accessing (510) images of the jobsite including the plurality of locations; accessing (520) sensor information describing cargo present at the jobsite; applying (530) one or more recognition models to the images and the sensor information, the one or more recognition models configured to: identify (532) the plurality of locations in the jobsite; identify (534) a presence or an absence of cargo at each location; responsive to a location indicating a presence of cargo at the location, identify (536) characteristics of the cargo at the location; and generate (538) a jobsite map comprising the plurality of locations, each location indicating the presence or absence of cargo at each location and characteristics of the cargo present at the location; inputting (540) the jobsite map and an accessed jobsite protocol into a navigation model to generate a cargo management plan that adheres to the accessed jobsite protocol, the cargo management plan representing actions for the machine to implement for managing cargo at the plurality of locations; controlling (550) the machine to traverse the jobsite to implement the cargo management plan.
  2. The method of claim 1, wherein the accessed jobsite protocol comprises instructions to improve navigation efficiency and generating the cargo management plan comprises: determining a navigation path for the machine based on a total time spent traversing the jobsite to implement the cargo management plan.
  3. The method of claim 1, wherein the accessed jobsite protocol comprises instructions to improve storage efficiency and generating the cargo management plan comprises: selecting a set of locations for the cargo management plan based on a total space occupied by cargo present at the jobsite.
  4. The method of claim 1, wherein the accessed jobsite protocol comprises instructions to store cargo based on cargo characteristics, and generating the cargo management plan comprises: selecting locations for cargo present at the jobsite such that cargo having similar characteristics are present in a similar region of locations.
  5. The method of claim 1, wherein the accessed jobsite protocol comprises instructions to manage cargo on two or more parameters.
  6. The method of any one of the preceding claims, wherein applying one or more recognition models to the images and sensor information comprises: applying an image classification model to the accessed images, the image classification model: identifying pixels in the images representing locations, and mapping identified pixels representing the locations to real world positions at the jobsite.
  7. The method of any one of claim 1 to 5, wherein applying one or more recognition models to the images and sensor information comprises: applying an image classification model to the accessed images, the image classification model: identifying pixels in the images representing characteristics of unloaded cargo, and determining the cargo characteristics using the identified pixels.
  8. The method of any one of the preceding claims, wherein identifying characteristics comprises receiving the characteristics from a client device or a network system.
  9. The method of any one of the preceding claims, wherein the cargo management plan is further based on cargo characteristics of cargo present in a cargo compartment area of the machine.
  10. The method of claim 9, further comprising: accessing one or more images of the cargo in the cargo compartment of the machine; and determining, based on the one or more images, cargo characteristics of the cargo present in the cargo compartment of the machine.
  11. The method of any one of the preceding claims, wherein accessing images of the jobsite comprises capturing the images using an image acquisition system of the machine as it travels through the jobsite.
  12. The method of any one of claims 1 to 10, wherein accessing images of the jobsite comprises accessing the images from an image acquisition system statically positioned at the jobsite.
  13. The method of any one of the preceding claims, wherein generating the cargo management plan for the machine occurs on a computational machine remote from the machine.
  14. A machine (100) comprising: one or more imaging systems (110) configured for capturing images of a jobsite comprising a plurality of locations; one or more sensor systems (110) configured for obtaining sensor information describing cargo present at the jobsite; one or more actuation mechanisms (120) configured for interacting with the jobsite; one or more processors; and a non-transitory computer readable storage medium storing instructions for controlling the machine to manage cargo loads at the plurality of locations of the jobsite, the instructions, when executed by the one or more processors, causing the machine to perform steps comprising: accessing (510), from the one or more imaging systems, images of the jobsite including the plurality of locations; accessing (520), from the one or more sensor systems, sensor information describing cargo present at the jobsite; applying (530) one or more recognition models to the images and the sensor information, the one or more recognition models configured to: identify (532) the plurality of locations in the jobsite; identify (534) a presence or an absence of cargo at each location; responsive to a location indicating a presence of cargo at the location, identify (536) characteristics of the cargo at the location; and generate (538) a jobsite map comprising the plurality of locations, each location indicating the presence or absence of cargo at each location and characteristics of the cargo present at the location; inputting (540) the jobsite map and an accessed jobsite protocol into a navigation model to generate a cargo management plan that adheres to the accessed jobsite protocol, the cargo management plan representing actions for the machine to implement for managing cargo at the plurality of locations; controlling (550), the one or more actuation mechanisms of the machine to traverse the jobsite to implement the cargo management plan.
  15. A non-transitory computer readable storage medium storing instructions for autonomously controlling a machine to manage cargo loads at a plurality of locations of a jobsite, the instructions, when executed by one or more processors, causing the one or more processors to perform the method according to any one of claims 1 to 13.

Description

CROSS REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Application 63/719,554, filed on November 12, 2024, which is hereby incorporated by reference herein in its entirety. BACKGROUND FIELD OF DISCLOSURE This disclosure relates to the field of identifying and selecting locations for an automated construction machine to unload cargo, and, more specifically, to determining an operation mode for an autonomous construction machine to efficiently unload cargo in cooperation with other autonomous and non-autonomous construction machines. DESCRIPTION OF RELATED ART In the sectors of construction and mining, machinery often implements a Load, Haul, Dump, and Final Load system in which hauling equipment is loaded at the site where loose resources are extracted or stockpiled in mass, transported to a primary processing location, unloaded, and eventually re-loaded into processing apparatus or stockpiled for subsequent processing near the plant. In large-scale endeavors (such as those concerning copper mining), autonomous hauling gear delivers a high volume of ore, which can be immediately processed or kept where there is sufficient room for individual mounds to be separately stored. In the coming years, smaller-scale construction, mining, aggregate production, agriculture, and other facilities will increasingly run mixed fleets of autonomous and non-autonomous machinery, encompassing haul trucks, wheel loaders, excavators, mobile crushers, and conveyance systems. Due to this combination of autonomous and non-autonomous equipment, relaying information regarding the status of transported materials which have been distributed across a jobsite will pose a major challenge. This necessitates a high level of independent operation for autonomous machines and an efficient coordination mechanism between human operators and autonomous systems. SUMMARY A vehicle (e.g., a farming, construction, mining, or forestry vehicle) moves through an environment (e.g., a farming, construction, mining, or forestry environment) and performs one or more actions (e.g., farming, construction, mining, or forestry actions) in the environment. Autonomous farming, construction, mining, or forestry vehicles (hereafter referred to as "hauling machines") can implement autonomous operating modes for performing one or more fundamental functions without human input. Such functions include transporting cargo to an unloading zone, identifying viable unloading locations within the unloading zone, and implementing an unloading plan to efficiently unload the cargo. In an embodiment, an autonomous hauling machine can map out a plurality of discrete unloading locations within a jobsite and apply machine learning image processing to the mapped locations to identify a subset of preferred unloading locations. In an embodiment, an autonomous hauling machine can identify one or more unloading locations corresponding to a specific type of cargo being transported. In an embodiment, an autonomous hauling machine can identify and navigate to one or more preferred unloading locations in cooperation with a fleet of autonomous or non-autonomous hauling machines. The descriptions above are applicable to a variety of different environments and hauling machines, such as mining vehicles (e.g., excavators and loaders), construction vehicles (e.g., motor graders), agricultural or farming vehicles (e.g., tractors), or forestry vehicles (e.g., forwarders). BRIEF DESCRIPTION OF DRAWINGS FIG. 1A illustrates a block diagram of a vehicle that performs loading or unloading actions of a protocol, in accordance with an example embodiment.FIG. 1B illustrates an isometric view of a construction vehicle, in accordance with an example embodiment.FIG. 1C illustrates an isometric view of a second construction vehicle, in accordance with an example embodiment.FIG. 2A illustrates a block diagram of a system environment for the vehicle, in accordance with an example embodiment.FIG. 2B illustrates a schematic representation of an operating environment of the vehicle, in accordance with an example embodiment.FIG. 3 illustrates a first workflow for determining an unloading plan for the vehicle to apply to one or more unloading zones of a jobsite, in accordance with one or more example embodiments.FIG. 4 illustrates a second workflow for determining an unloading plan for the vehicle to apply to one or more unloading zones of a jobsite, in accordance with one or more example embodiments.FIG. 5 illustrates a third workflow for determining control signals for a hauling machine to unload cargo at a preferred unloading location, in accordance with one or more example embodiments.FIG. 6 is a block diagram illustrating components of an example machine for reading and executing instructions from a machine-readable medium, in accordance with one or more example embodiments. The figures depict various embodiments for purposes of illustration only. One skilled in the art w