US-12619244-B2 - System and method for autonomous surveillance of an agricultural machine

US12619244B2US 12619244 B2US12619244 B2US 12619244B2US-12619244-B2

Abstract

In one aspect, an agricultural surveillance platform useful to identify an operating condition of an agricultural machine may include an autonomous sensor frame structured to fly in formation with an agricultural machine and detect an operating condition of the agricultural machine. The agricultural surveillance platform may include a propulsion system to provide propulsive power to the autonomous sensor frame, and an inertial measurement unit configured to generate flight data measurements. The agricultural surveillance platform may also include an object detection sensor having a field of view sized to capture an image scene that includes the agricultural machine and generate scene data representing the image scene. The agricultural surveillance platform may also include a sensor frame controller configured to maintain a position of the autonomous sensor frame about the agricultural machine and generate autonomous sensor frame data that includes plugged data indicative of a plugged operating condition of the agricultural machine.

Inventors

Michael R. Cozza

Assignees

CNH INDUSTRIAL AMERICA LLC

Dates

Publication Date: 20260505
Application Date: 20230203

Claims (20)

1 . An agricultural surveillance platform useful to identify an operating condition of an agricultural machine, the agricultural surveillance platform comprising: an autonomous sensor frame structured to fly in formation with an agricultural machine and detect an operating condition of the agricultural machine, the autonomous sensor frame having: a propulsion system structured to provide propulsive power to the autonomous sensor frame; an inertial measurement unit configured to generate flight data measurements; an object detection sensor affixed to the autonomous sensor frame and having a field of view sized to capture an image scene that includes a ground-engaging tool of the agricultural machine, the object detection sensor structured to generate scene data representing the image scene; and a sensor frame controller configured to: receive the flight data measurements; receive the scene data; generate a control command based at least in part on a relative orientation of the agricultural machine identified in the scene data and the flight data measurements, the control command being useful to modulate the propulsion system and maintain a position of the autonomous sensor frame about the ground-engaging tool of the agricultural machine; and generate autonomous sensor frame data based at least in part on the scene data, the autonomous sensor frame data including plugged data indicative of a plugged operating condition of the ground-engaging tool of the agricultural machine.
2 . The agricultural surveillance platform of claim 1 , wherein the plugged data includes data associated with a likelihood of the plugged operating condition.
3 . The agricultural surveillance platform of claim 1 , wherein the sensor frame controller is configured to generate the plugged data by: identifying an implement of the agricultural machine in the scene data having the ground-engaging tool; comparing a current condition of the implement identified in the scene data to a baseline condition for the implement; and generating the plugged data indicative of the plugged operating condition of the agricultural machine when the current condition of the implement identified in the scene data differs by a predetermined threshold from the baseline condition for the implement.
4 . The agricultural surveillance platform of claim 3 , wherein the plugged data includes an indication of at least one of a confidence of the plugged operating condition or a severity of the plugged operating condition based at least in part on a difference between the current condition of the implement and the baseline condition for the implement.
5 . The agricultural surveillance platform of claim 1 , wherein the sensor frame controller is further structured to receive agricultural machine data indicative of a global position of the agricultural machine, wherein the sensor frame controller is structured to generate the control command further based at least in part on the global position of the agricultural machine.
6 . The agricultural surveillance platform of claim 1 , wherein the sensor frame controller is structured to generate a flight path command based on the relative orientation of the agricultural machine identified in the scene data, the flight path command being configured to guide the autonomous sensor frame from its current position to a different position relative to the agricultural machine.
7 . The agricultural surveillance platform of claim 6 , wherein the flight path command is further based on a guidance flight path, the guidance flight path defining a standoff distance and height to be maintained by the autonomous sensor frame relative to the agricultural machine.
8 . The agricultural surveillance platform of claim 6 , wherein the flight path command is further based on a guidance flight path, the guidance flight path defining a regularly repeating pattern for switching between monitoring a first side of the agricultural machine and a second side of the agricultural machine along a lateral direction.
9 . The agricultural surveillance platform of claim 1 , wherein the sensor frame controller is configured to generate the plugged data by: identifying an area immediately behind at least a portion of the agricultural machine including the ground-engaging tool in the scene data; comparing a current condition in the area identified in the scene data to a baseline condition; and generating the plugged data indicative of the plugged operating condition of the agricultural machine when the current condition differs by a predetermined threshold from the baseline condition.
10 . The agricultural surveillance platform of claim 1 , wherein the sensor frame controller is further configured to receive an input from an operator associated with needing to inspect the ground-engaging tool, wherein the sensor frame controller is configured to generate the control command further based at least in part on the input from the operator.
11 . An agricultural surveillance system comprising: an agricultural machine configured to work soil, the agricultural machine comprising: an agricultural machine controller configured to control a state of operation of the agricultural machine; and an autonomous sensor frame configured to navigate around the agricultural machine, the autonomous sensor frame including: a propulsion system for providing propulsive power to the autonomous sensor frame; an object detection sensor having a field of view sized to capture an image scene that includes a ground-engaging tool of the agricultural machine, the object detection sensor structured to generate a scene data representing the image scene; a sensor frame controller configured to: receive the scene data; generate a flight path command for controlling the propulsion system based at least in part on a relative orientation of the agricultural machine identified in the scene data; and generate an autonomous sensor frame data that includes a plugged data indicative of a plugged operating condition of the ground-engaging tool of the agricultural machine based at least in part on the scene data, wherein the agricultural machine controller is configured to alter the state of operation of the agricultural machine based on the plugged operating condition of the ground-engaging tool of the agricultural machine.
12 . The agricultural surveillance system of claim 11 , wherein the sensor frame controller generates the flight path command based further at least in part on position data from the agricultural machine to change a position of the autonomous sensor frame to rendezvous with the agricultural machine, the position data from the agricultural machine being indicative of a global position of the agricultural machine.
13 . The agricultural surveillance system of claim 11 , wherein the object detection sensor is affixed to a gimballed platform structured to move relative to the autonomous sensor frame.
14 . The agricultural surveillance system of claim 11 , wherein the sensor frame controller includes an object avoidance controller structured to generate an object avoidance control command based on the scene data.
15 . The agricultural surveillance system of claim 11 , wherein the plugged data includes an indication of a source of a plug.
16 . The agricultural surveillance system of claim 11 , wherein the agricultural machine comprises an implement frame extending laterally between a first side and a second side, the implement frame being configured to support a plurality of ground-engaging tools configured to engage the field, wherein the plugged data includes an indication of the plugged operating condition being on the first side or the second side.
17 . A method of operating an agricultural surveillance system, the method comprising: formation flying an autonomous sensor frame based on an agricultural machine; capturing an image scene with an object detection sensor supported on the autonomous sensor frame and generating scene data representing the image scene, the image scene including at least part of the agricultural machine including a ground-engaging tool; generating a flight path command for controlling a propulsion system of the autonomous sensor frame based at least in part on a relative orientation of the agricultural machine identified in the scene data; generating sensor frame data from the autonomous sensor frame that includes plugged data indicative of a plugged operating condition of the ground-engaging tool of the agricultural machine, the plugged data being determined based at least in part on the scene data; and altering a state of operation of the agricultural machine based on the plugged operating condition of the ground-engaging tool of the agricultural machine.
18 . The method of claim 17 , further comprising, prior to the formation flying, transmitting a global position of the agricultural machine from the agricultural machine to the autonomous sensor frame, wherein formation flying of the autonomous sensor frame comprises formation flying the autonomous sensor frame based at least in part on the global position of the agricultural machine.
19 . The method of claim 17 , wherein the formation flying includes flying the autonomous sensor frame behind an implement of the agricultural machine.
20 . The method of claim 19 , wherein the scene data includes a residue distribution behind the implement, the plugged data being determined based at least in part on the residue distribution.

Description

FIELD OF THE INVENTION The present disclosure generally relates to surveillance of agricultural machines and, more particularly, to systems and methods for autonomous surveillance of an agricultural machine to detect a plugged condition, such as a plugged condition of an implement used in the machine. BACKGROUND OF THE INVENTION It is well known that, to attain the best agricultural performance from a field, a farmer must cultivate the soil, typically through a tillage operation. Modern farmers perform tillage operations by pulling a tillage implement behind an agricultural work vehicle, such as a tractor. Tillage implements typically include one or more ground engaging components configured to rotate relative to the soil as the implement is moved across the field. For example, in certain configurations, the implement may include one or more harrow discs, leveling discs, rolling baskets, and/or the like. Such rotating ground engaging component(s) loosen and/or otherwise agitate the soil to prepare the field for subsequent planting operations. During tillage operations, field materials, such as residue, soil, rocks, and/or the like, may become trapped or otherwise accumulate between adjacent rotating ground engaging components. Such accumulations of field materials may inhibit the operation of the rotating ground engaging components in a manner that prevents the components from providing adequate tillage to the field. In such instances, it is necessary for the operator to take certain corrective actions to remove the accumulated field materials. However, it may be difficult for the tillage implement operator to determine when field materials have accumulated between the rotating ground engaging components. Accordingly, an improved system and method for detecting accumulations of field materials between ground engaging components of an agricultural implement would be welcomed in the technology. SUMMARY OF THE INVENTION Aspects and advantages of the technology will be set forth in part in the following description, or may be obvious from the description, or may be learned through practice of the technology. In one aspect, the present subject matter is directed to an agricultural surveillance platform useful to identify an operating condition of an agricultural machine. The agricultural surveillance platform may include an autonomous sensor frame structured to fly in formation with an agricultural machine and detect an operating condition of the agricultural machine. The agricultural surveillance platform may include a propulsion system structured to provide propulsive power to the autonomous sensor frame. The agricultural surveillance platform may also include an inertial measurement unit configured to generate flight data measurements. The agricultural surveillance platform may further also include an object detection sensor affixed to the autonomous sensor frame and having a field of view sized to capture an image scene that includes the agricultural machine, the object detection sensor structured to generate scene data representing the image scene. The agricultural surveillance platform may still further also include a sensor frame controller configured to receive the flight data measurements and generate a control command useful to modulate the propulsion system and maintain a position of the autonomous sensor frame about the agricultural machine. The sensor frame controller can be configured to receive the scene data and generate autonomous sensor frame data that includes plugged data indicative of a plugged operating condition of the agricultural machine. In another aspect, the present subject matter is directed to an agricultural surveillance system. The agricultural surveillance system may include an autonomous sensor frame configured to navigate around an agricultural machine. The autonomous sensor frame may include a propulsion system for providing propulsive power to the autonomous sensor frame. The autonomous sensor frame may also include an object detection sensor having a field of view sized to capture an image scene that includes the agricultural machine. The object detection sensor may be structured to generate a scene data representing the image scene. The autonomous sensor frame may also include a sensor frame controller configured to receive the scene data and generate an autonomous sensor frame data that includes a plugged data indicative of a plugged operating condition of the agricultural machine. The autonomous sensor frame may also include an autonomous sensor frame transceiver structured to transmit the autonomous sensor frame data that includes the plugged data upon command of the sensor frame controller. The agricultural machine can be configured to work a soil. The agricultural machine may include an agricultural machine transceiver structured to receive the autonomous sensor frame data that includes the plugged data from the autonomous sensor frame. The agricultural machine may als