US-12620237-B2 - Agricultural machine

US12620237B2US 12620237 B2US12620237 B2US 12620237B2US-12620237-B2

Abstract

An agricultural machine includes an image recognition system to detect a row region including at least one of a crop or a ridge on a ground surface of a field, traveling equipment including a wheel responsible for steering, and a controller configured or programmed to control the traveling equipment and operate in a row-following travel mode to control the traveling equipment to travel along the row region as detected by the image recognition system. When the image recognition system detects a deletion site of the row region or the field's final end while operating in the row-following travel mode, the controller is configured or programmed to continue travel for a predetermined time or a predetermined distance.

Inventors

Shunsuke Miyashita
Toru TAMBO
Mitsuaki NAGAO

Assignees

KUBOTA CORPORATION

Dates

Publication Date: 20260505
Application Date: 20240617
Priority Date: 20211224

Claims (13)

1 . An agricultural machine comprising: an image recognition system to detect, from an acquired image, a row region including at least one of a crop or a ridge on a ground surface of a field; traveling equipment including a wheel responsible for steering; and a controller configured or programmed to control the traveling equipment and operate in a row-following travel mode to control the traveling equipment to travel along the row region as detected by the image recognition system; wherein when the image recognition system detects a deletion site of the row region or the field's final end while operating in the row-following travel mode, the controller is configured or programmed to continue travel for a predetermined time or a predetermined distance; and the controller is configured or programmed to stop after continuing the travel.
2 . The agricultural machine of claim 1 , wherein, when continuing the travel, the controller is configured or programmed to control the traveling equipment to travel along an extension of a path that has been traveled by the agricultural machine.
3 . The agricultural machine of claim 1 , further comprising a notifier to, when the image recognition system detects the field's final end, give a user an advance notice to stop the travel.
4 . The agricultural machine of claim 1 , wherein the controller is configured or programmed to determine the predetermined time or the predetermined distance in accordance with an implement towed by the agricultural machine.
5 . The agricultural machine of claim 1 , wherein the controller is configured or programmed to operate in the row-following travel mode when the image recognition system detects the row region and also detects a work path region having a predetermined width or greater on both sides or one side of the row region.
6 . The agricultural machine of claim 5 , wherein the controller is configured or programmed to determine whether the wheel responsible for steering is able to pass through the work path region or not based on positions of the work path region and the wheel responsible for steering, and if it is determined that the wheel responsible for steering is not able to pass through the work path region, stop the row-following travel mode.
7 . An agricultural machine comprising: an image recognition system to detect, from an acquired image, a row region including at least one of a crop or a ridge on a ground surface of a field; traveling equipment including a wheel responsible for steering; and a controller configured or programmed to control the traveling equipment and operate in a row-following travel mode to control the traveling equipment to travel along the row region as detected by the image recognition system; wherein when the image recognition system detects a deletion site of the row region or the field's final end while operating in the row-following travel mode, the controller is configured or programmed to continue travel for a predetermined time or a predetermined distance; and if the image recognition system detects a new row region while continuing the travel, the controller is configured or programmed to control the traveling equipment to travel along the new row region.
8 . An agricultural machine comprising: an image recognition system including: an imaging device to image a ground surface traveled by an agricultural machine to acquire time-series images including a portion of the ground surface; and a processor to perform image processing for the time-series images; wherein the processor is configured or programmed to: from the time-series images, select a search region that is split into a plurality of blocks; in each of the plurality of blocks, perform a regional classification for the ground surface; and based on a result of the regional classification in the search region, detect a row region including at least one of a crop or a ridge on a ground surface of a field; from the time-series images, the processor is configured or programmed to generate a plan view image of at least the search region of the ground surface; and the plurality of blocks are arranged in rows and columns in the plan view image.
9 . The agricultural machine of claim 8 , wherein, based on a two-dimensional block arrangement pattern within the search region by the row region detected in the search region, the processor is configured or programmed to generate a signal to control travel of the agricultural machine.
10 . The agricultural machine of claim 8 , wherein, based on a two-dimensional block arrangement pattern within the search region by the row region detected in the search region, the processor is configured or programmed to determine a direction in which the row region extends in the search region.
11 . The agricultural machine of claim 8 , wherein, based on a two-dimensional block arrangement pattern within the search region by the row region detected in the search region, the processor is configured or programmed to detect at least one of an edge line, a beginning end, a final end, and a deletion site of the row region.
12 . The agricultural machine of claim 8 , wherein, when performing the regional classification for the ground surface, the processor is configured or programmed to: generate a color-enhanced image in which a color of the row region is enhanced, from at least the search region of the time-series images; classify the color-enhanced image into first pixels having an index value regarding the color that is equal to or greater than a threshold, and second pixels of which index value is below the threshold; and detect the row region based on the index values of the first pixels.
13 . The agricultural machine of claim 8 , wherein the processor is configured or configured to: from a plurality of images among the time-series images that have been acquired at different points in time, determine a first amount of movement of each of a plurality of feature points in an image plane, through feature point matching; through perspective projection of each of the plurality of feature points from the image plane onto a reference plane corresponding to the ground surface, determine a second amount of movement of each projection point in the reference plane based on the first amount of movement; and based on the second amount of movement, estimate heights of the plurality of feature points from the reference plane to detect the row region.

Description

CROSS REFERENCE TO RELATED APPLICATIONS This application claims the benefit of priority to Japanese Patent Application Nos. 2021-210083 and 2021-210084 filed on Dec. 24, 2021 and is a Continuation Application of PCT Application No. PCT/JP2022/045042 filed on Dec. 7, 2022. The entire contents of each application are hereby incorporated herein by reference. BACKGROUND OF THE INVENTION 1. Field of the Invention The present disclosure relates to agricultural machines. 2. Description of the Related Art Research and development has been directed to the automation of work vehicles, such as tractors, to be used in fields. For example, work vehicles have been put to practical use which travel via automatic steering by utilizing a positioning system capable of precise positioning, e.g., GNSS (Global Navigation Satellite System). Work vehicles that automatically perform speed control as well as automatic steering have also been put to practical use. Moreover, vision guidance systems are being developed which detect rows of crops (crop rows) or ridges in a field by using an imaging device such as a camera, and control the travel of a work vehicle along the detected crop rows or ridges. Japanese Laid-Open Patent Publication No. 2016-208871 discloses a work machine that travels along a ridge in cultivated land where crops are planted in ridges which are formed in rows. Japanese Laid-Open Patent Publication No. 2016-208871 describes binarizing a raw image acquired by capturing cultivated land from obliquely above with an onboard camera, and thereafter generating a planar perspective projection image. SUMMARY OF THE INVENTION When an agricultural machine is traveling via automatic steering along a row region such as a crop row or a ridge by using an image recognition technique, it may not be possible for an image recognition system to continuously keep detecting a row region. Example embodiments of the present invention provide agricultural machines that can solve such problems. In an illustrative, non-limiting example embodiment, an agricultural machine according to an example embodiment of the present disclosure includes an image recognition system to detect, from an acquired image, a row region including at least one of a crop or a ridge on a ground surface of a field, traveling equipment including a wheel responsible for steering, and a controller configured or programmed to control the traveling equipment and operate in a row-following travel mode to control the traveling equipment to travel along the row region as detected by the image recognition system, wherein, when the image recognition system detects a deletion site of the row region or the field's final end while operating in the row-following travel mode, the controller is configured or programmed to continue travel for a predetermined time or a predetermined distance. General or specific aspects of various example embodiments of the present disclosure may be implemented using a device, a system, a method, an integrated circuit, a computer program, a non-transitory computer-readable storage medium, or any combination thereof. The computer-readable storage medium may be inclusive of a volatile storage medium, or a non-volatile storage medium. The device may include a plurality of devices. In the case where the device includes two or more devices, the two or more devices may be disposed within a single apparatus, or divided over two or more separate apparatuses. According to example embodiments of the present disclosure, while an agricultural machine is performing auto-steering travel along a row region such as a crop row or a ridge, even if an image recognition system detects a deletion site of the row region or the field's final end, the automatic steering mode will not be abruptly disengaged to upset the driver. As a result of this, the driver will have ample time to perform an operation to switch to manual steering, thus allowing travel via manual steering, or manipulations such as stopping of travel, etc., to be smoothly performed. The above and other elements, features, steps, characteristics and advantages of the present invention will become more apparent from the following detailed description of the example embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram schematically showing an example of a basic configuration of an agricultural machine according to an example embodiment of the present disclosure. FIG. 2 is a block diagram showing an example configuration of an image recognition system. FIG. 3 is a diagram schematically showing an example arrangement of switches and the like provided in the neighborhood of a steering wheel of the agricultural machine. FIG. 4 is a perspective view showing another example of a display device. FIG. 5 is a diagram schematically showing a displayed image on an operational terminal indicating an operating state of the agricultural machine. FIG. 6 is a diagram