US-12626477-B2 - Method and apparatus for automated plant necrosis

Abstract

A method of real-time plant selection and removal from a plant field including capturing a first image of a first section of the plant field, segmenting the first image into regions indicative of individual plants within the first section, selecting the optimal plants for retention from the first image based on the first image and the previously thinned plant field sections, sending instructions to the plant removal mechanism for removal of the plants corresponding to the unselected regions of the first image from the second section before the machine passes the unselected regions, and repeating the aforementioned steps for a second section of the plant field adjacent the first section in the direction of machine travel.

Inventors

• Lee Redden

Assignees

• DEERE & COMPANY

Dates

Publication Date

20260512

Application Date

20240520

Claims (18)

1. 1 . A method for a farming machine to treat a plant in a field comprising a plurality of plants, the method comprising: accessing an image of the field comprising the plurality of plants, the image captured as the farming machine moves through the field on a current pass; segmenting the image into a plurality of field regions by applying a machine learning model to the image, the machine learning model processing the image to localize the plurality of plants within the plurality of field regions, with each of the field regions localizing a plant of the plurality of plants within a two dimensional area in the image; selecting, for each field region of the plurality of field regions, a spray treatment for the plant localized in the field region based on characteristics describing the localized plant and parameters describing a plurality of prior plants treated by the farming machine on a previous pass of the farming machine through the field; and sending treatment instructions for each region of the plurality of field regions to a plurality of spray mechanisms of the farming machine, the treatment instructions indicating the spray treatment to apply to each region before the farming machine travels past the region in the field on the current pass.
2. 2 . The method of claim 1 , wherein applying the machine learning model to localize the plant within the two dimensional image comprises: classifying pixels in the image as points representing the plant; and determining a position of the points representing the plant.
3. 3 . The method of claim 2 , wherein classifying pixels in the image as points representing the plant comprises: for each pixel in the image: determining a confidence the pixel is a point of interest representing the plant; and determining, based on the confidence, the point of interest represents the plant.
4. 4 . The method of claim 2 , wherein the method further comprises: segmenting the image to localize the pixels classified as representing the plant at the determined position, the localized pixels within a portion of the two dimensional image.
5. 5 . The method of claim 1 , wherein the previous farming pass occurs during a first period of time and the current farming pass occurs during a second period of time after the first period of time.
6. 6 . The method of claim 1 , wherein selecting the spray treatment for the localized plant comprises: determining, based on information from the previous pass, if a first substance or a second substance is suitable for removing the localized plant from the field; and selecting the first substance or the second substance as the spray treatment based on the determination.
7. 7 . The method of claim 1 , wherein parameters describing the plurality of prior plants treated by the farming machine on a previous pass of the farming machine through the field comprises a yield metric for plants treated on the previous pass of the farming machine through the field.
8. 8 . The method of claim 1 , wherein parameters describing the plurality of prior plants treated by the farming machine on a previous pass of the farming machine through the field comprises a size of plants treated on the previous pass of the farming machine through the field.
9. 9 . The method of claim 8 , wherein parameters describing the size of plants treated on the previous pass are compared to an expected size of plants in the field.
10. 10 . The method of claim 1 , wherein parameters describing the plurality of prior plants treated by the farming machine on a previous pass of the farming machine through the field comprises a shape of plants treated on the previous pass of the farming machine through the field.
11. 11 . The method of claim 10 , wherein parameters describing the shape of plants treated on the previous pass are compared to an expected shape of plants in the field.
12. 12 . A farming machine: a camera configured to capture images of a field comprising a plurality of plants as the farming machine travels through the field; a plurality of spray treatment mechanism; one or more processors; and a non-transitory computer readable medium storing instructions that, when executed, cause the one or more processors to: access an image of the field comprising the plurality of plants, the image captured as the farming machine moves through the field on a current pass; segment the image into a plurality of field regions by applying a machine learning model to the image, the machine learning model processing the image to localize the plurality of plants within the plurality of field regions, with each of the field regions localizing a plant of the plurality of plants within a two dimensional area in the image; select, for each field region of the plurality of field regions, a spray treatment for the plant localized in the field region based on characteristics describing the localized plant and parameters describing a plurality of prior plants treated by the farming machine on a previous pass of the farming machine through the field; and send treatment instructions for each region of the plurality of field regions to the plurality of spray mechanisms of the farming machine, the treatment instructions indicating the spray treatment to apply to each region before the farming machine travels past the region in the field on the current pass.
13. 13 . The farming machine of claim 12 , wherein applying the machine learning model to localize the plant within the two dimensional image further causes the one or more processors to: classify pixels in the image as points representing the plant; and determine a position of the points representing the plant.
14. 14 . The farming machine of claim 13 , wherein classifying pixels in the image as points representing the plant further causes the one or more processors to: for each pixel in the image: determine a confidence the pixel is a point of interest representing the plant; and determine, based on the confidence, the point of interest represents the plant; and wherein executing the instructions further cause the one or more processors to: segment the image to localize the pixels classified as representing the plant at the determined position, the localized pixels within a portion of the two dimensional image.
15. 15 . The farming machine of claim 12 , wherein the previous farming pass occurs during a first period of time and the current farming pass occurs during a second period of time after the first period of time.
16. 16 . The farming machine of claim 12 , wherein parameters describing the plurality of prior plants treated by the farming machine on a previous pass of the farming machine through the field comprises a yield metric for plants treated on the previous pass of the farming machine through the field.
17. 17 . The farming machine of claim 12 , wherein parameters describing the plurality of prior plants treated by the farming machine on a previous pass of the farming machine through the field comprises a size and a shape of plants treated on the previous pass of the farming machine through the field.
18. 18 . A non-transitory computer readable medium storing instructions for a farming machine to treat a plant in a field comprising a plurality of plants, the instructions when executed by one or more processors, causing the one or more processors to: accessing an image of the field comprising the plurality of plants, the image captured as the farming machine moves through the field on a current pass; segmenting the image into a plurality of field regions by applying a machine learning model to the image, the machine learning model processing the image to localize the plurality of plants within the plurality of field regions, with each of the field regions localizing a plant of the plurality of plants within a two-dimensional area in the image; selecting, for each field region of the plurality of field regions, a spray treatment for the plant localized in the field region based on characteristics describing the localized plant and parameters describing a plurality of prior plants treated by the farming machine on a previous pass of the farming machine through the field; and sending treatment instructions for each region of the plurality of field regions to a plurality of spray mechanisms of the farming machine, the treatment instructions indicating the spray treatment to apply to each region before the farming machine travels past the region in the field on the current pass.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation of U.S. patent application Ser. No. 17/348,752 filed on Jun. 15, 2021 (now U.S. Pat. No. 12,020,465, issued Jun. 25, 2024), which is a continuation of U.S. patent application Ser. No. 16/720,021 filed on Dec. 19, 2019 (now U.S. Pat. No. 11,058,042, issued on Jul. 13, 2021), which is a continuation of U.S. patent application Ser. No. 15/665,025 filed on Jul. 31, 2017 (now U.S. Pat. No. 10,524,402, issued on Jan. 7, 2020), which is a continuation of U.S. patent application Ser. No. 14/713,362, filed on May 15, 2015 (now U.S. Pat. No. 9,756,771, issued on Sep. 12, 2017), which is a continuation of U.S. patent application Ser. No. 13/788,359, filed on Mar. 7, 2013 (now U.S. Pat. No. 9,064,173, issued on Jun. 23, 2015), which claims the benefit of U.S. Provisional Application Nos. 61/608,005 filed Mar. 7, 2012 and 61/609,767 filed Mar. 12, 2012, all of which are herein incorporated in their entirety by reference. TECHNICAL FIELD This invention relates generally to the agricultural field, and more specifically to a new and useful method and apparatus for automated plant necrosis inducement. BACKGROUND Induced plant necrosis, such as crop thinning, is a common practice in agriculture, in which plants are selectively removed from densely seeded plant beds to provide the remaining plants with adequate space for growth. Conventional crop thinning is performed manually, wherein a worker walks along a crop row and removes plants within the crop row with a hoe at his discretion. Not only are these methods costly and time consuming due to the use of human labor, but these methods also fail to offer a maximization of plant yield over the entire field, as the worker typically focuses on a single row and does not select plants for retention based on inter-row packing. While automatic crop thinning systems exist, these systems fail to offer the plant removal flexibility in plant selection and removal that human labor offers. In one example, a conventional crop thinning system removes plants at fixed intervals, whether or not the plant removal was necessary. In another example, a conventional crop thinning system removes plants using system vision, but fails to identify multiple close-packed plants as individual plants and treats the close-packed plants as a single plant. Therefore, there is a need in the agriculture implement field for a new and useful method and apparatus for automated inducement of plant necrosis. BRIEF DESCRIPTION OF THE FIGURES FIG. 1 is a schematic representation of the method of automated inducement of plant necrosis. FIGS. 2A, 2B, and 2C are schematic representations of a variation of segmenting the foreground from the background within an image or field of view, identifying points of interest within the image, and classifying points of interest as plant centers within the image using machine learning, respectively. FIGS. 3A, 3B, and 3C are schematic representations of a second variation of classifying the points of interest as plant centers within the image, including identifying points of interest in a first image, identifying points of interest in a second image, and classifying recurring points of interest between the two images as plant centers, respectively. FIGS. 4A and 4B are schematic representations of presenting the image and identified plant centers to a user and reassigning the points of interest as plant centers based on the user input, respectively. FIG. 5 is a schematic representation of segmenting the image into regions and sub-regions representative of plants. FIG. 6 is a schematic representation of capturing an image of a crop row segment. FIGS. 7A and 7B are schematic representations of updating a virtual map with the regions and sub-regions associated with plants from a first and second image, respectively. FIG. 8 is a schematic representation of the crop thinning mechanism. FIG. 9 is a side view of the detection mechanism utilized with a plant bed. FIG. 10 is a side view of a variation of the crop thinning mechanism including a first and a second detection mechanism and a first and a second elimination mechanism. FIG. 11 is a schematic representation of a crop row. DESCRIPTION OF THE PREFERRED EMBODIMENTS The following description of the preferred embodiments of the invention is not intended to limit the invention to these preferred embodiments, but rather to enable any person skilled in the art to make and use this invention. 1. The Plant Necrosis Inducement Method As shown in FIG. 1, the method of automated plant necrosis includes capturing an image of a plant field section, identifying individual plants within the image S100, selecting plants for retention from the image S200, removing plants from the plant field section S300, and repeating the aforementioned steps for a following plant field section S400. The plants removed by the method preferably include crops, but can alternatively include weeds or