Search

EP-4742176-A1 - METHOD FOR DETECTING A PRIMARY OBJECT

EP4742176A1EP 4742176 A1EP4742176 A1EP 4742176A1EP-4742176-A1

Abstract

The present invention relates to a computer-implemented method (1000) for detecting a primary object (2040) in a field environment (2070), the method (1000) comprising: providing a trained classification model (K); and detecting the primary object (2040) by acquiring an image of the field environment (2070) and detecting the primary object (2040) in the acquired image using the trained classification model (K). The present invention further relates to an agricultural machine (2000).

Inventors

  • Felske, Mirco

Assignees

  • CLAAS E-Systems GmbH

Dates

Publication Date
20260513
Application Date
20251002

Claims (11)

  1. Computer-implemented method (1000) for detecting a primary object (2040) in a field environment (2070), the procedure (1000) encompassing: Providing a trained classification model (K), wherein the classification model (K) was trained based on a training dataset (T) comprising a set ratio (V) of primary images (B1) and secondary images (B2), wherein the primary object (2040) is represented in the primary images (B1) and the secondary object (2050) in the secondary images (B2), and wherein the quantity ratio (V) was determined using a reference model (R) by iteratively training the reference model (R) based on different sets of primary images (B1) and secondary images (B2) such that a convergence of an evaluation metric (E) based on the reference model (R) was determined, and after the convergence the quantity ratio (V) of primary images (B1) and secondary images (B2) was established; and Identifying the primary object (2040) by capturing an image of the field environment (2070) and detecting the primary object (2040) in the captured image using the trained classification model (K).
  2. Method (1000) according to claim 1, comprehensive: Setting up an agricultural machine (2000) located in the field environment (2070) based on the detected primary object (2040).
  3. Method (1000) according to claim 1 or 2, where, after convergence, the evaluation metric (E) and a class-specific evaluation metric (4010) for the primary object (2040) lie within a range of values, and/or where, after convergence, the evaluation metric (E) and a class-specific evaluation metric (4010) for the secondary object (2050) lie within a range of values, and/or where, after convergence, a class-specific evaluation metric (4010) for the primary object and a class-specific evaluation metric (4010) for the secondary object (2050) lie within a range of values.
  4. Method (1000) according to claim 1 or 2, a relation between a class-specific evaluation metric (4010) for the primary object (2040) and a class-specific evaluation metric (4010) for the secondary object (2050) was determined.
  5. Method (1000) according to any one of the preceding claims, where the evaluation metric (E) includes sensitivity, precision, accuracy or an F1 score.
  6. Method (1000) according to any one of the preceding claims, wherein each of the primary images (B1) includes a label, the label indicating that the primary object (2040) is depicted in the corresponding primary image, and/or wherein each of the primary images (B1) includes a plurality of pixels and a number of pixels collectively attributable to the primary object (2040).
  7. Computer-implemented method (5100) for creating a training data set (T), comprising: Providing a reference model (R); Providing primary images (B1) and secondary images (B2); Training the reference model (R) iteratively based on different sets of primary images (B1) and secondary images (B2) such that a convergence of an evaluation metric (E) based on the reference model (R) is determined and, after convergence, a quantity ratio (V) of primary images (B1) and secondary images (B2) is established; and Creating the training dataset (T) based on the quantity ratio (V).
  8. Computer-implemented method (5000) for training a classification model (K), comprising: Providing a classification model (K); Training the classification model (K) based on a training data set (T), wherein the training data set (T) was created using the method (5100) according to claim 7.
  9. Computer system (2030) for providing a trained classification model (K), comprising means for sending the trained classification model (K) to an agricultural machinery (2000), wherein the classification model (K) was trained using the method (5000) according to claim 8.
  10. System (2010) for data processing for an agricultural work machine (2000), comprising a processor (2020) adapted and/or configured to perform the method according to one of the preceding claims.
  11. Agricultural work machine (2000) with a system (2010) for data processing according to claim 10.

Description

The present invention relates to a computer-implemented method for detecting a primary object in a field environment. The present invention further relates to an agricultural machine. The precise and efficient detection of objects in an agricultural field environment—be they animals, people, or vehicles—plays a crucial role in a wide range of agricultural applications. Once an object is identified, agricultural machinery can be adjusted based on this information. For example, if the system detects a tree in the machine's path, the operator can adjust the settings so that the machine navigates around the tree, thus avoiding a collision and ensuring a safe workflow. Object recognition in field environments typically follows different standards than for vehicles in traffic, as the requirements and conditions differ significantly. In agriculture, the focus is on identifying natural and often irregular objects such as plants, weeds, or soil contours. The field environment requires flexible models that can handle variable lighting conditions, weather, and unstructured landscapes. In contrast, object recognition in traffic usually focuses on standardized elements such as road signs, lanes, and other vehicles. For object recognition, classification models are typically used, which are capable, for example, of analyzing and classifying images captured by a camera system. These classification models make it possible to identify which object is depicted or represented in the captured image. Classification models are typically trained based on a training dataset. This training dataset can include image data, which in turn can be divided into primary and secondary image data. A primary object is typically represented in the primary images; that is, the primary object can be represented in each of the primary images. It may be assigned to one of the primary images. It is also possible that a secondary object is depicted in the secondary images. Traditionally, the training dataset contains an equal number or amount of primary and secondary image data (i.e., primary and secondary image data each comprise 50% of the training dataset). However, such a training dataset presents several challenges. For example, it results in a large dataset. Training with such a dataset typically requires significant computing power. Stability issues can also arise during training. Additionally, problems such as insufficient generalization, increased memory requirements, and longer training times can occur. Consequently, the classification models based on the training dataset may also reach their limits. One object of the present invention is therefore to further develop the existing methods and devices in such a way that objects can be recognized particularly efficiently and precisely. This problem is solved by the embodiments disclosed herein, which are defined in particular by the subject matter of the independent claims. The dependent claims relate to further embodiments. Various aspects and embodiments of these aspects are also disclosed in the following summary and description, which offer additional features and advantages. One aspect relates to a computer-implemented method for recognizing a primary object in a field environment. This method may involve providing a trained classification model. The classification model may have been trained on a training dataset comprising a set ratio of primary and secondary images. The primary object may be represented in the primary images, and the secondary object in the secondary images. Furthermore, the set ratio may have been determined using a reference model. This reference model was iteratively trained on various sets of primary and secondary images, resulting in the convergence of an evaluation metric based on the reference model. Following this convergence, the set ratio of primary and secondary images was established. The method may also include recognizing the primary object. The primary object can be recognized by analyzing an image of the The field environment is captured and the primary object is detected in the captured image using the trained classification model. A field environment can encompass a limited natural or agricultural space where plants are cultivated, cultivated, and harvested. A field environment can include an area to be worked by agricultural machinery. A field environment can include natural, infrastructural, or artificial elements such as rivers, trees, fences, roads, irrigation systems, and other structures. The primary object can refer to an object that is located in the field environment or is temporarily present. For example, the primary object could be an animal, a person, a vehicle, or another relevant object. The primary object can be assigned to a class. The primary object could represent an obstacle for the agricultural machinery. The same applies to the secondary object; that is, the secondary object can refer to another object located in the field envi