Search

CN-119497840-B - Local area mapping for robotic lawnmowers

CN119497840BCN 119497840 BCN119497840 BCN 119497840BCN-119497840-B

Abstract

A robotic lawnmower and a method of controlling a robotic lawnmower based on generating a local map. The method includes receiving an image from an imaging sensor on the robotic lawnmower, the image including a ground in an upcoming path, applying a semantic segmentation algorithm to generate a segmented image from the received image, the segmented image including regions corresponding to features in the image, applying a perspective transformation to the segmented image to obtain a top-down transformed image, determining a location of the region relative to a current location of the robotic lawnmower from the transformed image, drawing a local map of the robotic lawnmower environment based on the location of the region relative to the current location of the robotic lawnmower, and controlling the robotic lawnmower to navigate the lawn region using the local map.

Inventors

  • HOFFMANN JOHANN

Assignees

  • 苏州宝时得电动工具有限公司

Dates

Publication Date
20260508
Application Date
20230607
Priority Date
20220608

Claims (20)

  1. 1. A computer-implemented method of controlling a robotic lawnmower, the method comprising: Receiving an image from an imaging sensor onboard a robotic lawnmower, the image comprising a ground on a path that the robotic lawnmower is about to traverse; applying a semantic segmentation algorithm to generate a segmented image from the received image, the segmented image comprising regions corresponding to features in the image; Performing perspective transformation on the segmented image to obtain a top-view transformed image, wherein the region is reserved in the transformed image; Determining the position of each region relative to the current position of the robotic lawnmower according to the transformed image; drawing a local map of the environment of the robotic lawnmower based on the location of each of the areas relative to the current location of the robotic lawnmower, the local map being an environment map within a limited distance or radius from the robotic lawnmower, wherein the distance or radius is less than the size of the lawn area, and And controlling the robot mower to navigate in the lawn area by using the local map.
  2. 2. The method of claim 1, wherein the area comprises one or more areas containing non-grass features, wherein the non-grass features comprise one or more of obstacles, hazards, and/or grass area boundaries.
  3. 3. The method of claim 1, wherein the size of the local map is limited according to a threshold distance from the current location of the robotic lawnmower such that the local map includes an area that is below the threshold distance from the current location of the robotic lawnmower.
  4. 4. The method of claim 1, wherein drawing the local map comprises: acquiring a previous iteration of the local map, drawn using previously received image data, and One or more regions in the transformed image corresponding to the received image are added to a previous iteration of the local map.
  5. 5. The method of claim 4, wherein drawing the local map comprises: Deleting portions of the partial map corresponding to one or more regions of the previous iteration that are greater than a threshold distance from the current position of the robotic lawnmower, and One or more regions in the transformed image corresponding to the received image are added to a previous iteration of the local map, the one or more regions being less than a threshold distance from a current position of the robotic lawnmower.
  6. 6. The method of claim 4, wherein drawing the local map comprises: recording a number of previous images of a previous iteration for rendering the local map; Comparing the number of recorded images with the maximum number of images, and When the number of recorded images of a previous iteration for rendering the partial map is equal to the maximum number of images: after receiving an image from the imaging sensor, deleting a portion of the partial map corresponding to an oldest image of the previous images, and One or more regions in the transformed image corresponding to the received image are added to a previous iteration of the local map so as not to exceed a maximum number of images of the local map.
  7. 7. The method of claim 1, wherein determining a position of each of the areas relative to a current position of the robotic lawnmower based on the transformed image comprises: And processing the transformed image by using a VSLAM algorithm to obtain the position of each region in the transformed image relative to the current position of the robotic lawnmower.
  8. 8. The method of claim 1 further comprising: Acquiring additional data related to the movement of the robotic lawnmower, and And updating the position of each area in the local map relative to the current position of the robot lawnmower according to the additional data, wherein the additional data comprises sensor data from one or more additional sensors, and the sensor data comprises one or more of mileage data, IMU data and GPS data.
  9. 9. The method of claim 8, wherein updating the location of each region in the local map relative to the current location of the robotic lawnmower based on the additional data comprises: for each region in the local map: acquiring the previous position of the region from the local map, and The previous location of the area is modified based on the additional sensor data to determine an updated location of the area.
  10. 10. The method of claim 8, wherein the additional data comprises region tracking data for one or more regions in the received image.
  11. 11. The method of claim 10, further comprising: the region tracking data is generated for one or more regions present in the received image by: Identifying one or more regions in one or more previously received images; one or more regions are tracked by one or more previously received images and the received images to determine a tracking path for each of the one or more regions.
  12. 12. The method of claim 11 further comprising: Inferring a tracking path for the one or more regions when the one or more regions no longer appear in the subsequently received image, and And updating the position of one or more areas in the local map relative to the current position of the robot lawnmower according to the inferred tracking path.
  13. 13. The method of claim 8, wherein the frequency of updating the location of the area in the local map relative to the current location of the robotic lawnmower based on the additional data is higher than the frequency of updating the location of the area in the local map relative to the current location of the robotic lawnmower based on the received image.
  14. 14. The method of claim 1, wherein controlling the robotic lawnmower to navigate within a lawn area using the local map comprises: accessing the local map, and And controlling the robot mower to navigate in the lawn area according to the area position indicated by the local map.
  15. 15. The method of claim 14, wherein controlling the robotic lawnmower to navigate within a lawn area based on the area location indicated by the local map comprises: one or more drive mechanisms of the robotic lawnmower are controlled to move the robotic lawnmower to navigate within a lawn area.
  16. 16. A robotic lawnmower comprising: one or more drive mechanisms; An imaging sensor; memory module, and A control module comprising a processor communicatively coupled to one or more of the drive mechanism, the imaging sensor, and the storage module, the processor configured to perform the method of claim 1.
  17. 17. The robotic lawnmower of claim 16, wherein the imaging sensor is a camera configured to capture a wide angle image with a field of view exceeding 100 degrees.
  18. 18. The robotic lawnmower of claim 16, further comprising: One or more additional sensors communicatively coupled to the processor, the one or more additional sensors configured to provide sensor data related to at least one of: The speed of the robotic lawnmower; the orientation of the robotic lawnmower; the direction of the robotic lawnmower; the distance travelled by the robotic lawnmower; Acceleration of the robotic lawnmower, and The distance between the robotic lawnmower and the surrounding one or more obstacles.
  19. 19. The robotic lawnmower of claim 16, wherein the control module is configured to communicate with an external server.
  20. 20. A computer readable medium having stored thereon instructions which, when executed by a processor, cause the processor to perform the method of claim 1.

Description

Local area mapping for robotic lawnmowers Technical Field The present application relates to an apparatus, system and method for autonomously controlling a robotic lawnmower. Background A robot or robotic lawnmower is an autonomous robot configured to mow a lawn or field. With robotic lawnmowers, there is no need for a human to mow himself, which can be a trivial and burdensome task. Fig. 1 shows an example of a lawn area where a robotic lawnmower may operate. The robotic lawnmower 1 mows within a lawn area 3, which is designated by a boundary line or guide wire 4. Before the robotic lawnmower is started, the borderline or guide wire 4 needs to be actually installed. 1. After the robot lawnmower 1 detects the boundary line 4, it will typically randomly change direction to avoid the boundary line. The robotic lawnmower may include a plurality of sensors for detecting obstacles. For example, these sensors may be laser or radar based sensors. Likewise, the robotic lawnmower may rely on satellite communication to navigate within the lawn area and may include a radio sensor for navigating to a charging station. In fig. 1, the charging station 2 is located at the periphery of the lawn area 3. However, there is a need to balance the complexity of robotic lawnmowers with their relative capabilities. Robotic lawnmowers containing different types of sensors or containing more complex systems are often more expensive. Accordingly, there is a need for a robotic lawnmower that works efficiently without the need for overly complex systems and sensors. Disclosure of Invention This summary presents some concepts in a simplified form that are further described below in the detailed description. In a first aspect, the present disclosure provides a computer-implemented method of controlling a robotic lawnmower, the method comprising receiving an image from an imaging sensor located on the robotic lawnmower, the image comprising a ground area in a path of travel of the robotic lawnmower, generating a segmented image from the received image using a semantic segmentation algorithm, the segmented image comprising areas corresponding to features in the image, applying a perspective transformation to the segmented image to obtain a top-down transformed image, wherein the areas are retained in the transformed image, determining a position of the areas relative to a current position of the robotic lawnmower from the transformed image, drawing a local map of an environment in which the robotic lawnmower is located according to the position of the areas relative to the current position of the robotic lawnmower, and controlling the robotic lawnmower to navigate the lawn area using the local map. The local map provides the robotic lawnmower with relevant data regarding the location of areas and features in the physical environment relative to the robotic lawnmower so that the robotic lawnmower can be controlled toward or away from these areas and features. The regions may include one or more regions containing grass features and one or more regions containing non-grass features. By dividing the area into two categories, grassland and non-grassland, the robotic lawnmower can be controlled to avoid the non-grassland area and cut the grassland area. The non-grass features may include one or more of obstacles, hazards, and/or grass area boundaries. Thus, the non-grass areas may be further classified to determine the type of non-grass feature. The control of the robotic lawnmower may vary depending on its proximity to different types of non-grass features. Further, the local map may be limited in size according to a threshold distance from the current location of the robotic lawnmower such that the local map includes areas having a distance from the current location of the robotic lawnmower that is below the threshold distance. The regions within the threshold distance may include one or more regions that include grass and one or more regions that include non-grass features, which may include one or more of obstacles, hazards, and/or boundaries of lawn areas. Applying a threshold distance to the size of the local map limits the memory requirements for storing the local map in the robot lawnmower memory, while maintaining a map of the area and features closest to the current location of the robot lawnmower. This is more efficient than saving and storing larger maps. Rendering the local map may further include acquiring a previous iteration of the local map, rendering using data from a previously received image, and adding one or more regions to the previous iteration of the local map, the regions from transformed images corresponding to the received image. Thus, the method is iterative, i.e. each time a new image is captured from the imaging sensor of the robotic lawnmower, the local map is updated with the area segmented from the new image. The previous iteration of the local map is also updated according to the latest position of the robot l