EP-4737855-A1 - MOWING ROBOT POSITIONING METHOD AND APPARATUS, AND ELECTRONIC DEVICE AND READABLE STORAGE MEDIUM

EP4737855A1EP 4737855 A1EP4737855 A1EP 4737855A1EP-4737855-A1

Abstract

Embodiments of the present disclosure provide a method for localizing a lawnmower robot, an apparatus, and an electronic device and a readable storage medium. In the method according to the embodiments, during the operation of the lawnmower robot, corresponding data is acquired by a wheel speedometer, an inertial positioning device, and an image sensing device disposed on the lawnmower robot; the acquired wheel speedometer data is analyzed to derive a first residual term, the inertial positioning data is analyzed to derive a second residual term, and the image data is analyzed to determine a third residual term; and a cost function is constructed based on the first, second, and third residual terms, and the current position information of the lawnmower robot is obtained by minimizing the cost function. By tightly coupling the data collected from multiple sensors in the manner described above, the present disclosure is advantageous for improving the accuracy of localization results of the lawnmower robot.

Inventors

SU, Hans
LI, JACK
GU, JEFF
ZHAO, XUN
FAN, Wolory
REN, Sunny

Assignees

Yosemite Shanghai Robotics Co., Ltd

Dates

Publication Date: 20260506
Application Date: 20240605

Claims (10)

A method for localizing a lawnmower robot, comprising: acquiring wheel speedometer data and inertial positioning data of the lawnmower robot, and image data captured by an image sensing device disposed on the lawnmower robot; deriving a first residual term based on the wheel speedometer data; deriving a second residual term based on the inertial positioning data; calculating a visual reprojection error based on the image data; deriving a third residual term based on the visual reprojection error; constructing a cost function based on the first residual term, the second residual term, and the third residual term; and determining current position information of the lawnmower robot by minimizing the cost function.
The method according to claim 1, wherein the constructing the cost function based on the first residual term, the second residual term, and the third residual term, and determining the current position information of the lawnmower robot by minimizing the cost function comprise: determining a first weight, a second weight, and a third weight, wherein the first weight is related to the wheel speedometer data, the second weight is related to the inertial positioning data, and the third weight is related to the image data; deriving a first data term based on the first residual term and the first weight; deriving a second data term based on the second residual term and the second weight; deriving a third data term based on the third residual term and the third weight; and summing the first data term, the second data term, and the third data term to obtain the cost function.
The method according to claim 1, wherein the deriving the first residual term based on the wheel speedometer data comprises: integrating the wheel speedometer data to obtain first position information; and deriving the first residual term based on the first position information and an initial position estimate.
The method according to claim 1, wherein the deriving the second residual term based on the inertial positioning data comprises: pre-integrating the inertial positioning data to obtain second position information; and deriving the second residual term based on the second position information and an initial position estimate.
The method according to claim 1, wherein the calculating the visual reprojection error based on the image data and deriving the third residual term based on the visual reprojection error comprise: performing image recognition on the image data to identify an image containing a reference landmark and to determine an actual pixel position of the reference landmark in the image, wherein the reference landmark is a reference object within an operating area of the lawnmower robot; determining an estimated pixel position of the reference landmark in the image; deriving the visual reprojection error based on the estimated pixel position and the actual pixel position; and determining the visual reprojection error as the third residual term.
The method according to claim 1, wherein before constructing the cost function based on the first residual term, the second residual term, and the third residual term, the method further comprises: acquiring a marginalized residual term; wherein the constructing the cost function based on the first residual term, the second residual term, and the third residual term comprises: constructing the cost function based on the first residual term, the second residual term, the third residual term, and the marginalized residual term.
The method according to claim 2, wherein determining the first weight, the second weight, and the third weight comprises: determining the first weight based on a noise covariance corresponding to a wheel speedometer of the lawnmower robot; determining the second weight based on a covariance of a pre-integration noise term of the inertial positioning data; and determining the third weight based on a noise covariance corresponding to the image sensing device on the lawnmower robot.
An apparatus for localizing a lawnmower robot, comprising: an acquiring module, configured to acquire wheel speedometer data and inertial positioning data of the lawnmower robot, and image data captured by an image sensing device disposed on the lawnmower robot; a residual calculating module, configured to: derive a first residual term based on the wheel speedometer data, derive a second residual term based on the inertial positioning data, and calculate a visual reprojection error based on the image data and derive a third residual term based on the visual reprojection error; and a position determining module, configured to construct a cost function based on the first residual term, the second residual term, and the third residual term, and to determine current position information of the lawnmower robot by minimizing the cost function.
An electronic device, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor, when running the computer program, is caused to perform the steps of the method for localizing a lawnmower robot as defined in any one of claims 1 to 7.
A readable storage medium, storing a computer program thereon, wherein the computer program, when run by a processor, causes the processor to perform the steps of the method for localizing a lawnmower robot as defined in any one of claims 1 to 7.

Description

This application is based upon and claims priority to Chinese Patent Application No. 2023107496312, filed before China National Intellectual Property Administration on June 21, 2023 and entitled "METHOD AND APPARATUS FOR LOCALIZING LAWNMOWER ROBOT, AND ELECTRONIC DEVICE AND READABLE STORAGE MEDIUM," the entire contents of which are incorporated herein by reference. TECHNICAL FIELD The present disclosure relates to the technical field of lawnmower robots, and in particular, relates to a method and apparatus for localizing a lawnmower robot, and an electronic device and a readable storage medium therefor. BACKGROUND A lawnmower robot is an integrated robotic system that combines multiple functions such as environment perception, dynamic path planning, and behavior control, liberating users from the laborious tasks of lawn maintenance. Accurate localization is critical for the lawnmower robot in various scenarios, such as planning an operating path while performing lawn maintenance within a defined area, and automatically returning to a charging dock to recharge. Therefore, accurate localization is of extreme importance to the lawnmower robot. Conventionally, a lawnmower robot acquires satellite positioning signals from a mobile station. These satellite positioning signals are then verified using verification data from a real-time kinematic (RTK) base station to obtain a final localization result. However, factors such as signal obstruction by buildings and atmospheric interference can affect the satellite positioning signals. This may cause localization drift in the lawnmower robot, resulting in poor localization accuracy. SUMMARY In view of the above, embodiments of the present disclosure provide a method and apparatus for localizing lawnmower robot, and an electronic device and a computer-readable storage medium therefor, which improve the accuracy of localization of the lawnmower robot. In a first aspect of the embodiments of the present disclosure, a method for localizing a lawnmower robot is provided. The method includes: acquiring wheel speedometer data and inertial positioning data of the lawnmower robot, and image data captured by an image sensing device disposed on the lawnmower robot; deriving a first residual term based on the wheel speedometer data; deriving a second residual term based on the inertial positioning data; calculating a visual reprojection error based on the image data, and deriving a third residual term based on the visual reprojection error; and constructing a cost function based on the first residual term, the second residual term, and the third residual term, and determining current position information of the lawnmower robot by minimizing the cost function. In some embodiments, the constructing the cost function based on the first residual term, the second residual term, and the third residual term, and determining the current position information of the lawnmower robot by minimizing the cost function include: determining a first weight, a second weight, and a third weight, wherein the first weight is related to the wheel speedometer data, the second weight is related to the inertial positioning data, and the third weight is related to the image data; deriving a first data term based on the first residual term and the first weight, deriving a second data term based on the second residual term and the second weight, and deriving a third data term based on the third residual term and the third weight; and summing the first data term, the second data term, and the third data term to obtain the cost function. In some embodiments, the deriving the first residual term based on the wheel speedometer data includes: integrating the wheel speedometer data to obtain first position information; and deriving the first residual term based on the first position information and an initial position estimate. In some embodiments, the deriving the second residual term based on the inertial positioning data includes: pre-integrating the inertial positioning data to obtain second position information; and deriving the second residual term based on the second position information and an initial position estimate. In some embodiments, the calculating the visual reprojection error based on the image data, and the deriving the third residual term based on the visual reprojection error include: performing image recognition on the image data to identify an image containing a reference landmark and to determine an actual pixel position of the reference landmark in the image, wherein the reference landmark is a reference object within an operating area of the lawnmower robot; determining an estimated pixel position of the reference landmark in the image, and deriving the visual reprojection error based on the estimated pixel position and the actual pixel position; and determining the visual reprojection error as the third residual term. In some embodiments, before constructing the cost function based on the