CN-121994213-A - Relative attitude measurement system based on vision IMU

Abstract

A system includes a chassis of a vehicle, a movable member movably coupled to the chassis, a feature board attached to the vehicle, and a visual IMU mounted on the vehicle. The visual IMU includes a camera having a field of view including a feature board. An auxiliary IMU is also mounted on the vehicle. A processor on the vehicle communicates with the vision IMU and the auxiliary IMU. The processor hosts a program module that includes a visual inertial algorithm that performs a process including receiving visual information from a camera including features detected in an image of a feature plate captured by the camera, receiving inertial measurements from the visual IMU and the auxiliary IMU including acceleration data and angular rate data, and estimating an orientation of the movable member relative to the chassis by integrating the visual information with the inertial measurements.

Inventors

• CAO XIAO
• HU JUAN
• YU SIQIN

Assignees

• 霍尼韦尔国际公司

Dates

Publication Date

20260508

Application Date

20251013

Priority Date

20241105

Claims (3)

1. 1. A system, the system comprising: A chassis of the vehicle; the movable member(s) may be a movable member, the movable member is movably coupled to the chassis; a feature plate attached to the vehicle; A visual Inertial Measurement Unit (IMU) mounted on the vehicle, the visual IMU including a camera and a first set of inertial sensors, the camera configured to have a field of view including the feature board; an auxiliary IMU mounted on the vehicle, the auxiliary IMU including a second set of inertial sensors, and At least one processor on the vehicle, the at least one processor in operative communication with the visual IMU and the auxiliary IMU; Wherein the at least one processor hosts a program module that includes a visual inertia algorithm operable to perform a process comprising: Receiving visual information from the camera, the visual information including features detected in an image of the feature board captured by the camera; receiving inertial measurements from the vision IMU and the auxiliary IMU, the inertial measurements including acceleration data and angular rate data, and An orientation of the movable member relative to the chassis is estimated by integrating the visual information with the inertial measurement value.
2. 2. The system of claim 1, wherein the visual inertia algorithm is performed by: A tight coupling process in which the visual information and the inertial measurements are combined and processed together in a single optimization filter to estimate the orientation of the movable member relative to the chassis, or A loose coupling process in which the visual information and the inertial measurements are processed independently to estimate the orientation of the movable member relative to the chassis, and then fusing these estimates together.
3. 3. A method, the method comprising: providing a relative attitude measurement system, the relative attitude measurement system comprising: a feature plate attached to a movable member or chassis of a vehicle; A visual Inertial Measurement Unit (IMU) mounted on the vehicle, the visual IMU including a camera having a field of view including the feature board and a plurality of inertial sensors; An auxiliary IMU mounted on the vehicle and including a plurality of inertial sensors, and The vehicle-mounted processing unit is used for hosting a visual inertial algorithm, and the visual inertial algorithm executes a tight coupling function or a loose coupling function; performing a real-time data collection process, the real-time data collection process comprising: Determining whether the camera of the visual IMU receives an image; in response to determining that the camera receives an image, sending image data corresponding to the received image to the processing unit; Determining whether inertial data is received by the inertial sensor of the visual IMU; responsive to determining that the inertial sensor of the visual IMU receives inertial data, transmitting the inertial data from the visual IMU to the processing unit; Determining whether inertial data is received by the inertial sensor of the auxiliary IMU, and Responsive to determining that the inertial sensor of the auxiliary IMU receives inertial data, transmitting the inertial data from the auxiliary IMU to the processing unit; Wherein the data sent to the processing unit is analyzed and processed by the close-coupled function or the loose-coupled function to calculate an enhanced relative heading/attitude estimate, and The enhanced relative heading/pose estimate is sent from the processing unit to a control system of the vehicle.

Description

Relative attitude measurement system based on vision IMU Background Construction machines are widely used in industrial applications such as mining and construction. Primary operators often cannot accurately operate blades or buckets in bulldozers or excavators. Autopilot system providers have provided several off-the-shelf products that are integrated into the construction machine and that help operators to automatically operate the blade or bucket in many applications. One of the most important technical contributors is attitude sensing of each component of the construction machine. For example, bulldozers typically include a chassis, a C-frame, and a blade. In some existing designs, a dual antenna Global Navigation Satellite System (GNSS) receiver is used to obtain the heading of the blade and an additional tilt sensor is used to obtain the pitch/roll angle of the blade. These designs also require the installation of GNSS receivers and tilt sensors in the chassis to calculate the relative movement between the chassis and the blade to sense the relative movement and provide a control scheme. The use of GNSS receivers and tilt sensors in construction machines has several drawbacks. These disadvantages include high price and the need for an additional mast for dual antenna installation, which hampers operation and may be easily damaged. Furthermore, these designs are not able to support operations in a GNSS rejection scenario (e.g., in a tunnel). Disclosure of Invention A system includes a chassis of a vehicle, a movable member movably coupled to the chassis, a feature board attached to the vehicle, and a visual Inertial Measurement Unit (IMU) mounted on the vehicle. The visual IMU includes a camera and a first set of inertial sensors, wherein the camera is configured to have a field of view including the feature board. An auxiliary IMU is mounted on the vehicle, wherein the auxiliary IMU includes a second set of inertial sensors. At least one processor on the vehicle is in operative communication with the visual IMU and the auxiliary IMU. The at least one processor hosts a program module including a visual inertial algorithm operable to perform a process including receiving visual information from the camera, the visual information including features detected in images of the feature board captured by the camera, receiving inertial measurements from a visual IMU and the auxiliary IMU, the inertial measurements including acceleration data and angular rate data, and estimating an orientation of the movable member relative to the chassis by integrating the visual information with the inertial measurements. Drawings Features of the present invention will become apparent to those skilled in the art from the following description with reference to the accompanying drawings. Understanding that the drawings depict only typical embodiments and are not therefore to be considered to be limiting of its scope, the invention will be described with additional specificity and detail through the use of the accompanying drawings in which: FIG. 1 is a block diagram of a relative attitude measurement system for a vehicle according to one embodiment; FIG. 2 is a flow chart of a method of operation for a relative attitude measurement system according to one implementation; FIG. 3A is a flow chart of a close-coupling function that may be performed by the method of operation of FIG. 2 in an alternative implementation, and Fig. 3B is a flow chart of a loose coupling function that may be performed by the method of operation of fig. 2 in another alternative implementation. Detailed Description In the following detailed description, embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It is to be understood that other embodiments may be utilized without departing from the scope of the present invention. The following detailed description is, therefore, not to be taken in a limiting sense. A system and method for a visual Inertial Measurement Unit (IMU) based relative attitude measurement system is described herein. In general, the present technology is applicable to relative heading/attitude measurement systems in which there are at least two bodies having up to three degrees of freedom in roll, pitch, and yaw, wherein the bodies are physically coupled such that one body is within the field of view of the other body. The method employs a Visual Inertial Odometer (VIO) system for estimating pose by integrating visual information from one or more cameras with inertial measurements from one or more Inertial Measurement Units (IMUs). The present method may use a tightly coupled VIO system or a loosely coupled VIO system. In a close-coupled VIO system, visual information (features detected in the image) and inertial measurements (acceleration and angular rate) are integrated at an early stage in the state estimation process. This means that the raw data from both the camera and IMU a