CN-115565140-B - Fault detection method, device and storage medium

CN115565140BCN 115565140 BCN115565140 BCN 115565140BCN-115565140-B

Abstract

The embodiment of the application provides a fault detection method, equipment and a storage medium, wherein the method comprises the steps of acquiring three-dimensional point cloud data and an optical image of a vehicle to be detected, identifying the optical image, acquiring a first position of a target characteristic point on a wheel of the vehicle to be detected under a first coordinate system, determining a second position of a homonymous point of the target characteristic point in the three-dimensional point cloud data under a second coordinate system according to the first position and a conversion relation, determining a side line of a tail plate of a car of the vehicle to be detected and a side line of a side plate of the car in the three-dimensional point cloud data according to the second position, and generating a fault reminding message according to the side line of the tail plate and the side line of the side plate. The fault detection equipment provided by the embodiment of the application can timely send out the fault reminding message so as to avoid loss, the mode does not need to modify the vehicle, is not limited by the performance of a sensor, has higher stability and is safer than manual command.

Inventors

ZHOU BO
DUAN LIAN
MIAO RUI
WU XINYU
MO SHAOFENG

Assignees

深圳市海清视讯科技有限公司

Dates

Publication Date: 20260508
Application Date: 20220824

Claims (9)

1. A fault detection method, comprising: acquiring three-dimensional point cloud data and an optical image of a vehicle to be detected; performing identification processing on the optical image to obtain a first position of a target feature point on a wheel of the vehicle to be detected under a first coordinate system; Determining a second position of the same name point of the target characteristic point in the three-dimensional point cloud data under a second coordinate system according to the first position and a conversion relation, wherein the conversion relation is a coordinate conversion relation between the three-dimensional point cloud data and the optical image; determining a perpendicular line from the homonymy point to the ground according to the second position; Based on the vertical line, a plurality of groups of first pixel points in the horizontal direction are obtained from the three-dimensional point cloud data, wherein the pixel points in different groups are on the same horizontal plane, and the horizontal direction is vertical to the vertical line; for each group of first pixel points, calculating the distance between each first pixel point and the position of the camera, and determining the first pixel point with the shortest distance as a second pixel point; determining a first angle between the imaging surface and a side plate of the car according to the second pixel points and the positions of the optical axes; Determining the edge of the tail plate in the three-dimensional point cloud data according to the first angle, wherein the edge of the tail plate is used for being connected with the edge of the side plate when the tail plate is closed; generating a fault reminding message according to the edge of the tail plate and the edge of the side plate, wherein the fault reminding message comprises the following components: detecting a second angle between the edge of the tailboard and the ground; if the second angle is larger than a first threshold value, a first fault reminder is sent out, wherein the first fault reminder is used for indicating that the tail board is in a locking state; detecting the area ratio of point clouds in the area between the edge of the tail plate and the edge of the side plate; and if the area occupation ratio is larger than a second threshold value, sending out a second fault prompt so that the vehicle to be detected moves forwards according to the second fault prompt, wherein the second fault prompt is used for indicating that goods are blocked in the bucket lifting process.
2. The method of claim 1, wherein determining the edge of the tailboard in the three-dimensional point cloud data according to the first angle comprises: Screening a plurality of groups of third pixel points from a plurality of groups of first pixel points according to the first angle and a plurality of second pixel points, wherein each group of third pixel points are on the same straight line, and the angle between the straight line where each group of third pixel points is positioned and an imaging surface is the first angle; and determining the edge line of the side plate and the edge line of the tail plate according to the plurality of groups of third pixel points.
3. The method according to claim 2, wherein the method further comprises: and determining the bucket lifting angle of the lift car according to a plurality of groups of third pixel points and a plurality of second pixel points.
4. A method according to claim 3, wherein said generating a fault alert message from the edge of the tailboard and the edge of the side board comprises: determining whether the lift car enters a bucket-lifting state according to the bucket-lifting angle; if yes, generating a fault reminding message according to the edge of the tail plate and the edge of the side plate.
5. The method of any one of claims 1-4, wherein the tailgate is configured to pivot open in a car-scooped condition about a roof line of the tailgate.
6. The method of any one of claims 1-4, wherein the target feature point is an upper or lower vertex of the wheel, the method further comprising: determining a first straight line where an upper vertex and a lower vertex of the wheel are located according to the second position; determining a third angle between the first straight line and the side line of the side plate according to the first straight line and the side line of the side plate; Determining a fourth angle between the first straight line and the ground according to the first straight line; and if the third angle is larger than a third threshold value or the fourth angle is larger than a fourth threshold value, a third fault prompt is sent out, and the third fault prompt is used for indicating that the vehicle to be detected is inclined.
7. A fault detection apparatus, characterized by comprising: The acquisition module is used for acquiring three-dimensional point cloud data and an optical image of the vehicle to be detected; the processing module is used for carrying out identification processing on the optical image to obtain a first position of a target characteristic point on a wheel of the vehicle to be detected under a first coordinate system; The processing module is also used for determining a second position of the same name point of the target characteristic point in the three-dimensional point cloud data under a second coordinate system according to the first position and a conversion relation, wherein the conversion relation is a coordinate conversion relation between the three-dimensional point cloud data and the optical image; the processing module is also used for determining the perpendicular line from the homonymous point to the ground according to the second position; Based on the vertical line, a plurality of groups of first pixel points in the horizontal direction are obtained from the three-dimensional point cloud data, wherein the pixel points in different groups are on the same horizontal plane, and the horizontal direction is vertical to the vertical line; for each group of first pixel points, calculating the distance between each first pixel point and the position of the camera, and determining the first pixel point with the shortest distance as a second pixel point; determining a first angle between the imaging surface and a side plate of the car according to the second pixel points and the positions of the optical axes; Determining the edge of the tail plate in the three-dimensional point cloud data according to the first angle, wherein the edge of the tail plate is used for being connected with the edge of the side plate when the tail plate is closed; the processing module is also used for generating a fault reminding message according to the edge of the tail plate and the edge of the side plate; The processing module is specifically used for detecting a second angle between the edge of the tail plate and the ground; if the second angle is larger than a first threshold value, a first fault reminder is sent out, wherein the first fault reminder is used for indicating that the tail board is in a locking state; detecting the area ratio of point clouds in the area between the edge of the tail plate and the edge of the side plate; and if the area occupation ratio is larger than a second threshold value, sending out a second fault prompt so that the vehicle to be detected moves forwards according to the second fault prompt, wherein the second fault prompt is used for indicating that goods are blocked in the bucket lifting process.
8. A fault detection device is characterized by comprising an optical camera, a depth camera, at least one processor and a memory; The optical camera and the depth camera are connected with the processor The optical camera is used for shooting a vehicle to be detected, obtaining an optical image and sending the optical image to the processor; The depth camera is used for shooting the vehicle to be detected, obtaining three-dimensional point cloud data and sending the three-dimensional point cloud data to the processor; The memory stores computer-executable instructions; the at least one processor executing computer-executable instructions stored in the memory causes the at least one processor to perform the fault detection method of any one of claims 1 to 6.
9. A computer readable storage medium having stored therein computer executable instructions which, when executed by a processor, implement the fault detection method of any of claims 1 to 6.

Description

Fault detection method, device and storage medium Technical Field The embodiment of the application relates to the technical field of image processing, in particular to a fault detection method, equipment and a storage medium. Background The dumper belongs to engineering building type vehicles, is often combined with an excavator and a loader, and is mainly used for transporting earthwork, sand and stone and bulk materials, and is used for engineering construction. The car body can automatically rise under the action of the hydraulic cylinder, and when the weight of tens of tons rises, the gravity center of the car can be increased, and the car is destroyed and the car is dead once an accident occurs. In the related art, a sensor or the like in the vehicle may be used to monitor the process of the lift of the dump truck. However, in the process of implementing the present application, the inventor finds that at least the following problems exist in the prior art that the sensor is generally not suitable for such a mechanical device with a large load, the durability is affected by a plurality of factors such as severe environment, pressure, etc., and once the sensor is damaged, the dump truck cannot work normally. Disclosure of Invention The embodiment of the application provides a fault detection method, equipment and a storage medium, which are used for realizing supervision and fault early warning of a bucket lifting process of a dump truck and ensuring the stability of fault detection. In a first aspect, an embodiment of the present application provides a fault detection method, including: acquiring three-dimensional point cloud data and an optical image of a vehicle to be detected; performing identification processing on the optical image to obtain a first position of a target feature point on a wheel of the vehicle to be detected under a first coordinate system; Determining a second position of the same name point of the target characteristic point in the three-dimensional point cloud data under a second coordinate system according to the first position and a conversion relation, wherein the conversion relation is a coordinate conversion relation between the three-dimensional point cloud data and the optical image; Determining the edge of a tail plate of the car of the vehicle to be detected and the edge of a side plate of the car in the three-dimensional point cloud data according to the second position, wherein the edge of the tail plate is used for being connected with the edge of the side plate when the tail plate is closed; and generating a fault reminding message according to the edge of the tail plate and the edge of the side plate. In one possible design, the determining, according to the second position, a boundary line of a tail board of a car of the vehicle to be detected in the three-dimensional point cloud data includes: determining a perpendicular line from the homonymy point to the ground according to the second position; Based on the vertical line, a plurality of groups of first pixel points in the horizontal direction are obtained from the three-dimensional point cloud data, wherein the pixel points in different groups are on the same horizontal plane, and the horizontal direction is vertical to the vertical line; for each group of first pixel points, calculating the distance between each first pixel point and the position of the camera, and determining the first pixel point with the shortest distance as a second pixel point; determining a first angle between an imaging surface and a side plate of the car according to the second pixel points and the positions of the optical axes; And determining the edge of the tail plate in the three-dimensional point cloud data according to the first angle. In one possible design, the determining, according to the first angle, a side line of the tailboard in the three-dimensional point cloud data includes: Screening a plurality of groups of third pixel points from a plurality of groups of first pixel points according to the first angle and a plurality of second pixel points, wherein each group of third pixel points are on the same straight line, and the angle between the straight line where each group of third pixel points is positioned and an imaging surface is the first angle; and determining the edge line of the side plate and the edge line of the tail plate according to the plurality of groups of third pixel points. In one possible design, the method further comprises: and determining the bucket lifting angle of the lift car according to a plurality of groups of third pixel points and a plurality of second pixel points. In one possible design, the generating a fault alert message according to the edge of the tailboard and the edge of the side board includes: determining whether the lift car enters a bucket-lifting state according to the bucket-lifting angle; if yes, generating a fault reminding message according to the edge of the tail plate and