CN-115893136-B - Elevator fault early warning method and device

CN115893136BCN 115893136 BCN115893136 BCN 115893136BCN-115893136-B

Abstract

The invention provides an elevator fault early warning method and device, and belongs to the technical field of fault early warning. The method comprises the steps of shooting an elevator door and an occupant by using a camera to obtain image data, detecting the image data to obtain the edge of the elevator door, determining the movement speed of the elevator door at 2N position points according to the edge in the K round-trip processes before the current moment, forming a speed vector V by using 2N x K movement speeds, detecting the image data, identifying the abnormal behavior of the occupant, determining an abnormal behavior vector A in a time period before the current moment according to the abnormal behavior of the occupant, fusing the abnormal behavior vector A with the speed vector V to obtain a fused vector P, inputting the fused vector P into a pre-trained fault detection model, outputting the fault type in the time period T1 after the current moment, and sending the fault type to elevator maintenance personnel. The invention can early warn the elevator faults and ensure the personal safety of passengers.

Inventors

HU XICHI
ZHANG YANG

Assignees

株式会社日立制作所

Dates

Publication Date: 20260508
Application Date: 20210929

Claims (14)

1. An elevator fault early warning method is characterized by comprising the following steps: shooting an elevator door and passengers in the elevator by using a camera to obtain image data; Detecting the image data to obtain an edge of an elevator door, determining the motion speed of the elevator door at 2N position points in the K round trip processes before the current moment according to the edge, and forming a speed vector V by using 2N x K motion speeds, wherein K and N are positive integers; Detecting the image data, identifying abnormal behaviors of an occupant, and determining an abnormal behavior vector A in a time period T before the current moment according to the abnormal behaviors of the occupant; fusing the abnormal behavior vector A and the speed vector V to obtain a fusion vector P; Inputting the fusion vector P into a pre-trained fault detection model, and outputting fault types in a time period T1 after the current moment; and sending the fault type to elevator maintenance personnel.
2. The elevator malfunction alerting method of claim 1, wherein the step of detecting the image data to obtain an edge of an elevator door comprises: acquiring depth characteristics of the image data to obtain a segmentation area of the elevator door area on the image; and extracting the edges of the connected areas to obtain an edge coordinate linked list of the elevator door in the image, wherein the edge coordinate linked list comprises coordinates of a plurality of points on the edge of the elevator door in the image.
3. The elevator malfunction alerting method of claim 2, wherein determining the speed of movement of the elevator door comprises: Determining a reference point on the edge of the elevator door, and determining the movement speed of the reference point according to the position of the reference point in continuous multi-frame images; and determining the movement speed of the elevator door according to the movement speed of the reference point.
4. The elevator malfunction alerting method according to claim 1, wherein the step of detecting the image data and identifying abnormal behavior of the occupant comprises: detecting the image data, and identifying joint points of an occupant; and inputting coordinates of the joint points in the continuous multi-frame images into the behavior recognition model, and outputting abnormal behaviors of the passengers.
5. The elevator malfunction alerting method according to claim 1, wherein the step of determining the abnormal behavior vector a in the T2 period before the current time according to the abnormal behavior of the occupant comprises: establishing an abnormal behavior initial vector with the length of M, wherein M is the type number of the abnormal behaviors, and each element of the abnormal behavior initial vector represents one abnormal behavior; And counting the number of each abnormal behavior of the occupant in a time period T2 before the current moment, and updating the values of corresponding elements in the initial vector of the abnormal behavior according to the number of each abnormal behavior of the occupant to obtain an abnormal behavior vector A.
6. The elevator malfunction alerting method according to claim 1, wherein the step of fusing the abnormal behavior vector a with the velocity vector V to obtain a fused vector P includes any one of the following: directly splicing the abnormal behavior vector A and the speed vector V to obtain a fusion vector P; The abnormal behavior vector A is split with the speed vector V after the dimension of the abnormal behavior vector A is reduced, and a fusion vector P is obtained; and the speed vector V is split with the abnormal behavior vector A after the dimension of the speed vector V is reduced, and a fusion vector P is obtained.
7. The elevator failure warning method according to any one of claims 1 to 6, characterized in that before the step of inputting the fusion vector P into a pre-trained failure detection model, the method further comprises the step of training to obtain the failure detection model, the step of training to obtain the failure detection model comprising: Establishing a fault detection initial model; Detecting historical image data shot by a camera to obtain an edge of an elevator door, determining the movement speed of the elevator door at 2N position points in the K round trip processes of a historical time period according to the edge, and forming a historical speed vector V1 by using 2N x K movement speeds; Detecting historical image data shot by a camera, identifying abnormal behaviors of an occupant, and determining a historical abnormal behavior vector A1 in the historical time period according to the abnormal behaviors of the occupant; Fusing the historical abnormal behavior vector A1 with the historical speed vector V1 to obtain a historical fusion vector P1; And acquiring the historical fault type of the elevator in the historical time period, and training the fault detection initial model by utilizing the historical fault type and the historical fusion vector P1 to obtain the fault detection model.
8. An elevator fault warning device, characterized by comprising: The shooting module is used for shooting the elevator door and passengers in the elevator by using the camera to obtain image data; The first detection module is used for detecting the image data to obtain the edge of the elevator door, determining the motion speed of the elevator door at 2N position points in the K round trip processes before the current moment according to the edge, and forming a speed vector V by using 2N x K motion speeds, wherein K and N are positive integers; The second detection module is used for detecting the image data, identifying abnormal behaviors of the passengers, and determining an abnormal behavior vector A in a time period T before the current moment according to the abnormal behaviors of the passengers; The fusion module is used for fusing the abnormal behavior vector A with the speed vector V to obtain a fusion vector P; The prediction module is used for inputting the fusion vector P into a pre-trained fault detection model and outputting fault types in a T1 time period after the current moment; And the fault management module is used for sending the fault type to elevator maintenance personnel.
9. The elevator fault warning device of claim 8, wherein the first detection module comprises: The segmentation area acquisition unit is used for acquiring the depth characteristics of the image data to obtain a segmentation area of the elevator door area on the image; and the segmentation area processing unit is used for extracting the edges of the connected areas to obtain an edge coordinate linked list of the elevator door in the image, wherein the edge coordinate linked list comprises coordinates of a plurality of points on the edge of the elevator door in the image.
10. The elevator fault warning device of claim 9, wherein the first detection module comprises: The first calculation unit is used for determining a reference point on the edge of the elevator door, and determining the movement speed of the reference point according to the position of the reference point in the continuous multi-frame images; And the second calculation unit is used for determining the movement speed of the elevator door according to the movement speed of the reference point.
11. The elevator fault warning device of claim 8, wherein the second detection module comprises: the first processing unit is used for detecting the image data and identifying joint points of an occupant; And the second processing unit is used for inputting the coordinates of the joint points in the continuous multi-frame images into the behavior recognition model and outputting the abnormal behavior of the passengers.
12. The elevator fault warning device of claim 8, wherein the second detection module comprises: The system comprises a building unit, a judging unit and a judging unit, wherein the building unit is used for building an abnormal behavior initial vector with the length of M, M is the type number of the abnormal behaviors, and each element of the abnormal behavior initial vector represents one abnormal behavior; and the updating unit is used for counting the quantity of each abnormal behavior of the occupant in the time period T2 before the current moment, and updating the values of corresponding elements in the abnormal behavior initial vector according to the quantity of each abnormal behavior of the occupant to obtain an abnormal behavior vector A.
13. The elevator failure warning device of claim 8, wherein the fusion module is specifically configured to perform any one of: directly splicing the abnormal behavior vector A and the speed vector V to obtain a fusion vector P; The abnormal behavior vector A is split with the speed vector V after the dimension of the abnormal behavior vector A is reduced, and a fusion vector P is obtained; and the speed vector V is split with the abnormal behavior vector A after the dimension of the speed vector V is reduced, and a fusion vector P is obtained.
14. The elevator fault warning device of any one of claims 8-13, further comprising a training module for training to obtain the fault detection model, the training module being specifically configured to: Establishing a fault detection initial model; Detecting historical image data shot by a camera to obtain an edge of an elevator door, determining the movement speed of the elevator door at 2N position points in the K round trip processes of a historical time period according to the edge, and forming a historical speed vector V1 by using 2N x K movement speeds; Detecting historical image data shot by a camera, identifying abnormal behaviors of an occupant, and determining a historical abnormal behavior vector A1 in the historical time period according to the abnormal behaviors of the occupant; Fusing the historical abnormal behavior vector A1 with the historical speed vector V1 to obtain a historical fusion vector P1; And acquiring the historical fault type of the elevator in the historical time period, and training the fault detection initial model by utilizing the historical fault type and the historical fusion vector P1 to obtain the fault detection model.

Description

Elevator fault early warning method and device Technical Field The invention relates to the technical field of fault early warning, in particular to an elevator fault early warning method and device. Background With the acceleration of the urban process, medium-high-rise buildings are more and more, the use of the elevator becomes wide for facilitating the travel of people, meanwhile, the operation safety of the elevator is increasingly important, and the safe and reliable operation of the elevator is directly related to the personal safety and life safety of passengers. Disclosure of Invention The invention aims to solve the technical problem of providing an elevator fault early warning method and device, which can early warn elevator faults and ensure personal safety of passengers. In order to solve the technical problems, the embodiment of the invention provides the following technical scheme: In one aspect, an elevator fault early warning method is provided, including: shooting an elevator door and passengers in the elevator by using a camera to obtain image data; Detecting the image data to obtain an edge of an elevator door, determining the motion speed of the elevator door at 2N position points in the K round trip processes before the current moment according to the edge, and forming a speed vector V by using 2N x K motion speeds, wherein K and N are positive integers; Detecting the image data, identifying abnormal behaviors of an occupant, and determining an abnormal behavior vector A in a time period T before the current moment according to the abnormal behaviors of the occupant; fusing the abnormal behavior vector A and the speed vector V to obtain a fusion vector P; Inputting the fusion vector P into a pre-trained fault detection model, and outputting fault types in a time period T1 after the current moment; and sending the fault type to elevator maintenance personnel. In an alternative embodiment of the present invention, the step of detecting the image data to obtain an edge of an elevator door includes: acquiring depth characteristics of the image data to obtain a segmentation area of the elevator door area on the image; and extracting the edges of the connected areas to obtain an edge coordinate linked list of the elevator door in the image, wherein the edge coordinate linked list comprises coordinates of a plurality of points on the edge of the elevator door in the image. In an alternative embodiment of the invention, determining the speed of movement of the elevator door comprises: Determining a reference point on the edge of the elevator door, and determining the movement speed of the reference point according to the position of the reference point in continuous multi-frame images; and determining the movement speed of the elevator door according to the movement speed of the reference point. In an alternative embodiment of the present invention, the step of detecting the image data and identifying the abnormal behavior of the occupant includes: detecting the image data, and identifying joint points of an occupant; and inputting coordinates of the joint points in the continuous multi-frame images into the behavior recognition model, and outputting abnormal behaviors of the passengers. In an alternative embodiment of the present invention, the step of determining the abnormal behavior vector a in the period T2 before the current time according to the abnormal behavior of the occupant includes: establishing an abnormal behavior initial vector with the length of M, wherein M is the type number of the abnormal behaviors, and each element of the abnormal behavior initial vector represents one abnormal behavior; And counting the number of each abnormal behavior of the occupant in a time period T2 before the current moment, and updating the values of corresponding elements in the initial vector of the abnormal behavior according to the number of each abnormal behavior of the occupant to obtain an abnormal behavior vector A. In an alternative embodiment of the present invention, the step of fusing the abnormal behavior vector a with the velocity vector V to obtain a fused vector P includes any one of the following: directly splicing the abnormal behavior vector A and the speed vector V to obtain a fusion vector P; The abnormal behavior vector A is split with the speed vector V after the dimension of the abnormal behavior vector A is reduced, and a fusion vector P is obtained; and the speed vector V is split with the abnormal behavior vector A after the dimension of the speed vector V is reduced, and a fusion vector P is obtained. In an alternative embodiment of the present invention, before the step of inputting the fusion vector P into a pre-trained fault detection model, the method further includes a step of training to obtain the fault detection model, where the step of training to obtain the fault detection model includes: Establishing a fault detection initial model; Detecting historical image