CN-122024317-A - Driver abnormal behavior detection method, electronic device and readable storage medium

CN122024317ACN 122024317 ACN122024317 ACN 122024317ACN-122024317-A

Abstract

The application relates to the technical field of artificial intelligence and traffic safety monitoring, in particular to a driver abnormal behavior detection method, electronic equipment and a readable storage medium, aiming at solving the technical problem of how to realize real-time, accurate and efficient driver abnormal behavior detection. Therefore, the method and the device acquire the cross-modal characteristics based on the video acquisition result and the preset abnormal detection prompt words aiming at the driver, perform model reasoning of multi-modal detection based on the preset multi-modal detection model, acquire model reasoning state evaluation results according to the cross-modal characteristics, acquire the abnormal behavior detection results of the driver according to the model reasoning state evaluation results and the cross-modal characteristics, and can effectively detect various abnormal behaviors such as smoking, calling, yawning, closing eyes, leaving handles with one hand and the like of the driver, thereby improving the instantaneity, accuracy and high efficiency of the abnormal behavior detection.

Inventors

Du Ruoshuang
ZHU YAJIE

Assignees

浙江智谱新篇科技有限公司

Dates

Publication Date: 20260512
Application Date: 20260128

Claims (11)

1. A method for detecting abnormal behavior of a driver, the method comprising: acquiring a cross-modal characteristic based on a video acquisition result aiming at a driver and a preset abnormal detection prompt word; based on a preset multi-mode detection model, carrying out model reasoning on the multi-mode detection model according to the cross-mode characteristics to obtain a model reasoning state evaluation result of the multi-mode detection model; And acquiring an abnormal behavior detection result of the driver according to the model reasoning state evaluation result and the cross-modal characteristic.
2. The driver abnormal behavior detection method according to claim 1, wherein the model inference state evaluation result includes an uncertainty score and a splice hidden layer state of a model inference process of the multi-modal detection model.
3. The driver's abnormal behavior detection method according to claim 2, wherein, The step of obtaining the abnormal behavior detection result according to the model reasoning state evaluation result and the cross-modal characteristic comprises the following steps: Determining whether an external knowledge base needs to be searched or not according to uncertainty scores of a model reasoning process and a spliced hidden layer state, wherein the external knowledge base comprises a plurality of labeling samples of abnormal behaviors of a driver; if yes, acquiring the abnormal behavior detection result based on the retrieval result of the external knowledge base and the cross-modal characteristic; If not, acquiring the abnormal behavior detection result according to the cross-modal characteristic.
4. The method for detecting abnormal behavior of driver according to claim 3, wherein, The determining whether the external knowledge base needs to be retrieved according to the uncertainty score of the model reasoning process and the spliced hidden layer state comprises the following steps: Determining the retrieval probability of the external knowledge base retrieval based on a preset decision classification model according to the uncertainty score of the model reasoning process and the state of the spliced hidden layer; If the retrieval probability is greater than or equal to a preset probability threshold, determining that an external knowledge base needs to be retrieved; if the retrieval probability is less than the probability threshold, determining that no external knowledge base is required to be retrieved.
5. The method for detecting abnormal behavior of driver according to claim 3, wherein, The obtaining the abnormal behavior detection result based on the retrieval result of the external knowledge base and the cross-modal feature includes: according to the visual characteristics of the video acquisition result, searching the external knowledge base to obtain a search result of the external knowledge base; And carrying out model reasoning of the multi-mode detection model according to the retrieval result and the cross-mode characteristic to obtain the abnormal behavior detection result.
6. The method for detecting abnormal behavior of a driver according to claim 3, wherein the obtaining the abnormal behavior detection result according to the cross-modal feature includes: and carrying out model reasoning of the multi-mode detection model according to the cross-mode characteristics to obtain the abnormal behavior detection result.
7. The method for detecting abnormal behavior of driver according to claim 5, wherein, The searching of the external knowledge base according to the visual characteristics obtained by the video acquisition result to obtain the search result of the external knowledge base comprises the following steps: comparing the visual characteristics with visual characteristics of labeling samples in the external knowledge base to obtain similarity of the labeling samples; and taking the labeling sample with the similarity larger than a preset similarity threshold as the retrieval result.
8. The method for detecting abnormal behavior of driver according to claim 1, wherein, Based on a video acquisition result aiming at a driver and a preset abnormal detection prompt word, acquiring the cross-modal characteristic comprises the following steps: according to the video acquisition result, extracting video characteristics to acquire visual characteristics of the driver; Extracting text features according to the abnormality detection prompt words to obtain text features; And performing cross-modal feature fusion according to the visual features and the text features to obtain the cross-modal features.
9. The method for detecting abnormal behavior of a driver according to any one of claims 1 to 8, wherein the multimodal detection model is a model obtained by model-compressing CogVLM multi-modal large models using knowledge distillation technology.
10. An electronic device, comprising: At least one processor; And a memory communicatively coupled to the at least one processor; wherein the memory has stored therein a computer program which, when executed by the at least one processor, implements the driver abnormal behavior detection method of any one of claims 1 to 9.
11. A computer readable storage medium having stored therein a plurality of program codes, characterized in that the program codes are adapted to be loaded and executed by a processor to perform the driver abnormal behavior detection method according to any one of claims 1 to 9.

Description

Driver abnormal behavior detection method, electronic device and readable storage medium Technical Field The application relates to the technical field of artificial intelligence and traffic safety monitoring, in particular to a driver abnormal behavior detection method, electronic equipment and a readable storage medium. Background With the continuous development of artificial intelligence technology, the application of the technology in the field of traffic safety monitoring is more and more widespread. The method is used for monitoring abnormal behaviors of bus drivers, and combines an artificial intelligence technology. However, in the prior art, the following bottlenecks exist in the process of monitoring abnormal behaviors of a driver: model complexity and iteration capability that after the rule is determined, the existing deep learning model is difficult to iterate a new version after new requirements are added later, and the model complexity can be gradually overlapped. 2. The traditional detection method lacks the understanding capability of the driving scene context, and cannot distinguish normal driving operation from dangerous behavior, such as the distinction of emergency avoidance and dangerous driving. 3. The generalization capability is poor, namely, after the existing system is trained in a specific environment, when the existing system faces different vehicle types, driving habits and environment changes, the recognition accuracy is obviously reduced, and frequent retraining is needed. Accordingly, there is a need in the art for a new driver abnormal behavior detection scheme to solve the above-described problems. Disclosure of Invention The present application has been made to overcome the above drawbacks, and aims to solve or at least partially solve the technical problem of how to implement real-time, accurate and efficient detection of abnormal behavior of a driver. In a first aspect, there is provided a driver abnormal behavior detection method, the method comprising: acquiring a cross-modal characteristic based on a video acquisition result aiming at a driver and a preset abnormal detection prompt word; based on a preset multi-mode detection model, carrying out model reasoning on the multi-mode detection model according to the cross-mode characteristics to obtain a model reasoning state evaluation result of the multi-mode detection model; And acquiring an abnormal behavior detection result of the driver according to the model reasoning state evaluation result and the cross-modal characteristic. In one technical scheme of the driver abnormal behavior detection method, the model reasoning state evaluation result comprises uncertainty scores and spliced hidden layer states of a model reasoning process of the multi-mode detection model. In one technical scheme of the driver abnormal behavior detection method, the obtaining the abnormal behavior detection result according to the model reasoning state evaluation result and the cross-modal feature includes: Determining whether an external knowledge base needs to be searched or not according to uncertainty scores of a model reasoning process and a spliced hidden layer state, wherein the external knowledge base comprises a plurality of labeling samples of abnormal behaviors of a driver; if yes, acquiring the abnormal behavior detection result based on the retrieval result of the external knowledge base and the cross-modal characteristic; If not, acquiring the abnormal behavior detection result according to the cross-modal characteristic. In one technical scheme of the driver abnormal behavior detection method, the determining whether the external knowledge base needs to be searched according to the uncertainty score of the model reasoning process and the spliced hidden layer state comprises the following steps: Determining the retrieval probability of the external knowledge base retrieval based on a preset decision classification model according to the uncertainty score of the model reasoning process and the state of the spliced hidden layer; If the retrieval probability is greater than or equal to a preset probability threshold, determining that an external knowledge base needs to be retrieved; if the retrieval probability is less than the probability threshold, determining that no external knowledge base is required to be retrieved. In one technical scheme of the driver abnormal behavior detection method, the obtaining the abnormal behavior detection result based on the search result of the external knowledge base and the cross-modal feature includes: according to the visual characteristics of the video acquisition result, searching the external knowledge base to obtain a search result of the external knowledge base; And carrying out model reasoning of the multi-mode detection model according to the retrieval result and the cross-mode characteristic to obtain the abnormal behavior detection result. In one technical scheme of the driver abn