CN-121999444-A - Assembly line worker fatigue monitoring method and system based on image recognition

CN121999444ACN 121999444 ACN121999444 ACN 121999444ACN-121999444-A

Abstract

The invention discloses a pipelining worker fatigue monitoring method and system based on image recognition, which relate to the technical field of image recognition and comprise the steps of recognizing a low light area, carrying out targeted enhancement on the low light area, carrying out targeted brightness enhancement on different low light areas and providing a reliable basis for subsequent image analysis. The shielding boundary of each shielding type is determined, shielding compensation is carried out on the shielding type in the shielding boundary through CBAM attention mechanisms, and the shielding position existing in the image is compensated, so that the extraction and the identification of the key points are more accurate, and more rigorous foreground areas are distinguished. The physiological fatigue event and the working fatigue event are extracted from the physiological fatigue feature sequence and the working fatigue feature sequence, the fatigue degree of the worker is output through the working post differentiation model, the face image of the worker and the working action content image are related to carry out fatigue analysis, and the fatigue degree of the worker is output by considering the working post differentiation.

Inventors

CAO JINFENG
LI JUNTING
Qiu Linhao
CHEN YAN
WANG YUXIN

Assignees

西安石油大学

Dates

Publication Date: 20260508
Application Date: 20260410

Claims (9)

1. A method for monitoring fatigue of assembly line workers based on image recognition is characterized by comprising the steps of, Acquiring a worker main body image and a worker working point position image of a worker on a production line, counting respective brightness distribution of the worker main body image and the worker working point position image, identifying a low light area, carrying out targeted enhancement on the low light area, acquiring identity information and post information of the worker, and establishing a key point list of different posts on the production line according to the identity information and the post information; Identifying the shielding types in the main body image of the worker and the working point position image of the worker, determining the shielding boundary of each shielding type, performing shielding compensation on the shielding type in the shielding boundary through a CBAM attention mechanism, and distinguishing the respective foreground areas on the main body image of the worker and the working point position image of the worker through a key point list; extracting physiological fatigue characteristics in a main body image of a worker and working fatigue characteristics in a working point position image of the worker based on a foreground region, and aligning the physiological fatigue characteristics and the working fatigue characteristics according to time stamps to respectively form a physiological fatigue characteristic sequence and a working fatigue characteristic sequence; And extracting physiological fatigue events and working fatigue events from the physiological fatigue characteristic sequence and the working fatigue characteristic sequence, and outputting the fatigue degree of workers through a working post differential model so as to monitor the fatigue condition of the workers on the assembly line.
2. The method for monitoring fatigue of an assembly line worker based on image recognition according to claim 1, wherein the statistics of respective brightness distributions of the worker main body image and the worker work point image, the recognition of the low light region, and the targeted enhancement of the low light region, comprises, Calculating brightness distribution histograms of the worker main body image and the worker working point position image, and setting a brightness threshold value and a brightness standard deviation threshold value according to the feature requirements of the worker main body image and the worker working point position image; And identifying the low light areas in the main body image of the worker and the working point image of the worker by means of the brightness threshold value and the brightness standard deviation threshold value, confirming the positions of the low light areas and the low optical path length, calculating the brightness enhancement degree according to the positions of the low light areas and the low optical path length, and improving the brightness of the low light areas through local histogram equalization and the brightness enhancement degree.
3. The method for monitoring fatigue of workers in a pipeline based on image recognition according to claim 1, wherein the step of establishing a list of key points of different posts on the pipeline based on the identity information and the post information comprises, Collecting all existing candidate points on a worker main body image and a worker working point image, and carrying out multi-aspect evaluation on each candidate point through identity information and post information, wherein the multi-aspect evaluation comprises fatigue correlation evaluation, detection stability evaluation, individual difference evaluation, calculation efficiency evaluation and fatigue sensitivity evaluation; For fatigue sensitivity evaluation, constructing a fatigue sensitivity curve in a working period, determining a median interval, a lower interval and an upper interval of the fatigue sensitivity curve, calculating respective average values and slope change average values of the median interval, the lower interval and the upper interval, and calculating a stable value of the fatigue sensitivity based on the average values and the slope change average values; Determining recognition difficulty based on fatigue correlation evaluation and detection stability evaluation, grading the fatigue correlation evaluation, the detection stability evaluation, the individual difference evaluation, the calculation efficiency evaluation and the fatigue sensitivity evaluation one by one to obtain comprehensive evaluation scores of each candidate point, ranking all candidate point positions according to the comprehensive evaluation scores, and taking a plurality of candidate point positions ranked at the front as key points on a worker main body image and a worker working point image through a region to which the recognition difficulty belongs, so as to establish a key point list of different positions on a production line.
4. The method for in-line worker fatigue monitoring based on image recognition according to claim 1, wherein the types of occlusion present in the worker main body image and the worker work point image are recognized, and an occlusion boundary of each occlusion type is determined, comprising, Counting all existing shielding types in a worker main body image and a worker working point position image, collecting corresponding shielding scene images aiming at each shielding type, marking multiple features, key points and shielding boundaries on the shielding scene images, calculating marginal contribution of each feature to recognition accuracy, giving different feature weights based on the marginal contribution, defining a critical key point through the distance between the shielding boundary and the key point, calculating the average distance and standard deviation from the critical key point to the shielding boundary, and obtaining basic expansion scale under each shielding type; extracting multiple features on a current worker main body image and a worker working point position image, fusing the multiple features, calculating the confidence coefficient of each shielding type, and determining the shielding type; Dynamically adjusting a basic expansion scale under the current shielding type according to the identity information, and determining a shielding boundary of each shielding type; the multi-features include shape features, color features, position features, and texture features.
5. The method for in-line worker fatigue monitoring based on image recognition according to claim 3, wherein the distinguishing of the respective foreground areas on the worker main body image and the worker working point image by the key point list comprises, Based on key points of the main body image of the worker on the key point list, establishing a face key region, and superposing the face key region and a shielding region on the main body image of the worker to generate a foreground region of the main body image of the worker; And establishing a working key region based on the key points of the worker working point position images on the key point list, and overlapping the working key region and the shielding region on the worker working point position images to generate a foreground region of the worker working point position images.
6. The method of pipeline worker fatigue monitoring based on image recognition according to claim 1, wherein the physiological fatigue event and the work fatigue event are extracted on the physiological fatigue feature sequence and the work fatigue feature sequence, comprising, Setting respective corresponding characteristic rules of different physiological fatigue events and working fatigue events, and extracting the satisfied physiological fatigue events and working fatigue events from the physiological fatigue characteristic sequences and the working fatigue characteristic sequences by virtue of the characteristic rules.
7. The method for pipelined worker fatigue monitoring based on image recognition according to claim 6, wherein the method further comprises, after extracting the physiological fatigue event and the work fatigue event on the physiological fatigue feature sequence and the work fatigue feature sequence, Performing time lag correlation analysis on the physiological fatigue event and the working fatigue event, determining a correlation time interval, establishing a fatigue event time axis of two types of fatigue events according to time sequence of the physiological fatigue event and the working fatigue event, and classifying the fatigue events into two types of cooperative fatigue event and independent fatigue event through the correlation time interval.
8. The method for monitoring fatigue of an assembly line worker based on image recognition as set forth in claim 7, wherein outputting the fatigue degree of the worker through the job post differentiation model comprises, Fatigue weights of physiological fatigue events and working fatigue events are distributed through working post differentiation, fatigue grades of workers are calculated based on the collaborative fatigue events and the independent fatigue events, and the fatigue degrees are described through the fatigue grades.
9. An image recognition-based assembly line worker fatigue monitoring system is characterized by comprising, The first module is used for acquiring a worker main body image and a worker working point position image of a worker on the assembly line, counting respective brightness distribution of the worker main body image and the worker working point position image, identifying a low light area, carrying out targeted enhancement on the low light area, acquiring identity information and post information of the worker, and establishing a key point list of different posts on the assembly line according to the identity information and the post information; The second module is used for identifying the shielding types in the main body image of the worker and the working point position image of the worker, determining the shielding boundary of each shielding type, carrying out shielding compensation on the shielding type in the shielding boundary through a CBAM attention mechanism, and distinguishing the respective foreground areas on the main body image of the worker and the working point position image of the worker through a key point list; The third module is used for extracting physiological fatigue characteristics in the main body image of the worker and working fatigue characteristics in the working point position image of the worker based on the foreground area, aligning the physiological fatigue characteristics with the working fatigue characteristics according to the time stamp and respectively forming a physiological fatigue characteristic sequence and a working fatigue characteristic sequence; And the fourth module is used for extracting physiological fatigue events and working fatigue events from the physiological fatigue characteristic sequence and the working fatigue characteristic sequence, and outputting the fatigue degree of workers through the working position differential model so as to monitor the fatigue condition of the workers on the assembly line.

Description

Assembly line worker fatigue monitoring method and system based on image recognition Technical Field The invention relates to the technical field of image recognition, in particular to a pipelining worker fatigue monitoring method and system based on image recognition. Background In modern manufacturing line environments, worker fatigue is a core risk factor that leads to safety accidents and reduced efficiency. The traditional monitoring method (such as manual observation or wearing equipment) is difficult to adapt to the high-dynamic and multi-interference environment of the assembly line, and has the limitations of poor real-time performance, high false alarm rate, strong invasiveness and the like. The fatigue monitoring technology based on image recognition utilizes computer vision and deep learning to realize non-contact real-time fatigue state assessment. In the prior art, fatigue monitoring of image recognition only analyzes a face image of a worker, does not consider a working action content image of the worker, does not correlate the face image of the worker with the working action content image for image analysis, causes poor accuracy and adaptability of fatigue monitoring of the worker, and cannot meet the safety requirement on a manufacturing production line. Therefore, how to improve the accuracy and adaptability of fatigue monitoring of workers is a technical problem to be solved at present. Disclosure of Invention The invention aims to solve the problems of poor accuracy and adaptability of fatigue monitoring of workers in the prior art due to analysis of facial images of workers only, and provides an image recognition-based assembly line worker fatigue monitoring method which comprises the following steps of, Acquiring a worker main body image and a worker working point position image of a worker on a production line, counting respective brightness distribution of the worker main body image and the worker working point position image, identifying a low light area, carrying out targeted enhancement on the low light area, acquiring identity information and post information of the worker, and establishing a key point list of different posts on the production line according to the identity information and the post information; Identifying the shielding types in the main body image of the worker and the working point position image of the worker, determining the shielding boundary of each shielding type, performing shielding compensation on the shielding type in the shielding boundary through a CBAM attention mechanism, and distinguishing the respective foreground areas on the main body image of the worker and the working point position image of the worker through a key point list; extracting physiological fatigue characteristics in a main body image of a worker and working fatigue characteristics in a working point position image of the worker based on a foreground region, and aligning the physiological fatigue characteristics and the working fatigue characteristics according to time stamps to respectively form a physiological fatigue characteristic sequence and a working fatigue characteristic sequence; And extracting physiological fatigue events and working fatigue events from the physiological fatigue characteristic sequence and the working fatigue characteristic sequence, and outputting the fatigue degree of workers through a working post differential model so as to monitor the fatigue condition of the workers on the assembly line. In some embodiments of the present application, the respective brightness distributions of the worker's body image and the worker's work point position image are counted, a low light region is identified, and the low light region is pertinently enhanced, including, Calculating brightness distribution histograms of the worker main body image and the worker working point position image, and setting a brightness threshold value and a brightness standard deviation threshold value according to the feature requirements of the worker main body image and the worker working point position image; And identifying the low light areas in the main body image of the worker and the working point image of the worker by means of the brightness threshold value and the brightness standard deviation threshold value, confirming the positions of the low light areas and the low optical path length, calculating the brightness enhancement degree according to the positions of the low light areas and the low optical path length, and improving the brightness of the low light areas through local histogram equalization and the brightness enhancement degree. In some embodiments of the present application, a list of key points for different posts on a pipeline is created based on identity information and post information, including, Collecting all existing candidate points on a worker main body image and a worker working point image, and carrying out multi-aspect evaluation on each candidate poin