CN-122024283-A - Disaster environment intelligent human body posture recognition method based on visual information

Abstract

The invention provides a disaster environment intelligent human body posture identification method based on visual information, and belongs to the field of computer vision. The method comprises the steps of collecting visible light images and infrared images of a human body, fusing the infrared images and the visible light images to obtain the number of the human body and key nodes of the human body, classifying detection results to distinguish shielding conditions from non-shielding conditions, constructing and training a human body posture classification model under the non-shielding and shielding conditions, and outputting information of the number of people, pose information and confidence level information through the human body posture classification model. Aiming at the problems of human body posture and human number identification of trapped people caused by smoke in the underground buried disaster rescue field, the invention combines infrared and visible light information to construct a neural network model, adopts an algorithm optimization strategy, and can realize real-time identification of human body posture in a complex environment.

Inventors

• QU DANYANG
• ZHANG ZHENYU
• ZHANG YUE
• WEI GUO
• WU KAI
• CAO SHILONG
• QIN MINGFENG
• WEI BAOGUO
• JIA HAONAN
• MA LIYE
• DU DENGHUI
• HU YANAN
• Ren Kehan

Assignees

• 中国电子科技集团公司第五十四研究所
• 内蒙古自治区军民融合发展研究中心

Dates

Publication Date

20260512

Application Date

20260130

Claims (6)

1. 1. The disaster environment intelligent human body posture recognition method based on visual information is characterized by comprising the following steps of: Step 1, an infrared camera and a low-illumination camera are used for collecting visible light images and infrared images containing human bodies, and correction processing is carried out on the visible light images; Step 2, fusing the infrared image and the corrected visible light image, and detecting the fused image through a human body key node detection model to obtain the number of human bodies and human body key nodes; step 3, carrying out two classification on the detection result in the step 2 by adopting a key node geometric constraint condition, and distinguishing the shielding condition from the non-shielding condition; step 4, respectively constructing and training human body posture classification models under non-shielding and shielding conditions, wherein the human body posture classification model under the non-shielding condition is VGGNet neural network, and the human body posture classification model under the shielding condition is RSGNet model; And 5, inputting the detection result with the shielding condition obtained in the step 3 into a human body posture classification model under the shielding condition, inputting the detection result with the non-shielding condition obtained in the step 3 into a human body posture classification model under the non-shielding condition, and outputting corresponding number of people information, pose information and confidence information.
2. 2. The disaster environment intelligent human body posture recognition method based on visual information as set forth in claim 1, wherein the specific manner of step 1 is as follows: Step 101, acquiring video streams by using an infrared camera and a low-illumination camera, and performing format conversion processing on the acquired images by using an OpenCV and a Piclow technical stack; Step 102, converting RGB colors of the visible light image to obtain a luminance component Y: wherein, x and y represent pixel points in the image, R, G, B respectively represent red, green and blue colors; Step 103, performing nonlinear normalization processing on the luminance component Y: Wherein max represents taking the maximum value; Step 104, constructing a bright image I b and a dark image I d according to the normalized luminance component: Step 105, selecting Gamma correction parameters with maximized image entropy H b Correction was performed on the light map I b and the dark map I d , respectively: Wherein argmax represents a parameter value obtained by maximizing the expression, and γ b 、γ d is a Gamma correction parameter for light and dark; where u (x, y) is the fusion weight of shading correction for the x, y points.
3. 3. The disaster environment intelligent human body posture recognition method based on visual information according to claim 2, wherein the specific mode of step 2 is as follows: Step 201, constructing an infrared image feature extraction network and a visible light image feature extraction network, and obtaining a feature map of an infrared image through the infrared image feature extraction network Obtaining a feature map of a visible light image through a visible light image feature extraction network Generating a fused image according to : Wherein H is the weight calculated by the feature map of the infrared image and the visible light image; 202, carrying out human body key node detection training on a YOLOv-Pose model through a COCO data set, and reasoning a fusion image by utilizing the trained model to obtain personnel detection, 17 key point results of a human body and confidence degrees of all key points; And 203, filtering non-personnel targets, and further removing detection results in invalid areas by combining the interest areas ROI to obtain final number of people, key nodes and confidence results.
4. 4. The disaster environment intelligent human body posture identifying method based on visual information according to claim 3, wherein the specific mode of step 3 is as follows: Step 301, judging whether key points of shoulder, hip, knee and ankle parts of a human body are missing in an image, if so, judging that the key points are blocked, otherwise, further judging whether the confidence degree of the key points of the shoulder, hip, knee and ankle parts is less than 0.5, and if so, judging that the key points are blocked; Step 302, for the picture which is not judged to be blocked in step 301, calculating the center points and the relative distances of the shoulders, buttocks, knees and ankles of the human body in the image, calculating the body segment direction of the human body in the image, judging whether the same image is blocked among a plurality of human bodies or not by taking the relation among the aspect ratio, the joint angle threshold, the body verticality and the relative distances of the center points and the body segment direction as constraint conditions, and further distinguishing the picture with the blocking.
5. 5. The disaster environment intelligent human body posture recognition method based on visual information according to claim 1, wherein in step 4, training data of a human body posture classification model under a non-shielding condition is from OpenPose open source data sets; The training dataset of the human body posture classification model under the shielding condition is formed by mixing MSCOCO datasets and newly added key shielding part datasets, wherein the newly added key shielding part datasets comprise image data of left arm shielding, right arm shielding, left leg shielding, right leg shielding, left foot shielding, right foot shielding and head shielding, and each image data marks key points and posture information of a human body in the figure.
6. 6. The method for recognizing the intelligent human body posture in the disaster environment based on the visual information according to claim 1, wherein in the step 5, the posture information is one posture of lying, squatting and standing.

Description

Disaster environment intelligent human body posture recognition method based on visual information Technical Field The invention belongs to the field of computer vision, and particularly relates to a disaster environment intelligent human body posture recognition method based on visual information. Background In high-risk tasks such as emergency rescue, personnel rescue is important, and rescue personnel need to formulate and evaluate a rescue scheme according to various factors such as topography conditions, buried personnel conditions and the like. Therefore, reliable information acquisition and pose evaluation are carried out on buried personnel, so that the rapid and accurate judgment of the number of people suffering from the disaster and the pose position of the human body is realized, and the rescue progress and the rescue result are affected. However, extreme environmental problems such as collapse, smoke dust, irregular obstacles and the like existing in disaster (earthquake, fire and the like) sites often lead to the defects of detectability, limited visual range and increased rescue difficulty of disaster rescue personnel. Therefore, in order to realize efficient personnel rescue after disaster, research on an efficient and accurate intelligent human body detection and identification technology facing to a complex environment is needed. With the development of robots and AI, disaster relief personnel can realize post-disaster search and rescue in a larger range and faster speed by means of unmanned means and artificial intelligence technology. However, the integration of intelligent technology still faces problems such as difficulty in capturing human body images caused by severe illumination conditions and complex environments in sites of disasters (earthquakes, fires and the like). Based on the above, researchers mostly adopt a multi-sensor fusion mode to collect on-site human body images in human body recognition. The method can achieve the effects of eliminating noise, enhancing image information and filling the problem of single-mode information deletion. However, how to effectively fuse the multi-mode information of different sensors and solve the problem of image mismatch still remains a key to influence the input precision. In addition, the human gestures in the disaster scene often relate to multiple people and multiple gesture situations, including shielding among human bodies, mutual interference, similarity discrimination of multiple human body parts, shielding degree between human bodies and environment, and the like, and the complicated situations also increase difficulty in gesture recognition. Therefore, the recognition and judgment of the human body pose require tasks such as segmentation and association of human body examples in addition to general joint detection and joint connection. The traditional pose recognition method only carries out unified centralized consideration on crowded conditions and partial human body shielding conditions in the pictures, does not distinguish non-shielding from shielding conditions for pose recognition, and usually uses training data sets for non-shielding human body poses. Therefore, the risk of misjudgment is increased, and the difficulty of network training is also increased. Disclosure of Invention Aiming at overcoming the defects of the prior art and solving the problems of human body gesture and people number recognition of trapped people caused by smoke in the underground buried disaster rescue field, the invention provides a disaster environment intelligent human body gesture recognition method based on visual information. According to the method, infrared and visible light mode information is fused, and the human key joint identification based on infrared and visible light fusion and the intelligent human posture classification based on deep learning are adopted, so that the real-time classification and identification of the number of rescue persons and the human posture in a complex environment (weak light, shielding and the like) are realized, and the actual application requirements are met. In order to achieve the above purpose, the technical scheme adopted by the invention is as follows: a disaster environment intelligent human body posture recognition method based on visual information comprises the following steps: Step 1, an infrared camera and a low-illumination camera are used for collecting visible light images and infrared images containing human bodies, and correction processing is carried out on the visible light images; Step 2, fusing the infrared image and the corrected visible light image, and detecting the fused image through a human body key node detection model to obtain the number of human bodies and human body key nodes; step 3, carrying out two classification on the detection result in the step 2 by adopting a key node geometric constraint condition, and distinguishing the shielding condition from the non-shieldin