Search

CN-121999513-A - User state identification method, device, equipment and medium

CN121999513ACN 121999513 ACN121999513 ACN 121999513ACN-121999513-A

Abstract

The application relates to a user state identification method, a device, equipment and a storage medium. Acquiring image data of a user in real time, acquiring environment data of the user and distance information between the user and the sensor through a sensor, generating corrected target position information and target contour information of the user based on the image data and the distance information, acquiring object detection results around the user according to the target position information and the target contour information of the user, acquiring emotion detection results of the user according to the target position information, the target contour information and the object detection results of the user, and identifying state information of the user according to the target position information, the target contour information, the object detection results, the emotion detection results and the environment data of the user. The application can accurately identify the state of the user, comprehensively considers the interaction and emotion states of the user and surrounding objects, and remarkably improves the accuracy and reliability of dangerous behavior and abnormal emotion identification.

Inventors

  • Request for anonymity

Assignees

  • 深圳市深科创产业发展有限公司

Dates

Publication Date
20260508
Application Date
20251211

Claims (10)

  1. 1. A method for identifying a user state, the method comprising: Collecting image data of a user in real time, and collecting environment data of the user and distance information of the user and the sensor through the sensor; generating corrected target position information and target contour information of the user based on the image data and the distance information; Obtaining object detection results around the user according to the target position information and the target contour information of the user; obtaining emotion detection results of the user according to the target position information, the target contour information and the object detection results of the user; And identifying the state information of the user according to the target position information, the target outline information, the object detection result, the emotion detection result and the environment data of the user.
  2. 2. The user state recognition method of claim 1, wherein generating corrected target position information and target contour information of the user based on the image data and the distance information comprises: performing human body detection on the image data based on visual recognition, and extracting initial contour information and initial position information of a user; Geometrically correcting the initial position information based on the distance information to generate corrected target position information; and correcting the initial contour information based on the data acquired by the infrared sensor to obtain corrected target contour information.
  3. 3. The user state recognition method of claim 2, wherein the human body detection of the image data based on visual recognition, extracting initial contour information and initial position information of the user, comprises: performing preprocessing operation on the original image data to obtain a preprocessed image; Adjusting the resolution of the preprocessed image to a target resolution, and initializing an image feature map; dividing candidate areas corresponding to users based on the initial image feature images; and inputting the candidate region into a visual recognition model, and extracting initial contour information and initial position information of the user by utilizing a local-global mixed attention mechanism.
  4. 4. The user state recognition method of claim 2, wherein geometrically correcting the initial position information based on the distance information, generating corrected target position information, comprises: Performing time synchronization calibration on the distance information so that the time stamp error between the distance information and the image data is smaller than a preset threshold value; converting the calibrated distance information into three-dimensional space distance data corresponding to the pixel coordinates of the image data through a coordinate mapping relation between a pre-calibrated sensor and a camera; performing space matching on the body key point coordinates in the initial position information and the three-dimensional space distance data to determine a position offset; Dynamically adjusting a weight coefficient of the geometric correction based on the acceleration sensor data; And generating corrected target position information according to the position offset and the weight coefficient.
  5. 5. The user state recognition method of claim 1, wherein the obtaining the object detection result around the user according to the target position information and the target profile information of the user comprises: Determining an interested area image around the user based on the target position information and the target contour information of the user; Inputting the region of interest image into an object detection model, and identifying object types and object position coordinates; determining the relative distance between the user and the object according to the object position coordinates and the target position information of the user; And taking the relative distance between the object type and the relative distance as an object detection result around the user.
  6. 6. The user state recognition method of claim 1, wherein the obtaining the emotion detection result of the user based on the target position information of the user, the target contour information, and the object detection result comprises: Positioning a face area of a user based on the target contour information, and extracting expression feature points of the face area; performing interaction behavior analysis on the object detection result, and identifying the interaction type of the user and the object; And inputting the expression characteristic points and the interaction type into an emotion detection model to obtain an emotion detection result of the user.
  7. 7. The user state recognition method of claim 1, wherein the recognizing the state information of the user based on the target position information, the target profile information, the object detection result, the emotion detection result, and the environment data of the user comprises: Fusing according to the target position information, the target contour information, the object detection result, the emotion detection result and the environment data of the user to obtain fused data; And inputting the fusion data into a multi-modal fusion model for analysis to obtain a state detection result of the user.
  8. 8. A user state identification device, the device comprising: the acquisition unit is used for acquiring image data of a user in real time and acquiring environment data of the user and distance information of the user and the sensor through the sensor; the correction unit is used for generating corrected target position information and target contour information of the user based on the image data and the distance information; The object detection unit is used for obtaining object detection results around the user according to the target position information and the target contour information of the user; The emotion detection unit is used for obtaining emotion detection results of the user according to the target position information, the target outline information and the object detection results of the user; And the identification unit is used for identifying the state information of the user according to the target position information, the target outline information, the object detection result, the emotion detection result and the environment data of the user.
  9. 9. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus; A memory for storing a computer program; a processor for implementing the user state identification method according to any one of claims 1 to 7 when executing a program stored on a memory.
  10. 10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the user state identification method according to any of claims 1 to 7.

Description

User state identification method, device, equipment and medium Technical Field The present application relates to the field of state identification technologies, and in particular, to a method, an apparatus, a device, and a storage medium for identifying a user state. Background At present, the existing user state identification technology often depends on single data, and the single data is easy to be interfered by various factors, for example, the quality of image data may be damaged due to light change and background disorder, and the environmental data may be inaccurate due to the accuracy limitation of a sensor. In addition, the influence of surrounding objects of the user on the user state is not fully considered in the prior art, so that the accuracy of user state identification is low. Therefore, how to improve the accuracy of user state identification has become a technical problem to be solved by those skilled in the art. It should be noted that the information disclosed in the above background section is only for enhancing understanding of the background of the present disclosure and thus may include information that does not constitute prior art known to those of ordinary skill in the art. Disclosure of Invention In view of the foregoing, the present application provides a user state identification method, apparatus, device and storage medium, which aim to solve the above technical problems. In a first aspect, the present application provides a method for identifying a user state, the method comprising: Collecting image data of a user in real time, and collecting environment data of the user and distance information of the user and the sensor through the sensor; generating corrected target position information and target contour information of the user based on the image data and the distance information; Obtaining object detection results around the user according to the target position information and the target contour information of the user; obtaining emotion detection results of the user according to the target position information, the target contour information and the object detection results of the user; And identifying the state information of the user according to the target position information, the target outline information, the object detection result, the emotion detection result and the environment data of the user. In a second aspect, the present application provides a user state recognition apparatus, comprising: the acquisition unit is used for acquiring image data of a user in real time and acquiring environment data of the user and distance information of the user and the sensor through the sensor; the correction unit is used for generating corrected target position information and target contour information of the user based on the image data and the distance information; The object detection unit is used for obtaining object detection results around the user according to the target position information and the target contour information of the user; The emotion detection unit is used for obtaining emotion detection results of the user according to the target position information, the target outline information and the object detection results of the user; And the identification unit is used for identifying the state information of the user according to the target position information, the target outline information, the object detection result, the emotion detection result and the environment data of the user. In a third aspect, the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus; A memory for storing a computer program; and a processor, configured to implement the steps of the user state identification method according to any one of the embodiments of the first aspect when executing the program stored in the memory. In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored which, when being executed by a processor, implements the steps of the user state identification method according to any one of the embodiments of the first aspect. Compared with the prior art, the technical scheme provided by the embodiment of the application has the following advantages: The application acquires image data of a user in real time, acquires environment data of the user and distance information between the user and the sensor through a sensor, generates corrected target position information and target contour information of the user based on the image data and the distance information, obtains object detection results around the user according to the target position information and the target contour information of the user, obtains emotion detection results of the user according to the target position information, the t