Search

RU-2861737-C2 - SYSTEM FOR SOCIO-PSYCHOLOGICAL MONITORING OF PERSON

RU2861737C2RU 2861737 C2RU2861737 C2RU 2861737C2RU-2861737-C2

Abstract

FIELD: engineering. SUBSTANCE: invention relates to the field of determining a person's psychological state. System comprises an image capture unit configured to capture video images; a face recognition unit configured to isolate and recognise faces in a video stream; an emotional reaction determination unit configured to determine a person's emotional reactions and their intensity based on the captured video stream; an analysis unit configured to determine a person's psycho-emotional state based on an analysis of the number of manifestations of said person's emotional reactions and their intensity over a predetermined period of time, determined based on the captured video stream. EFFECT: providing psychological monitoring of a person with high accuracy and speed. 15 cl, 3 dwg, 2 tbl

Inventors

  • Zarechnaya Olga Yurevna
  • Molyanov Pavel Valerevich
  • Sichinava Zurabi Irodievich
  • Brazhkin Sergej Aleksandrovich

Dates

Publication Date
20260508
Application Date
20200313

Claims (20)

  1. 1. A system of psychological monitoring of a person, including:
  2. - an image capture unit capable of capturing video images;
  3. - a face recognition unit capable of identifying and recognizing faces in a video stream;
  4. - a block for determining emotional reactions, designed with the ability to determine a person’s emotional reactions and their intensity based on the captured video stream;
  5. - an analysis unit designed to determine the psycho-emotional state of a person based on an analysis of the number of manifestations of the said emotional reactions of a person and their intensity over a predetermined period of time, determined on the basis of the captured video stream,
  6. wherein said analysis includes determining the intensity of the emotional reaction and comparing the obtained intensity with a threshold value, wherein if the intensity is less than said threshold value, then the emotional reaction is defined as a neutral emotional reaction, and if the intensity is higher than said threshold value, then the emotional reaction is defined as a reliable emotional reaction,
  7. wherein the analysis unit is configured to divide the video stream into predetermined time intervals and determine for each time interval the emotional reactions recorded the greatest number of times, and when two or more emotional reactions occur the same number of times, the data for this time interval is excluded from the analysis, and the psycho-emotional state of the person in this time interval is not determined.
  8. 2. A system for psychological monitoring of a person according to paragraph 1, in which the face recognition unit is configured to recognize faces by comparing faces selected in the video stream with reference photographs loaded into the database.
  9. 3. A system for psychological monitoring of a person according to paragraph 2, in which said database is a face descriptor database (FDB), and the face recognition unit is configured to convert images of a person’s face to represent them in the form of numerical face descriptors and to compare said numerical face descriptors with the descriptors stored in the FDB.
  10. 4. A human psychological monitoring system according to paragraph 1, additionally containing a reporting unit configured to receive data from the analysis unit and generate reports on the dynamics and causes of changes in a person’s emotional reactions, their intensity, and the general psycho-emotional state of a person over certain time intervals.
  11. 5. The human psychological monitoring system according to paragraph 1, additionally containing a notification unit configured to receive data from the analysis unit and notify the operator of the psychological monitoring system about the psycho-emotional state of the person being monitored.
  12. 6. A human psychological monitoring system according to any one of paragraphs 1-5, in which at least one of the face recognition unit, the emotional reaction determination unit and the analysis unit is implemented on the basis of a self-learning artificial neural network (ANN) or a convolutional neural network (CNN).
  13. 7. A human psychological monitoring system according to paragraph 6, in which all three of the face recognition unit, the emotional reaction determination unit, and the analysis unit are implemented on the basis of neural networks.
  14. 8. A human psychological monitoring system according to paragraph 1, in which the analysis unit is designed with the ability to analyze the emotional reactions of a person associated with an event localized on an object and in time, and to determine said event as the cause of these reactions.
  15. 9. A human psychological monitoring system according to paragraph 1, in which the face recognition unit and the emotional reaction determination unit are implemented as a local video stream pre-processing server.
  16. 10. A human psychological monitoring system according to paragraph 1, in which the analysis unit is designed with the ability to determine the psycho-emotional state of a person by the minute by dividing the video stream into minute time intervals and determining for each resulting minute time interval the emotional reactions recorded the greatest number of times.
  17. 11. A human psychological monitoring system according to paragraph 10, in which the analysis unit is configured to determine the psycho-emotional state of a person by the hour by determining, for an hourly time interval, the emotional reactions recorded the greatest number of times, based on the said specific emotional reactions by minute time intervals included in the said hourly time interval.
  18. 12. A human psychological monitoring system according to paragraph 11, in which the analysis unit is designed with the ability to determine the psycho-emotional state of a person by day by determining, for a daily time interval, the emotional reactions recorded the greatest number of times, based on the said specific emotional reactions for hourly time intervals included in the said daily time interval.
  19. 13. A human psychological monitoring system according to paragraph 1, in which the analysis unit is designed with the ability to track the dynamics of a person’s psycho-emotional state over time.
  20. 14. A method for psychological monitoring of a person, carried out by means of a system for psychological monitoring of a person according to any of paragraphs 1-13, comprising the steps of:

Description

AREA OF TECHNOLOGY The invention relates to the assessment of a person's psychological state, in particular to a system and method for recognizing a person's emotions and their intensity based on a video image of a person's face and assessing a person's psychological state based on the recognized emotions. LEVEL OF TECHNOLOGY Currently, various methods for recognizing human emotions exist. Advances in computing technology, including artificial intelligence, neural networks, and other technologies, make it possible to quickly and accurately process detected human emotions to assess their psychological state. This can be used to identify destructive tendencies (aggression, suicidal tendencies, depression, etc.), potential conflict situations within a team, and so on, with the goal of providing timely psychological and other assistance to prevent and/or resolve these issues. For example, a known emotion detection system (patent RU 42905 U1, published December 20, 2004) is capable of determining an individual's emotional state based on the analysis of their voice data. This system is primarily used to determine an individual's honesty or dishonesty. However, this system has limited functionality, as it can detect a small number of emotions and has low accuracy in identifying a person's emotions due to its analysis of voice data alone. Furthermore, a system and method for video-computer diagnostics of trainees' emotional states and the development of solutions for rationally dividing training groups are known (patent RU 2598050 C1, published September 20, 2016). This method involves capturing images of trainees' faces, recognizing the facial expression parameters of each trainee, classifying them according to J. Russell's two-dimensional circular model of emotions, and then gradually increasing the complexity of the material being studied and dividing the initial set of trainees into subgroups based on their emotional state level, depending on the complexity of the material being studied. However, this solution is aimed only at determining the impact of specific training material on a person during the learning process. Furthermore, this solution uses no more than one or two features to determine a particular type of emotional state, resulting in low accuracy in determining emotional state. A prior art system for contactless emotion detection (document WO 2018192246 A1, published October 25, 2018) measures skin tone intensity (blood flow based on heart rate) in captured images. Based on skin tone intensity, an inference is made about the intensity of a person's emotions. However, by relying solely on skin tone intensity, this system has low accuracy in emotion detection. Additionally, prior art discloses a solution disclosing an electronic device and method for managing emotional states (US Patent Application No. 2012/120219 A1, published May 17, 2012), which determines a user's emotions based on images captured by a camera and implements a relaxation method based on the identified emotions. However, this solution can only recognize three emotions, and the relaxation method consists of selecting relaxing music based on the emotion detected by the device. All of the listed solutions are based on the analysis of situational emotional reactions and at the same time function as decision-making systems, which, from a psychological point of view, raises questions, since emotional reactions are not a psychological constant and are entirely dependent on current internal thought processes, which may not be related to the context of the events taking place. Thus, there is currently a need to create a system and method not only for recognizing human emotions, but for regularly analyzing them and determining a person’s psycho-emotional state with high accuracy, reliability, and sufficient speed. DISCLOSURE OF THE INVENTION The present invention is directed to solving at least some of the above problems. In accordance with one aspect of the invention, a system for psychological monitoring of a person is proposed, comprising: - an image capture unit capable of capturing video images; - a face recognition unit capable of identifying and recognizing faces in a video stream; - a block for determining emotional reactions, designed with the ability to determine a person’s emotional reactions and their intensity based on the captured video stream; - an analysis unit designed to analyze the aforementioned specific emotional reactions of a person and their intensity over a predetermined period of time in order to determine the dynamics of his psycho-emotional state. According to one embodiment of the system, the face recognition unit is configured to recognize faces by comparing faces selected in the video stream with reference photographs loaded into the database. According to another embodiment of the system, the said database is a face descriptor database (FDB), and the face recognition unit is configured to convert images of a pe