Search

EP-4097628-B1 - EXTRACTING INFORMATION ABOUT PEOPLE FROM SENSOR SIGNALS

EP4097628B1EP 4097628 B1EP4097628 B1EP 4097628B1EP-4097628-B1

Inventors

  • MORRISON, CECILY PEREGRINE BORGATTI
  • GRAYSON, Martin Philip
  • DUNPHY, Anja
  • CUTRELL, EDWARD BRYAN

Dates

Publication Date
20260506
Application Date
20201214

Claims (13)

  1. An information extraction system (10) for extracting information about a person comprising: an input configured to receive, from a sensor system (2), incoming sensor signals for monitoring people (102) within a field of view of the sensor system (2); and one or more processors (7) configured to process the incoming sensor signals, and in response to detecting a person (102) located within a notification region (210) of the field of view of the sensor system (2), cause an output device (3) associated with the sensor system (2) to output a notification to the detected person (102); wherein the one or more processors (7) are configured to continue processing the incoming sensor signals in order to monitor behaviour patterns of the person (102) after the notification has been outputted, and determine from his behaviour patterns whether he is currently in a consenting or non-consenting state; and wherein the one or more processors (7) are configured to implement: (i) an extraction function (21) that attempts to extract identity information about the person (102) irrespective of his determined state, and (ii) a sharing function (22) that determines whether or not to share an extracted piece of identity information about the person (102) with a receiving entity in accordance with his determined state, wherein the piece of identity information about the person (102) extracted when he is determined to be in the non-consenting state is not shared unless and until it is subsequently determined that the person (102) is now in the consenting state; wherein the one or more processors (7) are configured to determine the person (102) to be in the consenting state when located in a consenting region of the notification region (210), and the non-consenting state when located in a non-consenting region of the notification region (210).
  2. The information extraction system (10) according to claim 1, wherein the one or more processors (7) are configured to determine the person (102) to be in the consenting state when the person performs a predetermined consent gesture when located in the notification region (210), and the non-consenting state if the person has not performed the consent gesture in the notification region (210).
  3. The information extraction system (10) according to claim 1, wherein the one or more processors (7) are configured to determine the person (102) to be in the consenting state when the person (102) remains in the notification region (210) for a predefined period of time, and the person (210) is in the non-consenting state prior to the expiry of said predefined period of time.
  4. The information extraction system (10) according to claim 3, wherein the person (102) remains in the non-consenting state after the expiry of the predefined period of time if the person (102) performs a predefined non-consent gesture before expiry of the predefined period of time when located in the notification region (210).
  5. The information extraction system (10) according to any preceding claim, wherein the extraction function comprises a facial recognition process, a pose tracking process, and/or an object tracking process.
  6. The information extraction system (10) according to any preceding claim, wherein the receiving entity is a user (100), and the information is shared with the user (100) via a user interface (4).
  7. The information extraction system (10) according to claim 6, wherein the user interface (4), the sensor system (2), and the output device (3) are embodied in a wearable device.
  8. The information extraction system (10) according to any of claim 1 to claim 5, wherein the receiving entity is an external computer system or storage device, and the information is shared therewith with a data interface of the information extraction system.
  9. The information extraction system (10) according to any preceding claim, wherein the notification is rendered to the person (102) as a visual or audio notification, the one or more processors (7) configured to update the notification in response to changes in the behaviour pattern of the person (102).
  10. The information extraction system (10) according to any preceding claim, wherein, a confidence value (36) is calculated for each piece of extracted information, the piece of extracted information only being shared with the receiving entity if the confidence value (36) is greater than a predefined confidence threshold.
  11. A computer implemented method of extracting information about a person (102), the method comprising: Receiving, from a sensor system (2), incoming sensor signals for monitoring people (102) within a field of view of the sensor system (2); and processing the incoming sensor signals, and in response to detecting a person (102) located within a notification region (210) of the field of view of the sensor system (2), cause an output device (3) associated with the sensor system (2) to output a notification to the detected person (102); wherein the processing of the incoming sensor signals continues in order to monitor behaviour patterns of the person (102) after the notification has been outputted, and determine from his behaviour patterns whether he is currently in a consenting or non-consenting state; and wherein: (i) an extraction function (21) attempts to extract identity information about the person (102) irrespective of his determined state, and (ii) a sharing function (22) determines whether or not to share an extracted piece of identity information about the person (102) with a receiving entity in accordance with his determined state, wherein the piece of identity information about the person extracted when he is determined to be in the non-consenting state is not shared unless and until it is subsequently determined that the person (102) is now in the consenting state; wherein the person (1020 is determined to be in the consenting state when located in a consenting region of the notification region (210), and the non-consenting state when located in a non-consenting region of the notification region (210).
  12. The computer implemented method according to claim 11, wherein the person (102) is determined to be in the consenting state when he performs a predetermined consent gesture when located in the notification region (210), and the non-consenting state if he has not performed the consent gesture in the notification region (210).
  13. A computer program (6), stored on a storage medium (5), for programming a computer system (10) to implement the functionality or steps of any preceding claim.

Description

Technical Field of Invention The present disclosure relates to information extraction about people from sensor signals, including, for example, facial recognition applied to video images. Background Personally identifiable information can be extracted from image data through the use of recognition systems such as facial recognition systems. These systems compare distinguishing features such as facial features detected in the image data to faces stored in a database in order to identify the person captured in the image. Facial recognition systems generally employ machine learning techniques to train a facial recognition model to both detect faces in an image and to identify such detected faces by analysing patterns based on, for example, the person's facial textures and shape. Facial recognition systems are becoming more widespread. One such application is the use in camera systems. One use is that of identifying a person such as a friend or family member appearing in a video or image, recording in a family home for example. Another system is a security system that uses the person's face as a "key" to unlock a function of a device or access protected information. In US2016/080642A1, JUNG JAEYEON ET AL., a system for recording video of persons is disclosed, where the recording starts once a consenting gesture is detected. Recording is suspended until said gesture is detected. Summary This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Nor is the claimed subject matter limited to implementations that solve any or all of the disadvantages noted herein. As facial recognition and other forms of automated information extraction become increasingly prevalent and sophisticated, challenges around consent are amplified. On the one hand, such systems need to be able to operate effectively, and relatively seamlessly, without placing an undue burden on users and without unnecessarily disrupting the desired functions of the system. On the other hand, it is important that people maintain a level of control over their personal information, including facial recognition and other results extracted using modern information extraction technologies and algorithms. The present disclosure provides a "dynamic consent" mechanism which balances the need for information extraction in systems to operate effectively with the need to provide people with an acceptable level of control over their personal information. The dynamic consent mechanism allows people to set their consent preferences dynamically, without having to rely on predetermined consent preferences. According to a first aspect of the present disclosure, there is provided an information extraction system comprising: an input configured to receive, from a sensor system, incoming sensor signals for monitoring people within a field of view of the sensor system; and one or more processors. The processor(s) are configured to process the incoming sensor signals, and in response to detecting a person located within a notification region of the field of view, cause an output device associated with the sensor system to output a notification to the detected person. The one or more processors are configured to continue processing the incoming sensor signals in order to monitor behaviour patterns of the person after the notification has been outputted, and determine from his behaviour patterns whether he is currently in a consenting or non-consenting state. The one or more processors are configured to implement: (i) an extraction function that attempts to extract information about the person irrespective of his determined state, and (ii) a sharing function that determines whether or not to share an extracted piece of information about the person with a receiving entity in accordance with his determined state, wherein a piece of information about the person extracted when he is determined to be in the non-consenting state is not shared unless and until it is subsequently determined that the person is now in the consenting state, wherein the one or more processors are configured to determine the person to be in the consenting state when located in a consenting region of the notification region, and the non-consenting state when located in a non-consenting region of the notification region. Brief Description of the Drawings To assist understanding of the present disclosure and to show how embodiments of the present invention may be put into effect, reference is made by way of example to the accompanying drawings, in which: Figure 1 is a schematic block diagram of an information extraction system at the hardware level;Figure 2 is a schematic functional block diagram of the information extraction system;Figure 3 is