Search

CN-117121061-B - Computer-implemented method for generating attention areas

CN117121061BCN 117121061 BCN117121061 BCN 117121061BCN-117121061-B

Abstract

A computer-implemented method for generating a geometric attention area (1) for at least one person in a three-dimensional space (2), the computer-implemented method comprising using a single image capturing unit (3), a data processing unit (4), a database (5) and a display unit (6), and a computer-readable storage medium and apparatus for performing such a method.

Inventors

  • Michael Hedermoze
  • THOMAS LANG

Assignees

  • 传感3D有限责任公司

Dates

Publication Date
20260505
Application Date
20220426
Priority Date
20210505

Claims (11)

  1. 1. A computer-implemented method for generating a plurality of geometric attention areas (1) for at least one person in a three-dimensional space (2), the method using a single image capturing unit (3), a data processing unit (4), a database (5) and a display unit (6), comprising the steps of: a. Outputting, by the display unit (6), a request to the person to fixate at least one point in the space (2) corresponding to the vertex of the attention area (1) to be created, B. Capturing a first number N of photographs (7) of the space (2) accommodating the person by the image capturing unit (3), and extracting a first number N of three-dimensional gaze direction vectors BV 1 -BV N of the person by the data processing unit (4), where N is greater than or equal to 3, C. Outputting a request to the person by the display unit (6) to change his position in the space (2) and to gaze back at the vertex of the attention area (1), D. Capturing a second number N of photographs (7 ') of the space (2) accommodating the person by the image capturing unit (3), and extracting a second number N of three-dimensional gaze direction vectors BV 1 ′-BV N ' of the person by the data processing unit (4), E. Superimposing, by the data processing unit (4), the line of gaze direction vector BV 1 -BV N with the line of gaze direction vector BV 1 ′-BV N ' to determine N three-dimensional intersection points P 1 -P N , F. A polygon with vertices P 1 -P N in the space (2) is stored as an attention area (1) in the database (5).
  2. 2. The method of claim 1, wherein the number of vertices N is 4.
  3. 3. Method according to claim 1 or 2, characterized in that the space (2) is the interior of a vehicle (8).
  4. 4. The method according to claim 1, characterized in that the method is repeated to create multiple attention areas (1, 1',1 ") of a single person, n=4.
  5. 5. The method according to claim 1 or 2, characterized in that the method is repeated to create a plurality of attention areas (1, 1',1 ") of a different plurality of people.
  6. 6. A method according to claim 1 or 2, characterized in that, A. after generating and storing one or more attention areas (1, 1 '') of at least one person, the image capturing unit (3) continuously detects a gaze direction vector of the person and transmits the gaze direction vector to the data processing unit (4), and B. The data processing unit (4) checks whether the detected gaze direction vector falls into one of the stored attention areas (1, 1',1 ").
  7. 7. The method of claim 6, wherein the step of providing the first layer comprises, A. The data processing unit (4) calculates the degree of attention of at least one person by determining the frequency with which the detected gaze direction vector per unit time falls within one of the stored attention areas (1, 1 ''), and B. when the degree of attention falls below a predetermined threshold, the display unit (6) outputs a warning message.
  8. 8. A computer readable storage medium comprising a data processing unit (4) for performing the method according to any one of claims 1 to 7.
  9. 9. Device for generating a plurality of geometrical attention areas (1) for at least one person in a three-dimensional space (2), comprising a single image capturing unit (3), a data processing unit (4), a database (5) and a display unit (6), characterized in that the device is designed to perform the method according to any one of claims 1 to 7.
  10. 10. The arrangement according to claim 9, characterized in that the space (2) is the interior of a vehicle (8) and the image capturing unit (3) is arranged in front of the driver's seat or in front of the passenger seat in the direction of travel of the vehicle.
  11. 11. The device according to claim 10, characterized in that the image capturing unit (3) is arranged in a central position above the vehicle windscreen.

Description

Computer-implemented method for generating attention areas Technical Field The present invention relates to a computer-implemented method for generating a geometric attention area in a three-dimensional space. Background It is known from the prior art to use image-based methods to determine the attention of a person by analyzing the gaze direction of the person. This may be achieved by extracting the head direction or eye position from, for example, a photograph or video of the person. By classifying the gaze direction as a predetermined region, the level of attention may be calculated. Such attention modeling is mainly used in the automotive field, but is also used in other man-machine applications. In vehicles, this attention modeling may be used to determine whether people are paying attention to traffic or distraction. In a store, this attention modeling can be used to identify which products draw more or less attention. In robotics, the gaze direction of a person may be used to control the machine. In order to classify the gaze direction of a person, the field of view of the person needs to be divided into attention areas. Since these attention areas must be valid in three-dimensional space, they are typically defined by polygons in three-dimensional space, i.e., they include three or more points defined by the x, y, and z coordinates of the crossing area. After dividing the field of view of the person into such attention areas, each gaze direction of the person may be assigned to a defined attention area. However, one problem is that the image capturing unit that is expected to analyze the gaze direction of the person must be directed toward the person, and thus cannot simultaneously capture the person's field of view. In order to also capture the field of view of a person, two or more image capturing units are required. If the gaze direction of a person is extracted from the image of the image capturing unit, a problem also arises, in that the same gaze direction of different analyzed persons does not necessarily mean that these persons are also looking at the same area in space. For example, drivers and passengers in a vehicle may have the same gaze direction, but due to their different seat positions, the areas of their interest are quite different. Thus, in the prior art, additional sensors are often used, which have an attention area in the field of view and are connected to an image capturing unit capturing the person such that a direct geometrical relationship is established between the person's gaze direction and the attention area. For example, these additional sensors are mounted beside or behind the person so that an attention area can be defined for each person. Disclosure of Invention It is an object of the present invention to improve these methods known in the art, in particular to be able to create a geometrical attention area without using additional sensors. These and other objects of the invention are solved by a computer-implemented method and apparatus according to the independent claims. The computer-implemented method for generating a geometric attention area for at least one person in a three-dimensional space according to the present invention uses a single image capturing unit, a data processing unit, a database and a display unit arranged in space. The three-dimensional space may be the interior of a vehicle, however, the method of the present invention is not limited to vehicles. The image capturing unit, the data processing unit and the database may preferably be arranged entirely in the vehicle. However, it may also be provided that the data processing unit and the database are arranged in the vehicle and communicate via an interface (e.g. a wireless connection) with an external server (e.g. an internet server), on which there may be a database with previously stored and/or continuously supplementary reference data for processing the captured data. The image capturing unit may be a photo camera or a video camera designed to capture two-dimensional or three-dimensional photos or videos. In particular, the image capturing unit may be a TOF (time of flight) camera or the like. The use of TOF cameras helps to robustly detect people and extract gaze directions. However, conventional 2D cameras in combination with previously stored 3D models of a person or a person's face are also able to robustly extract gaze directions from the respective 2D images. The image capturing unit may preferably be arranged in the vehicle in such a way that the entire interior, at least all passenger seats, but at least the driver's seat and the passenger seats are visible in the image area, if possible. The data processing unit may be designed as a microcontroller or microcomputer and comprises a Central Processing Unit (CPU), volatile semiconductor memory (RAM), non-volatile semiconductor memory (ROM, SSD hard disk), magnetic memory (hard disk) and/or optical memory (CD-ROM), and an interface unit