CA-3159848-C - SYSTEM AND METHOD FOR DISPLAYING VIDEO DATA IN A TARGET ENVIRONMENT

Abstract

A system for automatically displaying output information on a display surface based on activities in a target environment includes an image capturing device for capturing data of the target environment in real-time, a processing unit for processing the data to determine a location for displaying the output information, and determine a content of the output information, and a video output unit for displaying the output information on the display surface. The video output unit is configured to rotate in a 360° manner, and wherein an angle of rotation of the video output unit is controlled by the processing unit. The image capturing device further captures the output information displayed on the display surface in real time, and provides feedback data to the processing unit in real-time, so as to enable the processing unit to control a focus, a direction, and the output information displayed on the video output unit.

Inventors

• Joe Allen

Assignees

• EVERSEEN LIMITED

Dates

Publication Date

20260505

Application Date

20201027

Priority Date

20191220

Claims (17)

1. {YB:01144826.DOCX } CLAIMS: 1. A system for automatically displaying output information on a display surface based on a location and activity of one or more entities in a target environment, the system comprising: an image capturing device configured to capture image and video data pertaining to the target environment using a fish eye view for generating a circular image frame of the target environment; a processing unit configured to: receive the circular input image frame representative of the fish eye view of the target environment, from the image capturing device; generate an image frame having a flattened representation of the target environment from the circular input image frame; and generate a grid view image of the image frame having the flattened representation, wherein the grid view image includes a live grid reference of the target environment on the image frame having the flattened representation, and wherein the live grid reference is used by the processing unit to determine information regarding the location and the activity of the one or more entities in the target environment; and a video output unit configured to display an output information on the display surface based on the information determined from the live grid reference, wherein the video output unit is configured to rotate in a 360° manner, and wherein an angle of rotation of the video output unit is controlled by the processing unit, wherein the image capturing device is further configured to capture the output information displayed on the display surface in real time, and provide feedback data to the processing unit in real-time, so as to enable the processing unit to control a focus, a direction, and a content of the information displayed on the display surface by the video output unit based on the feedback data.
2. 2. The system of claim 1 further comprising an audio recording device configured to record audio data of the target environment and transmit the recorded audio data to the processing unit, and wherein the audio recording device is configured to record audio data of the output information as feedback data, and provide the feedback data to the processing unit for enabling the processing unit to control, in real time, the focus, the direction, and the content {YB:01144826.DOCX } of the information displayed on the display surface by the video output unit based on the feedback data of the audio recording device.
3. 3. The system of claim 1, wherein the processing unit includes an Artificial Intelligence (AI) platform that is configured to direct visual information to one or more predefined locations in the target environment.
4. 4. The system of claim 1, wherein the processing unit includes a graphical processing unit (GPU) for processing video/image data.
5. 5. The system of claim 1, wherein the output information comprises pre-defined video and audio data including at least one of: alarms, notifications, advertisements, instructions, and training videos.
6. 6. The system of claim 1, wherein the video output unit comprises a projector, and the display surface comprises at least one of: a white projection screen, a grey projection screen, and a white wall for displaying a projected image.
7. 7. The system of claim 1, wherein the processing unit is configured to generate and display the output information based on the grid view image.
8. 8. The system of claim 1, further comprising a sensor unit configured to detect events as one or more changes in the target environment, wherein the sensor unit includes at least one of: a radar, an x-ray device, a sensor to detect the occurrence of a natural disaster.
9. 9. A method for automatically displaying an output information on a display surface based on a location and activity of one or more entities in a target environment, the method comprising: capturing image and video data from the target environment using a fish eye view of an image capturing device for generating a circular image frame of the target environment; receiving the circular input image frame representative of the fish eye view of the target environment from the image capture device; {YB:01144826.DOCX } generating an image frame having a flattened representation of the target environment from the circular input image frame; generating a grid view image of the image frame having the flattened representation, wherein the grid view image includes a live grid reference of the target environment on the image frame having the flattened representation, and wherein the live grid reference is used to determine information regarding a location and an activity of the one or more entities in the target environment; displaying an output information on the display surface based on the information determined from the live grid reference by a video output unit, wherein the video output unit is configured to rotate in a 360° manner; controlling an angle of rotation of the video output unit based on the captured image and video data; and capturing the information displayed on the display surface in real time, and generating feedback data to control a focus, a direction, and a content of the information displayed on the display surface by the video output unit based on the feedback data.
10. 10. The method of claim 9, further comprising: recording audio data of the target environment and transmitting the recorded audio data; and recording audio data of the output information as feedback data, and providing the feedback data for controlling, in real time, the focus, the direction, and the content of the information displayed on the display surface by the video output unit based on the feedback data.
11. 11. The method of claim 9, further comprising directing visual information to one or more predefined locations in the target environment using an AI platform.
12. 12. The method of claim 9, further comprising processing the video/image data using a graphical processing unit (GPU).
13. 13. The method of claim 9, wherein the output information comprises pre-defined video and audio data including at least one of: alarms, notifications, advertisements, instructions, and training videos. {YB:01144826.DOCX }
14. 14. The method of claim 9, wherein the video output unit comprises a projector, and the display surface comprises at least one of: a white projection screen, a grey projection screen, and a white wall for displaying a projected image.
15. 15. The method of claim 9, further comprising generating and displaying the output information based on the grid view image.
16. 16. The method of claim 9, further comprising detecting events as one or more changes in the target environment using a sensor selected from a group consisting of: a radar, an x-ray device, a sensor to detect the occurrence of a natural disaster.
17. 17. A computer programmable product for automatically displaying output information on a display surface based on one or more activities in a target environment, the computer programmable product comprising a set of instructions, the set of instructions when executed by a processor causes the processor to: capture image and video data from the target environment using a fish eye view of an image capturing device for generating a circular image frame of the target environment; receive the circular input image frame representative of the fish eye view of the target environment from the image capture device; generating an image frame having a flattened representation of the target environment from the circular input image frame; generate a grid view image of the image frame having the flattened representation, wherein the grid view image includes a live grid reference of the target environment on the image frame having the flattened representation, and wherein the live grid reference is used to determine information regarding a location and an activity of the one or more entities in the target environment; display an output information on the display surface based on the information determined from the live grid reference by a video output unit, wherein the video output unit is configured to rotate in a 360° manner; control an angle of rotation of the video output unit based on the captured image and video data; and {YB:01144826.DOCX } capture the information displayed on the display surface in real time, and generate feedback data to control a focus, a direction, and a content of the information displayed on the display surface by the video output unit based on the feedback data.

Description

CA 03159848 2022· 5- 27 1 SYSTEM AND METHOD FOR DISPLAYING VIDEO DATA IN A TARGET ENVIRONMENT TECHNICAL FIELD [0001] The present disclosure relates generally to a system and method for displaying video data in a target environment. and more paiticulary to displaying video data based on a dos.ed loop Artificial lntellgence (AI) driven process, using real-lime interactive visual and audio prompts. BACKGROUND [0002] Companies typically spend large amounts of money advertising newly launched products, by adve1tising on display &,Teens in public places such as shopping centers, train/bus stations, etc. To maximise their profits and return on investment. companies want a maximum number of customers to buy the new product. However, existing display screens used for advertisements only display pre-pro,grammed data. Ir: panicular, existing display screens do not display advertisements based on the customers being rargeted. [0003] Further, in Self Check-Out (SCO) stores. a cashier is not physically pres.em at che billing counter to handle billing for a customer. This has led to an increase in theft rates. Thus, there is. a need in SCO stores for a system to provide an audio/video alarm based on an observation of a theft. Similarly. in the event a customer is unable to use the SCO system properly, lhere is need fot· a system to alert a store supervisor that the customer may require attention and assistance. [0004] Furthermore, ir. the event of a natural disaster (e.g. earthquakes, fire, tsunami. etc.), an alam1 may be needed to alert people in public places about the disaster and provide them with directions to a place of greate1· safety. In emergency situations such as an attack on an army/naval base or a terrorist attack at a public pl.ace, current safety procedures do not help to locate the attacker or otherwise provide video/audio instructions ro help officials to catch lhe attacker. CA 03159848 2022· 5- 27 2 [0005] Furthermore. current systems for coaching/training students, fail to observe the environment of the students and adapt the coaching/training accordingly. For example, current training systems execute pre-established procedures for training medical or veterinary students and fail to take into account the environment of the students. [0006] Hence. in view of the above, there exists a need for a system that takes into account an environment of one or more target users, and provides automated audio/video outputs accordingly. SUMMARY [0007] In an aspect of the present disclosure, there is provided a system for auto1natically displaying output information on a display surface based on one or more activities in a target environment. The system includes an image capturing device configured to capture image and video data of the target environment in real-time for 1·ecognizing one or more activities. The system may include a processing unit configured to process the image and video data to determine a location for displaying the output information, aJld detemiine a content of the output infonnation_ The system may further include a video output unit configured to display the output information on the display surface, wherein the video output unit is configured to rotate in a 360° manner, and wherein a11 angle of rotation of the video output unit is controlled by the processing unit. The image capmring device is further configured to caprure the output information displayed on the display surface in real time, and provide feedback data to the pmcessing unit in real-time, so as to enable the processing unit to control a focus, a direction, and the output infonnation displayed on the video output unit. [0008] In another aspect of the present disclos!ll'e, there is provided a method for automatically displaying output information on a display surface based on one or mme activities in a target environment. The method includes capturing image and video data of the cai·get environment in real-time for recognizing one or more activities. The method may further include processing the image and video data to determine a location for displaying rhe output information, and determine a content of the output information. The method may furthc£ include displaying the output infonnation on the display surface by a video output unit. wherein the video output unit is configured to rotate in a 360° manner_ The method may further include controlling an angle of rotation of the video output unit based on the captured image and video data. The method CA 03159848 2022· 5· 27 3 may furthe1· include capturing the output infonnation displayed on the display surface in real time, and generating feedback data to contt·ol a focus, a direction, and the output information displayed on the video omput unit. [0009] In yet another aspect of the present disclosure. there is pmvided a computer programmable prnduct for automatically displaying output information on a display surface b.ased on one or more activities in a target environment. The com