KR-20260066621-A - UPDATES OF BACKGROUND LAYERS DURING ENCODING

Abstract

A technique for updating a background layer during the encoding of a scene is provided. The scene is encoded based on classifying objects depicted in the scene as foreground or background. The background is divided into sequential background layers, each background layer associated with a respective depth model. The method includes the step (s102) of detecting a change in an image portion of one background layer. The method includes the step (s104) of calculating the difference between the image portion and the corresponding image portion of a background layer sequentially placed behind the one background layer. The method includes the step (s106a) of selecting the background layer sequentially placed behind the one background layer to represent the image portion when the difference is smaller than a threshold value.

Inventors

• 요한 스테닝
• 송 위안

Assignees

• 엑시스 에이비

Dates

Publication Date

20260512

Application Date

20251015

Priority Date

20241104

Claims (12)

1. A method for updating a background layer during encoding of a scene, performed by an image processing device, wherein the scene is encoded based on classifying objects depicted in the scene into foreground or background, and the background is divided into ordered background layers associated with each depth model (dm), and the method A step of detecting changes in an image portion of a single background layer; A step of calculating the difference between the above image portion and the corresponding image portion of a background layer ordered after the above one background layer; If the difference is smaller than a threshold, a step of selecting a background layer ordered after the one background layer to represent the image portion; and A method comprising the step of creating a new background layer to represent the image portion when the difference is not smaller than the threshold.
2. In paragraph 1, The above method additionally, A method comprising the step of merging the new background layer with the background layer ordered after the new background layer when the merge time (mt) associated with the background layer ordered after the new background layer expires.
3. In paragraph 1, A method in which each background layer is associated with its respective merge time (mt).
4. In paragraph 3, A method in which the representation of an object in the above scene is merged into the given background layer when the object remains in a stationary state for longer than the merge time (mt) of the given background layer in the above scene.
5. In paragraph 2 or 3, A method in which a given background layer is merged into a background layer ordered after the given background layer after the given background layer has existed for a longer time (mt) than the merge time (mt) of the background layer ordered after the given background layer.
6. In paragraph 1, A method in which the above background layers are aligned according to a merge time (mt), and the background layer with the shortest merge time (mt) is positioned closest to the foreground.
7. In paragraph 1, A method in which the above image portion has a depth value provided by a depth model (dm) of a background layer representing the above image portion.
8. In paragraph 1, The above method additionally, A method comprising the step of encoding the foreground and background layers into an encoded video stream of the scene.
9. An image processing device for updating a background layer during encoding of a scene, wherein the scene is encoded based on classifying objects depicted in the scene as foreground or background, the background is divided into ordered background layers associated with each depth model (dm), and the image processing device includes a processing circuit, and the processing circuit is configured to cause the image processing device to perform the following: An operation to detect changes in an image portion of a single background layer; An operation to calculate the difference between the above image portion and the corresponding image portion of a background layer ordered after the above one background layer; When the difference is smaller than the threshold, the action of selecting a background layer ordered after the one background layer to represent the image portion; and If the above difference is not smaller than a threshold, the action of creating a new background layer to represent the image portion.
10. In Paragraph 9, An image processing device further configured to perform a method according to any one of paragraphs 2 through 8.
11. A computer program for updating a background layer during encoding of a scene, wherein the scene is encoded based on classifying objects depicted in the scene as foreground or background, the background is divided into ordered background layers associated with each depth model (dm), and the computer program comprises computer code that causes the image processing device to perform the following when the computer program is executed in the processing circuit of an image processing device: An operation to detect changes in an image portion of a single background layer; An operation to calculate the difference between the above image portion and the corresponding image portion of a background layer ordered after the above one background layer; When the difference is smaller than the threshold, the action of selecting a background layer ordered after the one background layer to represent the image portion; and If the above difference is not smaller than a threshold, the operation of creating a new background layer to represent the image portion.
12. A computer program product comprising a computer program according to claim 11, and a computer-readable storage medium on which said computer program is stored.

Description

Updates of background layers during encoding {UPDATES OF BACKGROUND LAYERS DURING ENCODING} The embodiments set forth in this specification relate to a method for updating a background layer during scene encoding, an image processing device, a computer program, and a computer program product. Depth perception is an essential element for understanding and interpreting the surrounding environment in various fields, and is particularly important in applications requiring three-dimensional (3D) spatial information. The ability to accurately determine the distance, shape, and size of objects within a scene enables more precise analysis, enhanced object detection, and improved decision-making capabilities. Depth information is particularly useful in environments where distinguishing objects based on distance or size is important, and can be utilized, for example, in security systems, robotics, autonomous vehicles, and other systems that operate based on visual data. Traditionally, depth perception has been implemented through three-dimensional (3D) cameras or other specialized sensors, which provide a detailed understanding of the environment. Depth perception capabilities offer several advantages, such as enabling more accurate object detection and reducing false alarms by filtering out objects that may appear larger or closer than they actually are. For example, depth information can improve upon situations where objects might be misidentified if based solely on two-dimensional (2D) images, by providing more comprehensive information about the actual spatial relationships within the scene. Depth information can be extracted through various methods. In some cases, monocular camera systems that estimate depth using advanced computational models based on images acquired from a single viewpoint may be used. In other cases, depth information is derived based on parallax measurements obtained from overlapping images captured by multiple sensors, such as in multi-sensor panoramic systems. These systems determine the relative distance of an object by calculating the difference in object position between images. Another method involves measuring the distance of an object using sampling data from a laser pointer in laser-equipped pan-tilt-zoom (PTZ) cameras. Additionally, depth information can be provided by analyzing the movement and positional changes of an object over time through self-learning techniques based on object tracking. While these approaches may be effective in relatively static environments, they face several challenges when applied to more dynamic scenes. Each method generally requires a certain amount of time to process data and produce accurate depth information. For example, monocular models often involve computationally intensive processing, and systems relying on PTZ cameras may require time for the camera to physically sweep or pan the scene to collect sufficient data. This latency can hinder the ability to update depth information in real-time or near-real-time, particularly in situations where large objects move quickly and depth changes rapidly. In dynamic environments where objects can move unpredictably or at varying speeds, it becomes more difficult to keep depth perception systems up to date in real time. This problem is particularly pronounced when large objects move significantly in the depth direction, at which point the system may not be able to adapt quickly enough to provide accurate and up-to-date information. Therefore, there is a need for a more efficient method to maintain accurate depth perception in situations where both static and dynamic elements exist within a scene. The concept of the present invention is described by example with reference to the attached drawings, wherein FIG. 1 is a schematic diagram illustrating an image processing device according to an embodiment; FIG. 2 schematically illustrates a foreground-background representation of a scene according to an embodiment; FIG. 3 shows a flowchart of a method according to an embodiment; FIG. 4 illustrates a first embodiment in which a change occurs in an image portion of a background layer according to an embodiment; FIG. 5 illustrates a second embodiment in which a change occurs in an image portion of a background layer according to an embodiment; FIG. 6 is a schematic diagram illustrating a structural unit of an image processing device according to an embodiment; FIG. 7 illustrates an example of a computer program product including a computer-readable storage medium according to an embodiment. The concept of the invention is described in more detail below with reference to the accompanying drawings, in which some embodiments according to the concept of the invention are illustrated. However, the concept of the invention may be embodied in various forms and should not be interpreted as being limited to the embodiments described herein. Rather, such embodiments are provided as examples to ensure that the disclosure is s