KR-20260063899-A - DISPLAY DEVICE FOR PROCESSING IMAGE INCLUDING OBJECT OF INTEREST AND METHOD THEREOF

KR20260063899AKR 20260063899 AKR20260063899 AKR 20260063899AKR-20260063899-A

Abstract

The present disclosure proposes a display device for performing motion estimation. The display device can identify the shape of an object of interest included in a first frame of an image, acquire motion information indicating the movement of the object of interest between the first frame and a second frame based on the shape of the object of interest, and perform motion compensation for the image based on the motion information. The operation of acquiring motion information may include an operation of setting different weights for a region of interest corresponding to the shape of the object of interest and a background region that is not a region of interest. The set weights are used to identify a matching block among candidate search blocks of the second frame that matches a reference block containing the object of interest of the first frame, and the reference block and the matching block may be used to acquire motion information for the object of interest.

Inventors

이영호

Assignees

삼성전자주식회사

Dates

Publication Date: 20260507
Application Date: 20241031

Claims (20)

In a display device, display; Memory comprising at least one storage medium containing instructions; and It includes at least one processor comprising a processing circuit, wherein the at least one processor is: Identify the object of interest included in the first frame of the image, and Based on the shape of the object of interest, motion information indicating the movement of the object of interest between the first frame and the second frame is obtained, and Based on the above motion information, a motion compensation frame is added between the first frame and the second frame, and The action of acquiring the above movement information is: It includes an operation of setting different weights for a region of interest corresponding to the object of interest and a background region other than the region of interest, A display device in which the above-determined weights are used to identify a matching block among the candidate search blocks of the second frame that matches a reference block containing the object of interest of the first frame, and the reference block and the matching block are used to obtain motion information regarding the object of interest.
In paragraph 1, The operation of setting the above different weights is: A display device comprising the operation of setting a first weight value to pixels included in the area of interest and setting a second weight value to pixels included in the background area, wherein the first weight value is greater than the second weight value.
In paragraph 1 or 2, The action of acquiring the above movement information is: An operation to identify the block through a block matching algorithm using a specified similarity measurement method; and A display device comprising an operation to acquire motion information for the object of interest based on the reference block and the matching block.
In paragraph 3, The operation of identifying the above matching block is: A display device comprising an operation to measure the similarity between the reference block and each candidate search block using the first weight value and the second weight value set above, through the specified similarity measurement method.
In paragraph 4, The aforementioned specified similarity measurement method corresponds to the SAD (sum of squared differences) method, and The above SAD method is a display device that is performed using the following mathematical formula. [Mathematical Formula] Here, Pre(i,j): Pixel value corresponding to position (i,j) in the reference block of the first frame Cur(i,j): Pixel value corresponding to position (i,j) in the candidate search block of the second frame wgt(i,j): Weight value of the pixel corresponding to position (i,j) m: vertical size of the block n: horizontal size of the block k: Index of the above candidate search block
In paragraph 1, The operation of identifying the shape of the object of interest above is: The method includes an operation of obtaining information about the shape of the object of interest included in the first frame using a trained artificial intelligence model, and A display device in which the above-mentioned learned artificial intelligence model is trained to output output data containing information about the shape of the object of interest based on an input frame.
In paragraph 6, The above output data further includes interest object detection information, and A display device comprising at least one of the above interest object detection information, type information for the type of the interest object, type information for the subtype of the interest object, or reliability information for the reliability of the interest object.
In paragraph 6, The operation of identifying the shape of the object of interest above is: A display device comprising an operation of sub-sampling the image at a specified ratio so that the object of interest is included in one block.
In paragraph 1, The first frame and the second frame correspond to consecutive frames, and the motion information corresponds to a motion vector for the object of interest, and The operation of performing the above motion compensation for the above image is: A display device comprising the operation of generating the motion compensation frame between the first frame and the second frame based on the motion vector.
In Paragraph 9, The above motion compensation frame is a display device used for frame rate conversion, motion compensation interpolation, or motion judder cancellation.
In a method of a display device, An action of identifying an object of interest included in the first frame of an image; An operation to acquire motion information indicating the movement of the object of interest between the first frame and the second frame based on the shape of the object of interest; and Based on the above motion information, the method includes adding a motion compensation frame between the first frame and the second frame, and The action of acquiring the above movement information is: A method comprising the operation of setting different weights for a region of interest corresponding to the object of interest and a background region other than the region of interest, wherein the set weights are used to identify a matching block among candidate search blocks of the second frame that matches a reference block containing the object of interest of the first frame, and wherein the reference block and the matching block are used to obtain motion information for the object of interest.
In Paragraph 11, The operation of setting the above different weights is: A method comprising the operation of setting a first weight value to pixels included in the area of interest and setting a second weight value to pixels included in the background area, wherein the first weight value is greater than the second weight value.
In Article 11 or Article 12, The action of acquiring the above movement information is: An operation of identifying the matching block through a block matching algorithm using a specified similarity measurement method; and A method comprising the operation of acquiring motion information for the object of interest based on the reference block and the matching block.
In Paragraph 13, The operation of identifying the above matching block is: A method comprising the operation of measuring the similarity between the reference block and each candidate search block through the specified similarity measurement method using the first weight value and the second weight value set above.
In Paragraph 14, The aforementioned specified similarity measurement method corresponds to the SAD (sum of squared differences) method, and The above SAD method is a method performed using the following mathematical formula. [Mathematical Formula] Here, Pre(i,j): Pixel value corresponding to position (i,j) in the reference block of the first frame Cur(i,j): Pixel value corresponding to position (i,j) in the candidate search block of the second frame wgt(i,j): Weight value of the pixel corresponding to position (i,j) m: vertical size of the block n: horizontal size of the block k: Index of the above candidate search block
In Paragraph 11, The operation of identifying the shape of the object of interest above is: The method includes an operation of obtaining information about the shape of the object of interest included in the first frame using a trained artificial intelligence model, and A method in which the above-mentioned learned artificial intelligence model is trained to output output data containing information about the shape of the object of interest based on an input frame.
In Paragraph 16, The above output data further includes interest object detection information, and A method comprising at least one of type information for the type of the object of interest, type information for the subtype of the object of interest, or reliability information for the reliability of the object of interest.
In Paragraph 16, The operation of identifying the shape of the object of interest above is: A method comprising the operation of sub-sampling the image at a specified ratio so that the object of interest is included in one block.
In Paragraph 11, The first frame and the second frame correspond to consecutive frames, and the motion information corresponds to a motion vector for the object of interest, and The operation of performing the above motion compensation for the above image is: A method comprising the operation of generating the motion compensation frame between the first frame and the second frame based on the motion vector.
In Paragraph 19, The above motion compensation frame is a method used for frame rate conversion, motion compensation interpolation, or motion judder cancellation.

Description

Display device for processing image including object of interest and method thereof The present disclosure relates to a display device and method for processing an image including an object of interest (object). When watching a video, viewers can focus their attention on a specific object. For example, when watching a sports video, viewers can focus their attention on the movement of a specific object, such as a ball. Accurately estimating the movement of the object of interest that the viewer focuses on is necessary to provide natural video to the viewer. For example, accurate motion compensation can be performed through the precise estimation of the object of interest's movement between consecutive frames. Through this, video containing moving objects of interest can be presented to the viewer more smoothly. The information described above may be provided as related art for the purpose of aiding understanding of this document. None of the above is to be claimed as prior art related to this document, nor can it be used to determine prior art. In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components. FIG. 1 is a configuration diagram of a display device according to one embodiment of the present disclosure. FIG. 2 is a drawing illustrating an image including an object of interest according to one embodiment of the present disclosure. FIG. 3 is a diagram illustrating the configuration of a display device for performing an operation to estimate the movement of an object of interest according to one embodiment of the present disclosure. FIG. 4a is a drawing illustrating the movement of an object of interest in a series of frames including the object of interest, according to an embodiment of the present disclosure. FIG. 4b is a drawing illustrating a region of interest and a background region corresponding to an object of interest according to one embodiment of the present disclosure. FIG. 5 is a flowchart illustrating the operation of a display device estimating the movement of an object of interest according to one embodiment of the present disclosure. FIGS. 6a to 6c are drawings illustrating the configuration of a display device for performing an operation to estimate the movement of an object of interest according to one embodiment of the present disclosure. FIG. 7a is a drawing illustrating an artificial intelligence model used to detect an object of interest and/or the shape of an object of interest according to one embodiment of the present disclosure. FIG. 7b is a drawing illustrating output data obtained through the artificial intelligence model of FIG. 7a according to one embodiment of the present disclosure. FIG. 8 is a diagram illustrating the operation of a display device setting different weights for a region of interest and a background region corresponding to an object of interest, according to one embodiment of the present disclosure. FIG. 9 is a diagram illustrating the operation of a display device subsampling an image including an object of interest according to one embodiment of the present disclosure. FIG. 10 is a diagram illustrating the operation of a display device performing motion compensation based on motion information according to one embodiment of the present disclosure. Hereinafter, embodiments of the present disclosure are described in detail with reference to the drawings so that those skilled in the art can easily practice them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein. In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and brevity. FIG. 1 is a configuration diagram of a display device according to one embodiment of the present disclosure. According to one embodiment, the display device (100) may be a smartphone, tablet PC, PC, smart TV, mobile phone, PDA (personal digital assistant), laptop, media player, micro server, digital broadcasting terminal, navigation, kiosk, home appliance, and other mobile or non-mobile computing devices, but is not limited thereto. The display device (100) may perform various computing functions such as real-time video viewing and communication. In the following description, the display device (100) is described on the premise that it is a TV or a monitor, but this is merely an example and the embodiments of the present disclosure may be equally applied to electronic devices having display functions. Referring to FIG. 1, the display device (100) may include a processor (110), memory (120), image input unit (130), display (140), and communication unit (150). According to one embodiment, the memory (120) is a storage medium used by the display device (100) and can s