EP-4742648-A1 - ELECTRONIC DEVICE AND CONTROL METHOD THEREFOR

EP4742648A1EP 4742648 A1EP4742648 A1EP 4742648A1EP-4742648-A1

Abstract

An electronic apparatus including memory storing one or more instructions and at least one processor that is connected with the memory and controls the electronic apparatus, wherein the at least one processor is configured to, by executing the one or more instructions, identify an area where a motion is maintained in an input image, identify motion information of the identified area and representative motion information of the identified area, identify a motion correction value based on the identified motion information and the identified representative motion information, perform motion compensation for an input frame included in the input image by applying the identified motion correction value to the identified area, obtain an interpolation frame by applying the identified motion correction value to the identified area, and obtain an output image based on the input frame for which the motion compensation was performed and the obtained interpolation frame.

Inventors

KIM, SHINHAENG
PARK, Woonsung
SONG, WONSEOK

Assignees

Samsung Electronics Co., Ltd.

Dates

Publication Date: 20260513
Application Date: 20240814

Claims (15)

An electronic apparatus comprising: memory storing one or more instructions; and at least one processor is configured to, by executing the one or more instructions: identify an area where a motion is maintained in an input image, identify motion information of the identified area and representative motion information of the identified area, identify a motion correction value based on the identified motion information and the identified representative motion information, perform motion compensation for an input frame included in the input image by applying the identified motion correction value to the identified area, obtain an interpolation frame by applying the identified motion correction value to the identified area, and obtain an output image based on the input frame for which the motion compensation was performed and the obtained interpolation frame.
The electronic apparatus of claim 1, wherein the at least one processor is configured to, by executing the one or more instructions: obtain the interpolation frame by using the identified motion correction value for the identified area, and not using the identified motion correction value for an area other than the identified area.
The electronic apparatus of claim 2, wherein the at least one processor is configured to, by executing the one or more instructions: perform motion compensation for the input frame by using the identified motion correction value for the identified area, and not using the identified motion correction value for the area other than the identified area.
The electronic apparatus of claim 1, wherein the at least one processor is configured to, by executing the one or more instructions: obtain a plurality of motion vectors corresponding to the identified area in a plurality of input frames included in the input image, and identify a representative motion vector that is consistently maintained in the identified area based on the obtained plurality of motion vectors.
The electronic apparatus of claim 4, wherein the at least one processor is configured to, by executing the one or more instructions: identify motion correction values corresponding to each of the plurality of input frames based on difference values between each of the plurality of motion vectors corresponding to the identified area and the representative motion vector.
The electronic apparatus of claim 5, wherein the at least one processor is configured to, by executing the one or more instructions: identify motion correction values corresponding to each of the plurality of input frames by applying a predetermined filter to the difference values between each of the plurality of motion vectors corresponding to the identified area and the representative motion vector.
The electronic apparatus of claim 1, wherein the at least one processor is configured to, by executing the one or more instructions: identify the area where the motion is maintained based on motion vector statistical values for each of a plurality of pixel areas included in the input image, or identify the area where the motion is maintained by inputting the input image into a trained artificial intelligence model.
The electronic apparatus of claim 1, wherein the at least one processor is configured to, by executing the one or more instructions: identify a moving distance where a sum of differences of luminance values of pixels included in the identified area is minimum in a plurality of input frames included in the input image as a motion vector of the identified area.
The electronic apparatus of claim 1, wherein the at least one processor is configured to, by executing the one or more instructions: identify pixels included in the identified area as a plurality of pixel areas, and identify motion vectors of the identified area based on statistical values of motion vectors identified in units of the plurality of pixel areas.
The electronic apparatus of claim 1, wherein the area where the motion is maintained comprises: an area where a text consistently moves in a specific location of the input image.
A control method of an electronic apparatus, the control method comprising: identifying an area where a motion is maintained in an input image; identifying motion information of the identified area and representative motion information of the identified area; identifying a motion correction value based on the identified motion information and the identified representative motion information; obtaining an output image by applying the identified motion correction value to the identified area; performing motion compensation for an input frame included in the input image by applying the identified motion correction value to the identified area; obtaining an interpolation frame by applying the identified motion correction value to the identified area; and obtaining an output image based on the input frame for which the motion compensation was performed and the obtained interpolation frame.
The control method of claim 11, wherein the obtaining the output image comprises: obtaining the interpolation frame by using the identified motion correction value for the identified area, and not using the identified motion correction value for an area other than the identified area.
The control method of claim 12, wherein the obtaining the output image comprises: performing motion compensation for the input frame by using the identified motion correction value for the identified area, and not using the identified motion correction value for the area other than the identified area.
The control method of claim 11, wherein the identifying the motion information of the identified area and the representative motion information of the identified area comprises: obtaining a plurality of motion vectors corresponding to the identified area in a plurality of input frames included in the input image, and identifying a representative motion vector that is consistently maintained in the identified area based on the obtained plurality of motion vectors.
A non-transitory computer readable medium storing computer instructions that make an electronic apparatus perform operations in case the instructions are executed by a processor of the electronic apparatus, wherein the operations comprise: identifying an area where a motion is maintained in an input image; identifying motion information of the identified area and representative motion information of the identified area; identifying a motion correction value based on the identified motion information and the identified representative motion information; obtaining an output image by applying the identified motion correction value to the identified area; performing motion compensation for an input frame included in the input image by applying the identified motion correction value to the identified area; obtaining an interpolation frame by applying the identified motion correction value to the identified area; and obtaining an output image based on the input frame for which the motion compensation was performed and the obtained interpolation frame.

Description

[Technical Field] The disclosure relates to an electronic apparatus and a control method thereof, and more particularly, to an electronic apparatus that performs a motion compensation for an input image, and a control method thereof. [Background Art] Spurred by the development of electronic technologies, various types of electronic apparatuses are being developed and distributed. In particular, development and distribution of display apparatuses such as a TV and a mobile, etc. are actively going on. As an example, for providing an image of better image quality to a user, various frame interpolation methods providing smooth motions are being studied. [Disclosure] [Technical Solution] An electronic apparatus according to an embodiment of the disclosure includes memory storing one or more instructions and at least one processor configured to, by executing the one or more instructions, identify an area where a motion is maintained in an input image, identify motion information of the identified area and representative motion information of the identified area, identify a motion correction value based on the identified motion information and the identified representative motion information, perform motion compensation for an input frame included in the input image by applying the identified motion correction value to the identified area, obtain an interpolation frame by applying the identified motion correction value to the identified area, and obtain an output image based on the input frame for which the motion compensation was performed and the obtained interpolation frame. According to an embodiment, the at least one processor may, by executing the one or more instructions, obtain the interpolation frame by using the identified motion correction value for the identified area, and not using the identified motion correction value for an area other than the identified area. According to an embodiment, the at least one processor may, by executing the one or more instructions, perform motion compensation for the input frame by using the identified motion correction value for the identified area, and not using the identified motion correction value for the area other than the identified area. According to an embodiment, the at least one processor may, by executing the one or more instructions, perform motion compensation for the identified area based on the identified motion correction value in a plurality of input frames included in the input image, and obtain an output image based on the obtained interpolation frame and the plurality of input frames for which the motion compensation was performed. According to an embodiment, the at least one processor may, by executing the one or more instructions, obtain a plurality of motion vectors corresponding to the identified area in a plurality of input frames included in the input image, and identify a representative motion vector that is consistently maintained in the identified area based on the plurality of obtained motion vectors. According to an embodiment, the at least one processor may, by executing the one or more instructions, identify motion correction values corresponding to each of the plurality of input frames based on difference values between each of the plurality of motion vectors corresponding to the identified area and the representative motion vector. According to an embodiment, the at least one processor may, by executing the one or more instructions, identify motion correction values corresponding to each of the plurality of input frames by applying a predetermined filter to the difference values between each of the plurality of motion vectors corresponding to the identified area and the representative motion vector. According to an embodiment, the at least one processor may, by executing the one or more instructions, identify the area where the motion is maintained based on motion vector statistical values for each of a plurality of pixel areas included in the input image, or identify the area where the motion is maintained by inputting the input image into a trained artificial intelligence model. According to an embodiment, the at least one processor may, by executing the one or more instructions, identify a moving distance where a sum of differences of luminance values of pixels included in the identified area is minimum in a plurality of input frames included in the input image as a motion vector of the identified area. According to an embodiment, the at least one processor may, by executing the one or more instructions, identify pixels included in the identified area as a plurality of pixel areas, and identify motion vectors of the identified area based on statistical values of motion vectors identified in units of the plurality of pixel areas. According to an embodiment, the area where the motion is maintained may include an area where a text consistently moves in a specific location of the input image. According to an embodiment, a control met