CN-114092407-B - Video conference shared document definition processing method and device

CN114092407BCN 114092407 BCN114092407 BCN 114092407BCN-114092407-B

Abstract

The embodiment of the invention provides a method for clearing a video conference shared document. The method comprises the steps of extracting brightness components based on YUV format video image data, denoising based on the brightness components to obtain a denoised image, extracting edge information of the denoised image, obtaining sharpening intensity based on the edge information, segmenting foreground and background of the denoised image, and sharpening the denoised image based on the sharpening intensity and the foreground information. The shared document in the video conference is sharpened, so that a user obtains a better visual effect, better experience is brought to the user, and the requirements of the user are met. In addition, the embodiment of the invention provides a device for the video conference shared document sharpening processing.

Inventors

HUANG HUILIN
BAI GANG
DU GUIYU
ZHAO XINGGUO

Assignees

上海赛连信息科技有限公司

Dates

Publication Date: 20260505
Application Date: 20211025

Claims (20)

1. A method for sharpness processing of a shared document for a video conference, comprising: extracting a luminance component based on YUV format video image data; denoising based on the brightness component to obtain a denoised image; Extracting edge information of the denoising image; acquiring sharpening strength based on the edge information; segmenting the foreground and the background of the denoising image; Sharpening the denoising image based on the sharpening strength and foreground information; judging whether the current frame image of the video conference sharing document is a first frame image or not; if the frame image is the first frame image, executing the above-mentioned sharpening processing step; dividing a plurality of areas if the frame image is not the first frame image; judging whether the content of each area and the corresponding area of the previous frame is changed or not; If the area changes, the above-mentioned step of clearing is carried out again on the area; if the area is unchanged, the area is sharpened directly by using the sharpening result of the previous frame.
2. The method according to claim 1, wherein the extracting a luminance component based on the YUV format video image data comprises: A luminance component of the pixel of the region to be processed is extracted based on each frame of image data.
3. The method of claim 1, wherein denoising based on the luminance component to obtain a denoised image comprises: smoothing and filtering the brightness component by adopting a bilateral filtering mode to obtain the denoising image; the bilateral filtering includes a spatial distance parameter and a gray scale distance parameter.
4. The method for sharpness processing of a shared video conference document according to claim 1, wherein the obtaining of the edge information of the denoised image includes: acquiring a low-frequency image based on the denoising image; the difference between the denoised image and the low frequency image is the edge information.
5. The method for sharpness processing of a shared document for video conferencing as claimed in claim 4, wherein the acquiring a low frequency image based on the denoised image comprises: And carrying out Gaussian blur processing on the denoising image to obtain the low-frequency image.
6. The method for sharpness processing of a shared video conference document according to claim 1, wherein the acquiring the sharpening strength based on the edge information includes: Dividing the edge into a white edge and/or a black edge based on positive and negative information of the edge; setting different enhancement coefficients for the white edge and/or the black edge; and acquiring the sharpening strength based on the enhancement coefficient and the edge information.
7. The method of claim 6, wherein the obtaining the sharpening strength based on the enhancement coefficient and the edge information comprises: When the denoising image is divided into white edges, when the edge information is larger than zero and smaller than a first threshold value, sharpening strength is zero; When the edge information is larger than or equal to a first threshold value and smaller than a second threshold value, the enhancement coefficient is a first enhancement coefficient, and the sharpening strength is the product of the first enhancement coefficient and the edge information; when the edge information is larger than or equal to a second threshold value and smaller than a third threshold value, the enhancement coefficient is a second enhancement coefficient, and the sharpening strength is the product of the second enhancement coefficient and the edge information; When the edge information is larger than or equal to a third threshold value and smaller than a fourth threshold value, the enhancement coefficient is a third enhancement coefficient, and the sharpening strength is the product of the third enhancement coefficient and the edge information; And when the edge information is larger than or equal to a fourth threshold value, the sharpening strength is the fourth threshold value.
8. The method of claim 6, wherein the obtaining the sharpening strength based on the enhancement coefficient and the edge information comprises: when the denoising image is divided into black edges, when the edge information is larger than a fifth threshold value and smaller than zero, sharpening strength is zero; when the edge information is larger than the sixth threshold and smaller than or equal to the fifth threshold, the enhancement coefficient is a fourth enhancement coefficient, and the sharpening strength is the product of the fourth enhancement coefficient and the edge information; When the edge information is larger than a seventh threshold value and smaller than or equal to a sixth threshold value, the enhancement coefficient is a fifth enhancement coefficient, and the sharpening strength is the product of the fifth enhancement coefficient and the edge information; When the edge information is larger than the eighth threshold value and smaller than or equal to the seventh threshold value, the enhancement coefficient is a sixth enhancement coefficient, and the sharpening strength is the product of the sixth enhancement coefficient and the edge information; And when the edge information is smaller than or equal to an eighth threshold value, the sharpening strength is the eighth threshold value.
9. The method for sharpness processing of a shared video conference document according to claim 1, wherein the segmentation of the denoised image from the foreground to the background comprises: carrying out gray value statistics on pixel points with sharpening intensities larger than a ninth threshold value in a given window; taking the statistical median value as the background value of the area; If the absolute value of the difference value between the current pixel value and the background value is larger than a preset value, the pixel is a foreground point, otherwise, the pixel is a background point; Pixels with sharpening intensities not greater than the ninth threshold default to foreground points that need to be enhanced.
10. The method for sharpening a shared video conference document according to claim 1, wherein the sharpening is performed on a denoised image based on the sharpening strength and foreground information: Setting the sharpening intensity corresponding to each pixel point in the background to zero; and adding each pixel point of the denoising image with the corresponding sharpening intensity value to sharpen the denoising image.
11. A video conference shared document sharpness processing apparatus, comprising: an extraction module for extracting a luminance component based on YUV format video image data; the denoising module is used for denoising based on the brightness component to obtain a denoised image; The edge information extraction module is used for extracting edge information of the denoising image; The sharpening strength acquisition module is used for acquiring sharpening strength based on the edge information; The segmentation module is used for segmenting the foreground and the background of the denoising image; The sharpening module is used for sharpening the denoising image based on the sharpening strength and the foreground information; The apparatus further comprises: a module for judging whether the current frame image of the video conference sharing document is a first frame image or not; A module for executing a sharpening process step for the first frame image; a module for dividing the non-first frame image into a plurality of regions; a module for judging whether the content of each area and the corresponding area of the previous frame is changed or not; means for performing the sharpening process step on the region if the region changes; And if the area is unchanged, sharpening the area by directly using the sharpening result of the previous frame.
12. The video conference shared document sharpness processing apparatus according to claim 11, wherein the extracting module includes: and a unit for extracting a luminance component of the region to be processed based on each frame of image data.
13. The video conference shared document sharpness processing apparatus according to claim 11, wherein the denoising module includes: A unit for smoothing the brightness component by adopting a bilateral filtering mode to obtain the denoising image; the bilateral filtering includes a spatial distance parameter and a gray scale distance parameter.
14. The video conference shared document sharpness processing apparatus according to claim 11, wherein the means for obtaining edge information of the denoised image includes: a unit for acquiring a low frequency image based on the denoised image; the difference between the denoised image and the low frequency image is the edge information.
15. The video conference shared document sharpness processing apparatus according to claim 14, wherein the means for acquiring a low-frequency image based on the denoised image comprises: And the unit is used for carrying out Gaussian blur processing on the denoising image to obtain the low-frequency image.
16. The video conference shared document sharpening processing device of claim 11, wherein the acquire sharpening strength module comprises: Means for dividing an edge into a white edge and/or a black edge based on the edge positive and negative information; means for setting different enhancement coefficients for the white edge and/or the black edge; And means for acquiring the sharpening strength based on the enhancement coefficient and the edge information.
17. The video conference shared document sharpness processing apparatus of claim 16, wherein the means for obtaining the sharpening strength based on the enhancement coefficient and the edge information includes: When the denoised image is divided into white edges, means for sharpening the intensity to zero when the edge information is greater than zero and less than a first threshold; When the edge information is greater than or equal to a first threshold value and less than a second threshold value, the enhancement coefficient is a first enhancement coefficient, and the sharpening strength is the product of the first enhancement coefficient and the edge information; when the edge information is greater than or equal to a second threshold value and less than a third threshold value, the enhancement coefficient is a second enhancement coefficient, and the sharpening strength is the product of the second enhancement coefficient and the edge information; When the edge information is greater than or equal to a third threshold value and less than a fourth threshold value, the enhancement coefficient is a third enhancement coefficient, and the sharpening strength is the product of the third enhancement coefficient and the edge information; And when the edge information is larger than or equal to a fourth threshold value, the sharpening strength is the fourth threshold value.
18. The video conference shared document sharpness processing apparatus of claim 16, wherein the means for obtaining the sharpening strength based on the enhancement coefficient and the edge information includes: when the denoising image is divided into black edges, when the edge information is greater than a fifth threshold value and less than zero, sharpening the image at zero; When the edge information is greater than a sixth threshold and less than or equal to a fifth threshold, the enhancement coefficient is a fourth enhancement coefficient, and the sharpening strength is the product of the fourth enhancement coefficient and the edge information; when the edge information is greater than a seventh threshold and less than or equal to a sixth threshold, the enhancement coefficient is a fifth enhancement coefficient, and the sharpening strength is the product of the fifth enhancement coefficient and the edge information; When the edge information is greater than an eighth threshold and less than or equal to a seventh threshold, the enhancement coefficient is a sixth enhancement coefficient, and the sharpening strength is the product of the sixth enhancement coefficient and the edge information; And when the edge information is smaller than or equal to an eighth threshold value, the sharpening strength is the eighth threshold value.
19. The video conference shared document sharpness processing apparatus according to claim 11, wherein the segmentation module includes: A unit for carrying out gray value statistics on pixel points with sharpening intensities larger than a ninth threshold value in a given window; means for taking the statistical median as a background value for the region; a unit for determining the pixel as a foreground point if the absolute value of the difference between the current pixel value and the background value is greater than a preset value, or determining the pixel as a background point if the absolute value of the difference is greater than the preset value; the method comprises defaulting to a foreground point requiring enhancement for pixels having sharpening strengths no greater than a ninth threshold.
20. The video conference shared document sharpness processing apparatus according to any of claims 11 to 18, wherein the sharpening module includes: a unit for setting the sharpening strength corresponding to each pixel point in the background to zero; and the unit is used for adding each pixel point of the denoising image with the corresponding sharpening intensity value to realize sharpening of the denoising image.

Description

Video conference shared document definition processing method and device Technical Field The embodiment of the invention relates to the technical field of video communication, in particular to a method and a device for clearing a shared document of a video conference. Background This section is intended to provide a background or context to the embodiments of the invention that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section. Currently, video communication technology based on the internet is widely used. The screen sharing function is a function which is frequently used among a plurality of functions of the video conference, and is an indispensable large function in daily conferences and remote training. However, due to the lossy compression in the data transmission, the phenomena of blurring edges, ghost shadows and the like exist in the shared document of the receiving end, and the fonts are virtual. In order to improve the visual experience quality of users, the shared picture needs to be sharpened and enhanced, so that the shared picture is clear in edges and corners and fonts. Due to the lossy compressed transmission of video, certain artifacts exist in the received shared document or screen, and if sharpening enhancement is directly performed, the artifacts are amplified, so that the visual effect is reduced. Meanwhile, as the shared document is mainly focused on the characters, the characters are characterized by sharp edge changes, the traditional sharpening method can cause obvious overshoot phenomenon, and obvious white edges are generated on the periphery of the characters in a certain area, so that the picture quality is seriously reduced. The invention designs a method and a device for carrying out real-time sharpening processing on a shared document of a video conference, which carry out smooth denoising on the received document, pay attention to keeping the edges of characters when denoising, segment the foreground and the background based on a denoising image, wherein the foreground is a part (such as the edges of characters and images) needing to be enhanced, acquire a high-frequency part based on the denoising image, divide a detail area and a character edge area according to a set threshold value, acquire sharpening intensity pixel by combining the foreground image, carry out edge sharpening on the denoising image, realize real-time sharpening on the shared screen and the document in the video conference by combining invariance among the same page frames of the shared document, and enable a user to acquire a better visual effect. Disclosure of Invention The invention aims to provide a feasible real-time sharpening scheme for sharing documents in video conferences. The document is sharpened by performing edge protection denoising, foreground background segmentation and foreground high-frequency part sharpening enhancement on the document and the unchanged relation among the document frames of the same page in the video conference. For the existing defects that no real-time sharpening enhancement technology mainly aims at a shared screen and a document and the existing sharpening enhancement technology is directly applied to the shared screen and the document, obvious overshoot phenomenon is caused, and obvious white edges are generated on the periphery of characters in a certain area, an improved technical scheme for carrying out real-time sharpening processing on the document is very needed, so that the problems are solved. In this context, the embodiment of the invention is expected to provide a document sharpness processing method and device. In a first aspect of the embodiment of the invention, a method for sharpening a video conference shared document is provided, which comprises the steps of extracting a brightness component based on YUV format video image data, denoising based on the brightness component to obtain a denoised image, extracting edge information of the denoised image, acquiring sharpening intensity based on the edge information, segmenting a foreground and a background of the denoised image, and sharpening the denoised image based on the sharpening intensity and the foreground information. In one embodiment of the invention, the extracting the luminance component based on the text region includes extracting the luminance component of the region to be processed based on each frame of image data. In another embodiment of the present invention, denoising the luminance component pair to obtain a denoised image includes smoothing the luminance component pair by bilateral filtering to obtain the denoised image, wherein the bilateral filtering includes a spatial distance parameter and a gray scale distance parameter. In yet another embodiment of the present invention, the obtaining the edge information of the denoised image includes obtaining a low frequency image based on the denoised image, and a difference betw