CN-122023566-A - Video mosaic real-time drawing method based on off-screen Canvas

CN122023566ACN 122023566 ACN122023566 ACN 122023566ACN-122023566-A

Abstract

The application relates to the technical field of Web image processing and discloses a video mosaic real-time drawing method based on off-screen Canvas, which comprises the steps of detecting a target video frame, and determining a target mosaic region needing mosaic processing and position coordinates and region sizes corresponding to the region; the method comprises the steps of creating an off-screen Canvas independent of a main Canvas, scaling and rendering an original video frame to the off-screen Canvas, positioning and capturing a local area which has a mosaic effect generated by scaling on the off-screen Canvas based on the position coordinates and the area size of a target mosaic area, reversely amplifying the local area according to the scaling proportion relation, and rendering the local area to the main Canvas to finish single-frame mosaic superposition, detecting a rendering mode adopted by a current video, and selecting a corresponding rendering strategy according to a detection result.

Inventors

WU JIE
LIU WENZE
WAN MINGXI

Assignees

重庆锐明信息技术有限公司

Dates

Publication Date: 20260512
Application Date: 20260130

Claims (8)

1. The real-time video mosaic drawing method based on off-screen Canvas is characterized by comprising the following steps of: S1, detecting a target video frame through image recognition or external annotation data, and determining a target mosaic area needing mosaic processing and position coordinates and area sizes corresponding to the target mosaic area, wherein the target video frame is rendered and displayed by a main Canvas; S2, creating an off-screen Canvas independent of the main Canvas; S3, scaling and rendering the original video frame to the off-screen Canvas, and determining a local area corresponding to the target mosaic area on the off-screen Canvas according to the position coordinates and the area size of the target mosaic area; S4, cutting the local area, amplifying the local area according to the scaling relationship in the step S3, and drawing the local area to the target mosaic area corresponding to the main Canvas to finish single-frame mosaic superposition; s5, detecting a rendering mode adopted by the current video through a prefabricated multi-mode adapter, and selecting a corresponding rendering strategy according to a detection result; and S6, based on the rendering strategy selected in the step S5, scheduling a corresponding rendering engine to execute the drawing operation in the step S4, completing the single-frame mosaic superposition, and outputting the single-frame mosaic superposition to a display terminal.
2. The method according to claim 1, wherein the step S2 further comprises: S210, acquiring the width and the height of an original video frame, and reading the width and the height of a preset mosaic block; S220, dividing the width of the original video frame by the unit width of the mosaic block to obtain the width of the Canvas element; dividing the height of the original video frame by the mosaic block unit height to obtain the height of the Canvas element; S230, the width and the height obtained through calculation are respectively assigned to the width attribute and the height attribute of the Canvas element, and the creation of the off-screen Canvas is completed.
3. The method according to claim 1, wherein the step S3 further comprises: S310, calling a drawImage function, and rendering the original video frame onto the off-screen Canvas by taking the reduced size of the off-screen Canvas as a target rendering size; S320, dividing the position coordinates of the target mosaic area by the preset width and height of the mosaic, and mapping to obtain corresponding local area coordinates on the off-screen Canvas; s330, determining a local area finally used for amplifying and synthesizing to the main Canvas based on the local area coordinates.
4. The method according to claim 1, wherein the rendering modes detected and adapted by the prefabricated multimode adapter in step S5 include a Canvas mode for two-dimensional graphics rendering, a WebGL mode for hardware-accelerated three-dimensional or two-dimensional rendering, and an MSE mode for streaming video playback.
5. The method of claim 4, wherein when the rendering mode is detected as a Canvas mode, the multi-mode adapter directly renders the off-screen Canvas through a Canvas 2D context to draw the enlarged local mosaic area to a master Canvas; When the rendering mode is detected to be a WebGL mode, the multimode adapter uploads the off-screen Canvas to be a WebGL texture, samples the texture through a fragment shader and renders the texture to a main Canvas so as to complete superposition display of local mosaic areas.
6. The method of claim 4, wherein when detecting that the rendering mode is an MSE mode, the policy is: Acquiring the actual display width and the actual display height of the target video frame; Acquiring original inherent width and original inherent height of a video stream corresponding to the target video frame, wherein the original inherent width and original inherent height are standard width and standard height preset when a video stream file leaves a factory; calculating a width proportional coefficient and a height proportional coefficient corresponding to the target video frame, wherein the width proportional coefficient is the ratio of the actual display width to the original inherent width, and the height proportional coefficient is the ratio of the actual display height to the original inherent height; and correcting the position coordinates and the region size of the target mosaic region based on the width proportional coefficient and the height proportional coefficient to obtain corrected coordinates and corrected dimensions for drawing in the step S4.
7. The method as recited in claim 1, further comprising: before the step S4 is executed, a merging processing operation is performed on the multiple target mosaic areas, and multiple areas meeting a preset spatial relationship are merged into a smaller number of areas to be drawn, so as to reduce the number of drawing calls in the step S4.
8. The method of claim 7, wherein the merge processing operation comprises: when a plurality of target mosaic areas exist, traversing and analyzing the position relation among the areas; if any two target mosaic areas are adjacent or overlapped, combining the two target mosaic areas into a composite drawing area; In step S4, a single drawing operation is performed on each of the resulting composite drawing areas, instead of multiple drawing of the original multiple target mosaic areas.

Description

Video mosaic real-time drawing method based on off-screen Canvas Technical Field The specification relates to the technical field of Web image processing, in particular to a video mosaic real-time drawing method based on off-screen Canvas. Background The real-time drawing of the video mosaic at the Web end is a common requirement in the field of Web image processing, and is characterized in that the accurate positioning and distortion-free rendering of a mosaic area are realized on the premise of guaranteeing the display smoothness of a picture. The prior art scheme has multiple defects that at the drawing level, the traditional scheme directly carries out mosaic pixel processing on an original video frame, the drawing and rendering process of a video picture are highly coupled, video playing is blocked and the picture drops due to the fact that a main thread is blocked by pixel operation, meanwhile, coordinate positioning and size calculation of a mosaic area are completed by depending on real-time display pictures of the video frame, and if resolution adaptation or rendering mode switching exists in the video, the problems of mosaic area dislocation and block size distortion are easy to occur. In the adaptation level, part of the proposed optimization schemes focus on drawing logic optimization in a single rendering mode or adapt to different scenes through simple parameter adjustment, but the schemes do not design a unified adaptation strategy for multiple rendering modes such as MSE (mean square error) common to a Web end, so that an optimal drawing path is difficult to autonomously select according to the current rendering environment, so that the mosaic processing efficiency and accuracy in different scenes fluctuate greatly, and multi-terminal compatibility and processing performance cannot be considered. Therefore, a real-time video mosaic drawing method based on off-screen Canvas is needed to overcome the defect of low drawing efficiency of the traditional mosaic and the limitation of multi-rendering mode adaptation, and realize efficient, accurate and smooth drawing of the video mosaic at the Web end. Disclosure of Invention In view of the above, the invention aims to provide an off-screen Canvas-based real-time video mosaic drawing method, which solves the problems of playing stuck frame dropping, lack of multi-rendering mode adaptation and mosaic region positioning dislocation distortion caused by direct pixel processing and painting flow coupling at the picture end in the conventional Web end video mosaic drawing, and realizes efficient and smooth manufacture, accurate region matching and multi-mode stable adaptation of the video mosaic. In order to achieve the above purpose, the invention adopts the following technical scheme: an off-screen Canvas-based video mosaic real-time drawing method, comprising the following steps: S1, detecting a target video frame through image recognition or external annotation data, and determining a target mosaic area needing mosaic processing and position coordinates and area sizes corresponding to the target mosaic area, wherein the target video frame is rendered and displayed by a main Canvas; S2, creating an off-screen Canvas independent of the main Canvas; S3, scaling and rendering the original video frame to the off-screen Canvas, and determining a local area corresponding to the target mosaic area on the off-screen Canvas according to the position coordinates and the area size of the target mosaic area; S4, cutting the local area, amplifying the local area according to the scaling relationship in the step S3, and drawing the local area to the target mosaic area corresponding to the main Canvas to finish single-frame mosaic superposition; s5, detecting a rendering mode adopted by the current video through a prefabricated multi-mode adapter, and selecting a corresponding rendering strategy according to a detection result; and S6, based on the rendering strategy selected in the step S5, scheduling a corresponding rendering engine to execute the drawing operation in the step S4, completing the single-frame mosaic superposition, and outputting the single-frame mosaic superposition to a display terminal. The method has the beneficial effects that in the prior art, the problems of play cartoon frame dropping, regional dislocation distortion and scene adaptation performance fluctuation caused by coupling of the painting and dyeing flow of the video mosaic drawing at the Web end are generally solved, and the smoothness and the accuracy of the mosaic drawing are restricted. The method performs mosaic pixel processing by adopting the off-screen Canvas in the internal memory state, the off-screen Canvas is independently created and is not mounted to a DOM tree, mosaic processing operation and video display drawing are separated, real-time drawing efficiency of the video mosaic is greatly improved, picture blocking is avoided, accurate superposition of mosaic areas is e