Search

CN-121985176-A - Cross-platform video decoding and playing system and method

CN121985176ACN 121985176 ACN121985176 ACN 121985176ACN-121985176-A

Abstract

The invention discloses a cross-platform video decoding and playing system and a method, which belong to the technical field of video monitoring and data processing, and initiate a three-level degradation strategy, wherein single manufacturer plug-in faults do not affect the overall situation, and the full compatibility of multi-brand equipment and old codes is realized by combining standardized interface mapping. The edge gateway integrates code stream adjustment, double code stream switching and caching, ensures multi-channel concurrency stability, adopts synchronous ready, GPU texture synthesis and compensation algorithm, breaks through bandwidth bottleneck of multi-channel independent pushing, supports 16-channel synchronous smooth playing, abandons old plug-ins, realizes cross-platform non-differential access of PC, mobile phone and browser, and is flexibly adapted to internal and external network scenes by dual-mode deployment.

Inventors

  • YANG LEI
  • HE WEIZHI
  • ZHANG KAI

Assignees

  • 北京安视华业科技有限责任公司

Dates

Publication Date
20260505
Application Date
20260407

Claims (10)

  1. 1. A cross-platform video decoding and playing system applied to a comprehensive security platform for processing multi-manufacturer heterogeneous video streams, comprising: The protocol adaptation layer is configured to dock monitoring equipment of different manufacturers through the unified protocol adaptation module, and converts heterogeneous video streams based on a private protocol or a standard protocol into standardized video streams with unified formats; the client decoding layer is deployed at the user end, is configured to receive the standardized video stream, and comprises an intelligent decoding unit, wherein the intelligent decoding unit is used for monitoring hardware resources of the client in real time and dynamically switching a hard decoding mode and a soft decoding mode according to the state of the hardware resources; the network transmission optimization layer is deployed in the equipment-side network environment and comprises an edge gateway, wherein the edge gateway is configured to aggregate multiple paths of video streams and execute cross-network-segment forwarding, and meanwhile, code stream parameters are dynamically adjusted according to network bandwidth; the playing composition layer is deployed at the user end and comprises a GPU acceleration composition engine, and is configured to perform synchronous alignment and pixel level fusion on the video frames after multipath decoding to generate a single composite picture frame; and the cross-platform rendering layer is configured to receive the synthesized picture frame at the browser end through a WebSocket protocol and call a WebGL interface to perform hardware acceleration rendering.
  2. 2. The system of claim 1, wherein the protocol adaptation layer comprises: The abstract interface module defines standardized streaming, decoding and control interfaces; a plurality of vendor specific adapters, each adapter corresponding to a vendor's private SDK, for mapping calls of the standardized interface to a corresponding vendor's private API; And the dynamic plug-in manager is used for dynamically loading or unloading the vendor proprietary adapter, and automatically triggering a three-level degradation strategy when detecting the decoding failure of the adapter, wherein the three-level degradation strategy is that firstly, the adapter is switched to RTSP standard protocol decoding, and if the adapter still fails, the universal soft decoder is called.
  3. 3. The system of claim 2, wherein the intelligent decoding unit of the client decoding layer comprises: The coding format normalization subunit is used for automatically identifying the coding format of the video stream and converting codes which are not in the H.265/H.264 standard into standard formats in real time; the hardware resource monitoring subunit is used for collecting the calculation power and the CPU occupancy rate of the GPU at the client; And the decision tree model decides whether the current decoding task is allocated to the hardware decoder or the software decoder according to the output of the hardware resource monitoring subunit.
  4. 4. The system of claim 3, wherein the edge gateway of the network transport optimization layer is configured to: Executing double code stream logic switching, requesting subcode streams of a camera in a real-time preview scene, and requesting main code streams in a video playback scene; built-in buffer queues provide buffer data to clients to maintain playback continuity when the network is down.
  5. 5. The system of claim 4, wherein the GPU acceleration composition engine of the play composition layer comprises: The synchronous controller adopts a full channel frame ready strategy, aligns decoding progress of the multi-channel video, and enables a dynamic frame compensation algorithm for a channel with overtime decoding; A GPU video memory synthesizer which calls a graphic API to draw and splice a plurality of YUV format frames as textures at pixel level by opening a unified buffer area in the GPU video memory; And the self-adaptive encoder is used for carrying out secondary compression encoding on the synthesized single picture frame and dynamically adjusting the encoding rate according to the network condition.
  6. 6. The system of claim 5, wherein the cross-platform rendering layer further comprises: WebAssembly decoding module, which is used for carrying out soft decoding on the received compressed data packet at the browser end; And the WebGL rendering pipeline is used for uploading the decoded RGBA data into GPU textures, and realizing multi-picture segmentation, self-adaptive scaling and OSD information superposition through a shader.
  7. 7. The system of claim 6, wherein the system comprises a dual mode deployment: in a client mode, the client decoding layer, the play synthesis layer and the cross-platform rendering layer are all operated on a user terminal; In the server mode, the client decoding layer and the playing synthesis layer run on a cloud server, and the cross-platform rendering layer runs on a mobile browser or a lightweight client.
  8. 8. A cross-platform video decoding playing method based on the system of any one of claims 1 to 7, comprising the following steps: docking multi-manufacturer equipment through a protocol adaptation layer, and converting the heterogeneous protocol into a standardized video stream; The client decoding layer receives the video stream and dynamically selects a hard decoding mode or a soft decoding mode according to local hardware resources; the edge gateway gathers multipath streams and dynamically adjusts code rate to optimize network transmission; The play composition layer synchronously composes the multipath decoding frames into a single picture through the GPU; the cross-platform rendering layer pushes the synthesized picture to the browser through the WebSocket, and calls the WebGL to render.
  9. 9. The method of claim 8, wherein the playing composition layer synchronously composes the multiple decoded frames into a single picture by the GPU comprises: detecting ready state of each path of decoding frame, and carrying out frame rate alignment by taking the lowest frame rate channel as a reference; generating a compensation frame for a channel which is not ready after timeout through an interpolation algorithm based on the difference value between the previous frame and the next frame; And uploading the aligned multipath YUV frames to a GPU video memory, and performing texture drawing in a shader according to a preset layout to complete pixel-level fusion.
  10. 10. A computer-readable storage medium, characterized in that it stores a computer program which, when executed by a processor, implements the steps of the method according to claim 8 or 9.

Description

Cross-platform video decoding and playing system and method Technical Field The invention belongs to the technical field of video monitoring and data processing, and particularly relates to a cross-platform video decoding and playing system and method. Background At present, the video monitoring system is widely applied in the fields of security protection, internet of things and the like. The comprehensive security platform is an intelligent security center integrating functions of video monitoring, alarm linkage, access control management, fire monitoring, intelligent analysis and the like, and realizes the upgrade from 'passive monitoring' to 'active early warning' by integrating multi-dimensional security equipment and data, and is widely applied to multiple scenes such as government enterprises, parks, traffic, communities and the like, wherein the compatibility of equipment for multi-manufacturer video monitoring is a core function and a main technical difficulty. At present, main stream monitoring equipment manufacturers in the market all have respective video coding and decoding schemes and are only used on respective platforms of the manufacturers, and when the comprehensive security platform is integrated with multi-manufacturer monitoring equipment, the comprehensive security platform is faced with the technical bottlenecks of poor compatibility, low decoding efficiency, video playing delay, blocking, screen display, high network bandwidth requirement, difficult cross-platform adaptation and the like, and the specific problems are as follows: The compatibility is limited in that each manufacturer depends on a private protocol and an SDK, the interface isomerism is strong, an adaptive code is required to be developed in a targeted way during integration, and the interface incompatibility is easy to be caused by the iteration of newly added equipment or the SDK of the manufacturer; the traditional server centralized decoding mode occupies a large amount of server resources, the display of a client is easy to cause screen display and blocking, the CPU occupancy rate is too high, so that the client cannot normally operate other services; The network transmission problem is outstanding, namely the problems of high delay, blocking, cutoff and the like exist in the transmission of a cross-network segment and a public network, and the network bandwidth pressure is high in the concurrent playing of multiple channels; The cross-platform adaptation is difficult, the browser playing is realized by relying on ActiveX, NPAPI and other old plug-ins, the modern browser eliminates the plug-ins, and the mobile terminal (iOS/Android/Harmony OS) and the PC terminal need to be subjected to end-to-end development adaptation, so that the multi-terminal deployment cost is high, and the compatibility is difficult to unify. In summary, when the current comprehensive security platform integrates multi-manufacturer monitoring equipment, four core pain points of heterogeneous protocols, low decoding efficiency, unstable network transmission and difficult cross-platform adaptation are faced. The prior solution adopts a single technical means, such as accessing only through ONVIF/GB28181 standard protocol, leading to the loss of advanced functions of manufacturers, or adopting centralized decoding at a server end, leading to overload of CPU and screen-display blocking in high concurrency scene, or relying on an ActiveX plug-in to realize webpage playing, which has been eliminated by modern browsers. In addition, the prior art lacks a systematic architecture, and each module is isolated and optimized, so that the problems of dynamic fault tolerance, hardware resource self-adaptive scheduling, low-delay picture synthesis, cross-terminal consistent experience and the like in a multi-manufacturer environment can not be cooperatively solved. Disclosure of Invention In order to overcome the defects in the prior art, the application provides a cross-platform video decoding and playing system and a method. In a first aspect, the present application provides a cross-platform video decoding and playing system, which is applied to a comprehensive security platform to process multi-manufacturer heterogeneous video streams, and includes: The protocol adaptation layer is configured to dock monitoring equipment of different manufacturers through the unified protocol adaptation module, and converts heterogeneous video streams based on a private protocol or a standard protocol into standardized video streams with unified formats; the client decoding layer is deployed at the user end, is configured to receive the standardized video stream, and comprises an intelligent decoding unit, wherein the intelligent decoding unit is used for monitoring hardware resources of the client in real time and dynamically switching a hard decoding mode and a soft decoding mode according to the state of the hardware resources; the network transmission optimization