CN-121996444-A - Video reinjection method, system, equipment and readable storage medium based on memory

CN121996444ACN 121996444 ACN121996444 ACN 121996444ACN-121996444-A

Abstract

A video reinjection method, a system, a device and a readable storage medium based on a memory relate to the technical field of video reinjection and comprise the steps of preprocessing acquired image data to obtain an image frame sequence, wherein each frame of data in the image frame sequence is a scheduling record, the image frame sequence comprises image index information of a plurality of virtual video devices belonging to the same synchronous group at the same scheduling moment respectively, the frames are ordered according to a time stamp increasing sequence, a video reinjection layer is controlled to read current frame data from the image frame sequence and write the current frame data into a shared memory, a file descriptor corresponding to the shared memory is sent to the virtual video devices, and a sensor receiving layer is controlled to acquire the file descriptor through the virtual video devices and access and process the current frame data in the shared memory based on the file descriptor so as to realize zero-copy video reinjection. The application realizes video reinjection through the shared memory, can effectively reduce cost and deployment complexity, and improves flexibility.

Inventors

BI YANSHUAI
WANG GUOJUN
RONG LE
LIU YONGGUANG

Assignees

湖北芯擎科技股份有限公司

Dates

Publication Date: 20260508
Application Date: 20260408

Claims (10)

1.A memory-based video reinjection method, the method comprising: preprocessing the acquired image data to obtain an image frame sequence, wherein each frame of data in the image frame sequence is a scheduling record, the scheduling record comprises image index information of a plurality of virtual video devices belonging to the same synchronous group at the same scheduling moment respectively, and frames are ordered according to the increasing sequence of time stamps; The video reinjection layer is controlled to read current frame data from the image frame sequence and write the current frame data into a shared memory, and a file descriptor corresponding to the shared memory is sent to virtual video equipment; And the control sensor receiving layer acquires a file descriptor through the virtual video equipment and accesses and processes the current frame data in the shared memory based on the file descriptor so as to realize zero-copy video reinjection.
2. The memory-based video reinjection method according to claim 1, wherein each frame of data is composed of a single-frame schedule entry and an image index entry, the single-frame schedule entry comprises global timestamp information, synchronization group information and corresponding video channel information, the synchronization group information is used for marking the synchronization group id corresponding to the frame of data, the video channel information is used for marking all video channel ids contained in the corresponding synchronization group, and the image index entry comprises video processing channel ids and image data paths corresponding to the video channel ids.
3. The memory-based video reinjection method according to claim 1, further comprising, after the step of the control video reinjection layer reading current frame data from the image frame sequence and writing it into a shared memory: Determining a target prediction time according to the system trigger time of the next frame, the target IO reading time consumption and a preset safety margin; when the real-time system time reaches the target prediction time, controlling a video reinjection layer to read next frame data from an image frame sequence and pre-write the next frame data into a shared memory; The target IO reading time is obtained through prediction based on average IO reading time, burst jitter time, maximum IO reading time and a system load correction factor, and the time window length corresponding to the maximum IO reading time is smaller than the time window length corresponding to the average IO reading time.
4. The memory-based video reinjection method according to claim 3, wherein the calculation expression of the target prediction time is: in the formula, The target predicted time of day is indicated, Indicating the system trigger time for the next frame, Indicating that the target IO read is time consuming, Representing a preset safety margin.
5. The memory-based video reinjection method according to claim 3 or 4, wherein the target IO reading time-consuming calculation expression is: in the formula, Representing the average IO read time consumption, Indicating that the maximum IO read is time consuming, Indicating that burst jitter is time consuming, Representing the system load correction factor.
6. The memory-based video reinjection method according to claim 5, wherein the burst jitter time consumption is determined based on a last frame IO read time consumption and a last frame IO read time consumption, the system load correction factor is determined based on a CPU usage rate in a statistical period and a time duty ratio of the CPU in an IO waiting state in the statistical period, and the burst jitter time consumption is calculated by the following expression: The calculation expression of the system load correction factor is as follows: in the formula, Indicating that the last frame IO read is time consuming, Indicating that the last frame IO read is time consuming, Represents the CPU utilization during the statistical period, Representing the time duty cycle of the CPU in the IO wait state during the statistical period, And All of which represent the weight coefficient, Indicating the load correction factor upper limit.
7. The memory-based video reinjection method according to claim 1, wherein the sending the file descriptor corresponding to the shared memory to the virtual video device comprises: If the synchronous group of the virtual video equipment is detected to have a synchronous requirement lower than a preset requirement, when the real-time system time is the same as the timestamp information corresponding to the frame data, controlling a video reinjection layer to send a file descriptor corresponding to the shared memory to the virtual video equipment; If the synchronous group pair synchronous requirement of the virtual video equipment is higher than the preset requirement, when the timing value reaches the preset trigger time, the video reinjection layer is controlled to send a file descriptor corresponding to the shared memory to the virtual video equipment.
8. A video reinjection system based on a memory is characterized by comprising a data storage module, a video reinjection layer, at least one virtual video device and a sensor receiving layer; the data storage module is used for preprocessing the acquired image data to obtain an image frame sequence and storing the image frame sequence, wherein each frame of data in the image frame sequence is a scheduling record which contains image index information of a plurality of virtual video devices belonging to the same synchronous group at the same scheduling moment respectively, and the frames are ordered according to the increasing sequence of time stamps; The video reinjection layer is used for reading current frame data from the image frame sequence stored on the data storage module, writing the current frame data into the shared memory, and sending a file descriptor corresponding to the shared memory to the virtual video equipment; and the sensor receiving layer acquires a file descriptor through the virtual video equipment and accesses and processes the current frame data in the shared memory based on the file descriptor so as to realize zero-copy video reinjection.
9. A memory-based video reinjection device, characterized in that it comprises a processor, a memory and a memory-based video reinjection program stored on the memory and executable by the processor, wherein the memory-based video reinjection program, when executed by the processor, implements the steps of the memory-based video reinjection method according to any of claims 1 to 7.
10. A computer readable storage medium, wherein a memory-based video reinjection program is stored on the computer readable storage medium, wherein the memory-based video reinjection program, when executed by a processor, implements the steps of the memory-based video reinjection method according to any of claims 1 to 7.

Description

Video reinjection method, system, equipment and readable storage medium based on memory Technical Field The application relates to the technical field of video reinjection, in particular to a memory-based video reinjection method, a memory-based video reinjection system, memory-based video reinjection equipment and a readable storage medium. Background In an automatic driving system, a camera is widely applied to tasks such as lane line detection, target recognition, obstacle tracking and the like as a core sensor. In order to verify the stability and robustness of the visual perception module, the acquired or constructed image sequence is often re-injected into the system for functional testing, simulation verification and scene reproduction. In order to achieve the above objective, the industry commonly adopts a "video reinjection" technology, that is, existing image frame data is reinjected into a camera data path for testing, verifying or debugging of a sensing module, so that the whole system is "in order to be" the image from a real sensor. The mainstream technology at present is implemented by using a video reinjection card, which essentially simulates one or more cameras in a hardware manner, so that a receiving platform collects image frame data received by a link, as shown in fig. 1, and a typical working flow is as follows: (1) The host machine drive, wherein the video reinjection card is required to be installed on a host machine device, and the host machine realizes the configuration, image loading and reinjection control of the reinjection card through a specific drive; (2) The host computer transmits the image frame prepared in advance to the frame buffer zone of the video reinjection card through communication interfaces such as USB, PCIe (PERIPHERAL COMPONENT INTERCONNECT EXPRESS, high-speed serial computer expansion bus standard) and the like, and the image frame is used as an image source for simulating the signal output of the camera; (3) The video reinjection card can output the simulated camera video signal to the receiver through a high-speed serial interface such as GMSL (Gigabit Multimedia SERIAL LINK ‌, ‌ kilomega multimedia serial link) and the like so as to realize the image data injection effect similar to that of a real camera; (4) The receiving side restores the received camera video signal into MIPI-CSI (a serial communication interface) data stream through the deserializer, and the MIPI-CSI data stream is accessed into a camera interface module CIF of a system-on-chip SoC, the CIF completes channel access, and the data is converted into an image frame data stream which accords with the input specification of an image signal processor ISP and is sent into the ISP, so that the image receiving processing flow is completed. Therefore, when the video reinjection card is used for realizing the video reinjection, the prior art has the problems that 1) the high-cost high-performance video reinjection card is high in price, the high-performance video reinjection card needs to be installed on a host supporting specific driving, the system cost is high, when more multiple cameras are needed, multiple video reinjection cards are needed, the cost is multiplied, 2) the deployment is complex, the special interface connection, the power supply, the driving installation, the platform matching and the like are involved, the debugging is complex and time-consuming, and 3) the flexibility is insufficient, and under the condition that the hardware resources of a team are limited, the use requirements of each member are difficult to flexibly and rapidly meet. Disclosure of Invention The application provides a video reinjection method, a system, equipment and a readable storage medium based on a memory, which can solve the technical problems of high cost, complex deployment and insufficient flexibility caused by adopting a video reinjection card to realize video reinjection in the prior art. In a first aspect, an embodiment of the present application provides a memory-based video reinjection method, where the method includes: preprocessing the acquired image data to obtain an image frame sequence, wherein each frame of data in the image frame sequence is a scheduling record, the scheduling record comprises image index information of a plurality of virtual video devices belonging to the same synchronous group at the same scheduling moment respectively, and frames are ordered according to the increasing sequence of time stamps; The video reinjection layer is controlled to read current frame data from the image frame sequence and write the current frame data into a shared memory, and a file descriptor corresponding to the shared memory is sent to virtual video equipment; And the control sensor receiving layer acquires a file descriptor through the virtual video equipment and accesses and processes the current frame data in the shared memory based on the file descriptor so as to realize zero-copy vide