Search

CN-122018822-A - Heterogeneous SoC data shunt acquisition and self-adaptive storage method and system

CN122018822ACN 122018822 ACN122018822 ACN 122018822ACN-122018822-A

Abstract

The invention relates to the technical field of data processing, in particular to a heterogeneous SoC (system on chip) data shunt acquisition and self-adaptive storage method and system, which are used for solving the problems of overhigh CPU load and large storage delay when a plurality of sensors are accessed. The invention establishes a logic annular buffer zone in the DDR memory at the PS end, adjusts the number of DMA transmission blocks according to the occupation condition, and updates a disk read pointer after completing data self-adaptive storage in three modes of low delay, standard batch and high-speed continuity, releases the buffer zone after completing disk dropping, and is used for the PL end to circularly write new data. According to the invention, high-efficiency and low-loss storage of the large data stream of the multi-channel sensor is realized through the synergistic effect of the heterogeneous SoCs, the data acquisition continuity and the storage reliability are improved, and the service life of the storage equipment is prolonged.

Inventors

  • WANG ZONGSHENG
  • Ji Zhenghe
  • ZHANG WEI
  • WANG SHENGLI
  • ZHANG LINYU
  • Qin Huiqi
  • LIU CHENGMING
  • Ping Jinzhi
  • LIU JIA
  • ZHANG ZHIHAO

Assignees

  • 山东科技大学

Dates

Publication Date
20260512
Application Date
20260413

Claims (9)

  1. 1. The heterogeneous SoC data shunt acquisition and self-adaptive storage method is based on the heterogeneous SoC, wherein the heterogeneous SoC comprises a PL end and a PS end, and is characterized by comprising the following steps: s1, accessing a plurality of types of sensors by using a PL (programmable logic device), and splitting a data stream generated by the sensors into a high-speed data stream channel and a low-speed control stream channel by a hardware level splitting mechanism; S2, processing the original data of various sensors at the PL end, and simultaneously packaging the processed original data into data in a unified frame format by combining a hardware time stamp; s3, mapping a high-speed data Stream in a high-speed data Stream channel to a DDR memory at a PS end through an AXI4-Stream interface, and mapping a low-speed control Stream in a low-speed control Stream channel to a register through the AXI4-Lite interface; S4, constructing a logic annular buffer zone in the DDR memory of the PS end, and detecting the occupation amount of the logic annular buffer zone in real time And according to the occupation amount of the logic ring buffer Dynamically adjusting the number of blocks in a batch of a single DMA transfer to NVMe memory And after the data is dropped, updating a disk read pointer, and releasing a buffer area where the disk is dropped, so that the PL end can circularly write new data.
  2. 2. The heterogeneous SoC data splitting and collecting and self-adaptive storage method according to claim 1, wherein the hardware level splitting mechanism in S1 is based on a classification of a payload size and a transmission bandwidth of a data frame, sets a splitting threshold, classifies data with the payload size and the transmission bandwidth of the data frame being greater than or equal to the splitting threshold into a high-speed data flow channel, and classifies data with the payload size and the transmission bandwidth of the data frame being less than the splitting threshold into a low-speed control flow channel.
  3. 3. The heterogeneous SoC data shunt collection and self-adaptive storage method according to claim 1, wherein when the PL end processes raw data of a plurality of types of sensors, an independent FIFO memory is configured for each sensor interface to realize real-time caching of corresponding sensor data.
  4. 4. The heterogeneous SoC data shunt acquisition and adaptive storage method according to claim 1, wherein the constructing in the DDR memory of the PS side is performed by Each buffer block A logical ring buffer of components, said buffer blocks The size of the buffer block is aligned with the physical page or the erase block of the NVMe solid state disk, and the PL end is fully written with one buffer block The PS side is notified by interrupt or register mode, and then the PL side writes the hardware write pointer of the address Automatically move to the next buffer block.
  5. 5. The heterogeneous SoC data stream acquisition and adaptive storage method as claimed in claim 4, wherein, The calculation formula of the occupation amount of the logic ring buffer area is as follows: ; In the formula, For the number of buffer blocks that are not currently dropped, A hardware write pointer to write an address for the PL side, To point to the disk read pointer of the disk drop address, The total number of buffer blocks for the circular buffer, To take the following measures The modulus is obtained by modular operation.
  6. 6. The heterogeneous SoC data offload collection and adaptive storage method of claim 1, wherein the dynamically adjusting the number of blocks in bulk from a single DMA transfer to NVMe memory The strategy of (1) comprises: setting a low water level threshold And a high water level threshold ; When (when) When the system enters a low-delay waiting mode, the system does not immediately drop a disc, post-command meeting overtime forced disc-drop condition ; When (when) When the standard batch writing mode is entered, the preset batch number is set Carrying out batch tray dropping; When (when) When entering the continuous writing mode, the method sets Is the maximum value supported by the DMA and promotes the priority of the writing thread.
  7. 7. The heterogeneous SoC data shunt acquisition and self-adaptive storage method according to claim 1, wherein the data landing is that a PS end sends a physical address instruction to an NVMe controller, and the controller directly reads DDR data through a PCIe bus and lands, so that zero copy in the whole process is realized.
  8. 8. The heterogeneous SoC data shunt acquisition and adaptive storage method according to claim 1, wherein after the data landing is completed, the system updates the disk read pointer according to the number of transmitted blocks And releasing the buffer zone of the completed landing disk for the PL end to circularly write new data.
  9. 9. Heterogeneous SoC data based split acquisition and adaptive storage system, characterized in that the heterogeneous SoC data based split acquisition and adaptive storage method according to any of claims 1 to 8 is applied.

Description

Heterogeneous SoC data shunt acquisition and self-adaptive storage method and system Technical Field The invention relates to the technical field of data processing, in particular to a heterogeneous SoC data shunt acquisition and self-adaptive storage method and system. Background Along with the rapid iteration of the automatic driving technology and the intelligent upgrading of complex equipment (such as intelligent unmanned equipment, special detection equipment and the like), the multi-source sensor is fused into a core support for realizing accurate perception and real-time decision of a system. In an actual application scene, the system is usually required to be simultaneously connected with a plurality of types of sensors such as a laser radar, a binocular camera, an Inertial Measurement Unit (IMU), a Global Navigation Satellite System (GNSS) and the like, and the various sensors work cooperatively to provide omnibearing data support such as environment sensing, position positioning, gesture detection and the like for the system, and the data acquisition efficiency, transmission stability and storage reliability of the system directly determine the running performance of the whole system. However, there are still many pain points to be solved in the multi-source sensor data acquisition and storage technology in the prior art, specifically as follows: The core parameters of different types of sensors have significant differences, the data types cover continuous video streams (such as binocular cameras), discrete state values (such as angular velocity and acceleration of an IMU) and the like, communication protocols are diversified and comprise different standards of CAN buses, RS485, ethernet, LVDS and the like, the transmission rates and data frame formats of various protocols are mutually incompatible, the sampling frequency span is extremely large, the sampling frequency is from 10Hz low-frequency sampling of GNSS to 1kHz high-frequency sampling of the IMU, the traditional data acquisition card mostly adopts a fixed interface design, flexible hardware adaptation capability is lacked, and unified access and synchronous acquisition of the multi-source sensor are difficult to realize. In the traditional data acquisition architecture, all sensor data are required to be carried, analyzed and forwarded by a Central Processing Unit (CPU), and the CPU bears all data processing and transmission tasks. When high-frequency high-bandwidth sensors such as a laser radar and a high-definition binocular camera are connected, massive data can continuously occupy CPU calculation power, so that CPU interruption requests are frequent, system response delay is increased, problems such as untimely data processing and cache overflow can occur, data packet loss is further caused, real-time sensing and decision making capability of a system on a complex scene is seriously influenced, and even potential safety hazards of automatic driving vehicles and intelligent equipment can be caused. For high-frequency large data flow, the traditional storage mode can write data into storage media such as a hard disk after links such as multi-layer cache copying of an operating system and protocol analysis, and the process can introduce extremely large transmission delay, so that the requirement of scenes such as automatic driving on real-time data storage cannot be met. Meanwhile, the multi-layer copying process additionally occupies hardware resources such as a system memory and a CPU, so that the storage throughput is reduced, the efficiency of data storage is affected, the load pressure of the CPU is further increased, a vicious circle of acquisition-processing-storage is formed, and the high-efficiency storage requirement of high-frequency multi-source sensor data is difficult to adapt. Disclosure of Invention The invention provides a heterogeneous SoC data shunt acquisition and self-adaptive storage method and system, which are used for solving the problems of overhigh CPU load and large storage delay when a plurality of sensors are accessed in the prior art. The heterogeneous SoC data shunt acquisition and self-adaptive storage method comprises a PL end and a PS end, and comprises the following steps: s1, accessing a plurality of types of sensors by using a PL (programmable logic device), and splitting a data stream generated by the sensors into a high-speed data stream channel and a low-speed control stream channel by a hardware level splitting mechanism; S2, processing the original data of various sensors at the PL end, and simultaneously packaging the processed original data into data in a unified frame format by combining a hardware time stamp; s3, mapping a high-speed data Stream in a high-speed data Stream channel to a DDR memory at a PS end through an AXI4-Stream interface, and mapping a low-speed control Stream in a low-speed control Stream channel to a register through the AXI4-Lite interface; S4, constructing a logic ann