CN-121262546-B - Data interaction method, platform, device, medium and program product

CN121262546BCN 121262546 BCN121262546 BCN 121262546BCN-121262546-B

Abstract

The invention relates to the technical field of data processing and provides a data interaction method, a platform, equipment, a medium and a program product, wherein the method comprises the steps of receiving a transmission request for sending data from source equipment to target equipment; the method comprises the steps of responding to a transmission request, reading data to be transmitted in source equipment through a first working thread, writing the data to be transmitted into a shared storage space, reading the data to be transmitted in the shared storage space through a second working thread, and writing the data to be transmitted into target equipment, wherein the first working thread and the second working thread are executed in parallel in a pipeline mode. According to the invention, the shared memory space is introduced into the master control equipment and the dual-thread pipeline parallel processing mechanism is matched, so that the software and hardware decoupling and high-speed concurrent transmission of data transmission among heterogeneous equipment are realized, and the method has the advantages of strong universality, high transmission efficiency and good system compatibility.

Inventors

Request for anonymity
Request for anonymity
Request for anonymity
Request for anonymity
Request for anonymity

Assignees

上海壁仞科技股份有限公司

Dates

Publication Date: 20260512
Application Date: 20251127

Claims (12)

1. The data interaction method is characterized in that the method is applied to a master control device, the master control device is provided with a shared storage space, the master control device is respectively in communication connection with a source device and a target device, and the source device and the target device are heterogeneous devices, and the method comprises the following steps: receiving a transmission request for transmitting data from the source device to the target device; responding to the transmission request, reading data to be transmitted in the source equipment through a first working thread, and writing the data to be transmitted into the shared storage space; reading the data to be transmitted in the shared storage space through a second working thread, and writing the data to be transmitted into the target device; wherein the first worker thread and the second worker thread execute in parallel in a pipelined manner.
2. The method for data interaction according to claim 1, wherein the reading, by the first working thread, the data to be transmitted in the source device further includes: and dividing the data to be transmitted into a plurality of data packets according to the transmission request, so that the first working thread and the second working thread execute the reading and writing operation of the data by taking the data packets as granularity.
3. The data interaction method according to claim 2, wherein the dividing the data to be transmitted into a plurality of data packets according to the transmission request includes: Analyzing the transmission request to obtain the total data quantity and the target address of the data to be transmitted; determining the data size of a single data packet based on a preset slicing strategy, and slicing the data to be transmitted into a plurality of data segments according to the total data amount and the data size; Generating the data packet containing a metadata header for each of the data segments, the metadata header including a packet sequence number of the data packet and the destination address.
4. The method according to claim 2, wherein the reading, by the first working thread, the data to be transmitted in the source device, and writing the data to be transmitted into the shared storage space, includes: Sequentially adding the divided data packets into a first task queue; extracting a first data packet to be processed currently from the first task queue through the first working thread; And reading target data corresponding to the first data packet from the source equipment based on the information of the first data packet, and writing the target data into the shared storage space.
5. The method of data interaction according to claim 4, wherein extracting, by the first worker thread, the first data packet currently to be processed from the first task queue includes: The first task queue is monitored through the first working thread while the data to be transmitted is divided into a plurality of data packets; and under the condition that the existence of the data packet in the first task queue is detected, extracting the first data packet from the first task queue according to a first-in first-out principle.
6. The data interaction method of claim 4, wherein after the step of writing the target data into the shared memory space, the method further comprises: Generating a second data packet carrying the storage address of the target data in the shared storage space, and adding the second data packet into a second task queue; The reading the data to be transmitted in the shared memory space and writing the data to be transmitted into the target device through the second working thread comprises the following steps: extracting the second data packet from the second task queue through the second work thread; And reading target data corresponding to the storage address from the shared storage space based on the storage address carried in the second data packet, and writing the target data into the target device.
7. The data interaction method of claim 6, wherein after the step of writing the target data to the target device, the method further comprises: And destroying the second data packet in the second task queue through the second working thread, and releasing memory resources occupied by the target data in the shared storage space.
8. The data interaction method of any one of claims 1 to 7, further comprising: acquiring a current system load state; and according to the system load state, adjusting the data size of a single data packet in a preset slicing strategy and the queue depths of the first task queue and the second task queue.
9. A data interaction platform, comprising a main control device, at least one source device and at least one target device, wherein a shared storage space is arranged on the main control device, the main control device is respectively in communication connection with the at least one source device and the at least one target device, and the main control device is used for realizing the data interaction method according to any one of claims 1 to 8.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and running on the processor, characterized in that the processor implements the data interaction method of any of claims 1 to 8 when executing the computer program.
11. A non-transitory computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when executed by a processor, implements the data interaction method according to any of claims 1 to 8.
12. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the data interaction method according to any of claims 1 to 8.

Description

Data interaction method, platform, device, medium and program product Technical Field The present invention relates to the field of data processing technologies, and in particular, to a data interaction method, a platform, a device, a medium, and a program product. Background With the evolution of the architecture of the artificial intelligence radio access network (ARTIFICIAL INTELLIGENCE Radio Access Network, AI-RAN), the cooperative processing of radio access networks and artificial intelligence services by heterogeneous computing platforms such as field programmable gate arrays (Field Programmable GATE ARRAY, FPGA) and General-purpose graphics processors (GPGPU) has become a mainstream trend. Currently, in order to implement data transmission between devices, a technology based on direct memory access (Direct Memory Access, DMA) is generally adopted to implement direct interconnection between devices. However, the technology is seriously dependent on specific hardware design and proprietary data transmission protocol, so that the universality is poor, and in the face of different hardware platforms, developers often need to perform targeted bottom layer adaptation and repeated development, so that the problems of high development cost, high coupling degree of software and hardware and poor portability exist in the existing scheme. Disclosure of Invention The invention provides a data interaction method, a platform, equipment, a medium and a program product, which are used for solving the defects of poor compatibility and difficult migration caused by excessive dependence on specific hardware and protocols in data transmission among heterogeneous equipment in the related technology. The invention provides a data interaction method, which is applied to main control equipment, wherein the main control equipment is provided with a shared storage space and is respectively in communication connection with source equipment and target equipment, and the source equipment and the target equipment are heterogeneous equipment, and the method comprises the following steps: receiving a transmission request for transmitting data from the source device to the target device; responding to the transmission request, reading data to be transmitted in the source equipment through a first working thread, and writing the data to be transmitted into the shared storage space; reading the data to be transmitted in the shared storage space through a second working thread, and writing the data to be transmitted into the target device; wherein the first worker thread and the second worker thread execute in parallel in a pipelined manner. According to the data interaction method provided by the invention, the reading of the data to be transmitted in the source device by the first working thread further comprises: and dividing the data to be transmitted into a plurality of data packets according to the transmission request, so that the first working thread and the second working thread execute the reading and writing operation of the data by taking the data packets as granularity. According to the data interaction method provided by the invention, the data to be transmitted is divided into a plurality of data packets according to the transmission request, and the data interaction method comprises the following steps: Analyzing the transmission request to obtain the total data quantity and the target address of the data to be transmitted; determining the data size of a single data packet based on a preset slicing strategy, and slicing the data to be transmitted into a plurality of data segments according to the total data amount and the data size; Generating the data packet containing a metadata header for each of the data segments, the metadata header including a packet sequence number of the data packet and the destination address. According to the data interaction method provided by the invention, the data to be transmitted in the source device is read through the first working thread, and the data to be transmitted is written into the shared storage space, and the method comprises the following steps: Sequentially adding the divided data packets into a first task queue; extracting a first data packet to be processed currently from the first task queue through the first working thread; And reading target data corresponding to the first data packet from the source equipment based on the information of the first data packet, and writing the target data into the shared storage space. According to the data interaction method provided by the invention, the extracting, by the first working thread, the first data packet to be processed currently from the first task queue includes: The first task queue is monitored through the first working thread while the data to be transmitted is divided into a plurality of data packets; and under the condition that the existence of the data packet in the first task queue is detected, extracting the fir