CN-121478505-B - Data processing method and system based on hardware resource reconstruction

CN121478505BCN 121478505 BCN121478505 BCN 121478505BCN-121478505-B

Abstract

The application discloses a data processing method and a system based on hardware resource reconstruction, wherein the method comprises the steps of obtaining hardware configuration parameters and processing types of data to be processed, determining operator types and a running water level processing sequence corresponding to the processing types, splitting data to be processed according to the hardware configuration parameters and the operator types to obtain a plurality of data blocks to be processed, determining a plurality of candidate data processing running water levels corresponding to the operator types in a preset data processing running water level list, determining a plurality of target data processing running water levels corresponding to the data blocks to be processed according to available processing parameters corresponding to the candidate data processing running water levels and target processing parameters of the data blocks to be processed, and processing the corresponding data blocks to be processed according to the running water level processing sequence based on the target data processing running water levels.

Inventors

LIU MAN
WANG LIYU
ZHANG QIHUI

Assignees

广州万协通信息技术有限公司

Dates

Publication Date: 20260508
Application Date: 20260109

Claims (9)

1. A data processing method based on hardware resource reconstruction, the method comprising: Acquiring a hardware configuration parameter and a processing type of data to be processed, and determining an operator type and a pipeline stage processing sequence corresponding to the processing type, wherein the hardware configuration parameter comprises an inherent performance parameter and a real-time resource state parameter of available hardware resources in an auxiliary driving system, the operator type comprises a data processing function class of each computing module in the available hardware resources, and each pipeline stage comprises a processing step or a processing stage; According to the hardware configuration parameters and the operator types, carrying out data splitting on the data to be processed to obtain a plurality of data blocks to be processed; Determining a plurality of candidate data processing pipeline stages corresponding to the operator types in a preset data processing pipeline stage list, and determining a plurality of target data processing pipeline stages corresponding to each data block to be processed according to available processing parameters corresponding to each candidate data processing pipeline stage and target processing parameters of each data block to be processed; Determining a plurality of necessary data processing pipeline stages corresponding to the pipeline stage processing sequence, determining a data processing pipeline stage to be supplemented corresponding to each data block to be processed according to the plurality of necessary data processing pipeline stages and a plurality of target data processing pipeline stages corresponding to each data block to be processed, and supplementing the plurality of target data processing pipeline stages corresponding to each data block to be processed according to the plurality of candidate data processing pipeline stages and the data processing pipeline stages to be supplemented to obtain a plurality of final data processing pipeline stages corresponding to each data block to be processed; And processing the corresponding data blocks to be processed according to the pipeline processing order based on the target data processing pipeline stages.
2. The method for processing data based on hardware resource reconstruction according to claim 1, wherein the splitting the data to be processed according to the hardware configuration parameters and the operator type to obtain a plurality of data blocks to be processed includes: determining a basic splitting length according to the hardware configuration parameters, identifying a data format of the data to be processed, and determining an association splitting length according to the data format and the operator type; And performing length bidirectional expansion on the basic splitting length according to the association splitting length to obtain a target splitting length, and performing data splitting on the data to be processed according to the target splitting length to obtain a plurality of data blocks to be processed.
3. The data processing method based on hardware resource reconstruction according to claim 2, wherein the number of operator types is a plurality; The determining the association splitting length according to the data format and the operator type comprises the following steps: under the condition that the data format is a structured data format, determining the data processing association demand length corresponding to each operator type, comparing the magnitude relation of a plurality of data processing association demand lengths, and taking the maximum data processing association demand length in the magnitude relation comparison result as an association splitting length; Under the condition that the data format is not a structured data format, determining a target similarity algorithm corresponding to each operator type, respectively calculating actual similarity between adjacent data to be processed in the data to be processed according to each target similarity algorithm, determining the length of associated adjacent data of the data to be processed according to a preset similarity threshold and the actual similarity, and taking the length of the associated adjacent data as an associated splitting length.
4. A data processing method based on hardware resource reconstruction according to claim 3, wherein said determining the associated adjacent data length of the data to be processed according to a preset similarity threshold and the actual similarity comprises: Carrying out algorithm consistency check on a plurality of target similarity algorithms, and grouping a plurality of actual similarities according to an algorithm consistency check result to obtain a plurality of similarity groups, wherein the similarity dimensions corresponding to the similarity groups are different; Normalizing the actual similarity, and screening the similarity normalization result according to a preset similarity threshold to obtain a plurality of target similarity under each similarity dimension; And calculating a single-dimensional average value of a plurality of target similarities under each similarity dimension, and calculating a multi-dimensional average value of all the single-dimensional average values to obtain the associated adjacent data length of the data to be processed.
5. The method for processing data based on hardware resource reconstruction according to claim 1, wherein the available processing parameters include an available processing dimension parameter and an available processing precision parameter, and the target processing parameters include a target processing dimension parameter and a target processing precision parameter; Determining a plurality of target data processing pipeline stages corresponding to the data blocks to be processed according to the available processing parameters corresponding to the candidate data processing pipeline stages and the target processing parameters of the data blocks to be processed, including: determining an available processing dimension range corresponding to each candidate data processing pipeline stage according to the available processing dimension parameters, and performing processing dimension compatibility verification on the available processing dimension range corresponding to each candidate data processing pipeline stage and the target processing dimension parameters of each data block to be processed to obtain a plurality of first target data processing pipeline stages compatible with the processing dimension of each data block to be processed; And carrying out processing precision consistency check on the available processing precision parameters corresponding to the first target data processing pipeline stages and the target processing precision parameters of the data blocks to be processed, and determining a plurality of second target data processing pipeline stages matched with the processing precision of the data blocks to be processed according to the processing precision consistency check result to obtain a plurality of target data processing pipeline stages corresponding to the data blocks to be processed.
6. The hardware resource reconfiguration based data processing method of claim 1, wherein after obtaining a plurality of final data processing pipeline stages corresponding to each of the data blocks to be processed, the method further comprises: And respectively sending a pipeline stage activation instruction to a plurality of final data processing pipeline stages corresponding to the data blocks to be processed, and sending a clock turn-off instruction to the rest data processing pipeline stages in the preset data processing pipeline stage list.
7. A data processing system based on hardware resource reconstruction, the system comprising: The operator type determining module is used for acquiring a hardware configuration parameter and a processing type of data to be processed, determining an operator type and a flow level processing sequence corresponding to the processing type, wherein the hardware configuration parameter comprises an inherent performance parameter and a real-time resource state parameter of available hardware resources in an auxiliary driving system, the operator type comprises a data processing function category of each computing module in the available hardware resources, and each flow level comprises a processing step or a processing stage; the data splitting module is used for splitting the data to be processed according to the hardware configuration parameters and the operator types to obtain a plurality of data blocks to be processed; the pipeline stage determining module is used for determining a plurality of candidate data processing pipeline stages corresponding to the operator types in a preset data processing pipeline stage list, and determining a plurality of target data processing pipeline stages corresponding to each data block to be processed according to available processing parameters corresponding to each candidate data processing pipeline stage and target processing parameters of each data block to be processed; The complementary pipeline stage determining module is used for determining a plurality of necessary data processing pipeline stages corresponding to the pipeline stage processing sequence, and determining the complementary data processing pipeline stages corresponding to the data blocks to be processed according to the necessary data processing pipeline stages and a plurality of target data processing pipeline stages corresponding to the data blocks to be processed; The pipeline level supplementing module is used for supplementing the pipeline level of the target data processing pipeline levels corresponding to the data blocks to be processed based on the candidate data processing pipeline levels and the data processing pipeline levels to be supplemented, so as to obtain a plurality of final data processing pipeline levels corresponding to the data blocks to be processed; And the data processing module is used for processing the corresponding data blocks to be processed according to the pipeline stage processing sequence based on the target data processing pipeline stages.
8. An electronic device comprising a processor, a memory and a program or instruction stored on the memory and running on the processor, which when executed by the processor, implements the steps of the hardware resource reconstruction based data processing method of any one of claims 1-6.
9. A readable storage medium, characterized in that the readable storage medium has stored thereon a program or instructions which, when executed by a processor, implement the steps of the data processing method based on hardware resource reconstruction as claimed in any one of claims 1-6.

Description

Data processing method and system based on hardware resource reconstruction Technical Field The application belongs to the technical field of electronic digital data processing, and particularly relates to a data processing method and system based on hardware resource reconstruction. Background Along with the development of the driving assistance technology, the driving assistance functions are continuously increased, and the vehicle control end also presents diversified characteristics for realizing tasks to be processed for the driving assistance functions. In order to ensure accurate processing of data corresponding to tasks to be processed and avoid waste of irrelevant hardware resources, data processing based on hardware resources has become a key research direction of auxiliary driving industry. In the prior art, a data processing mode mainly includes constructing a plurality of different types of data processing hardware architectures in advance, and when data to be processed is received, inputting the data to be processed into the hardware architectures of the corresponding types according to the data types for data processing. However, the prior art has the problems that the utilization mode of hardware resources is inflexible in the data processing process and the data processing result is inaccurate. Disclosure of Invention The embodiment of the application aims to provide a data processing method and a system based on hardware resource reconstruction, which solve the problems of inflexible utilization mode of hardware resources and inaccurate data processing result in the prior art, split data to be processed through hardware configuration parameters and operator types, and determine a target data processing pipeline stage according to available processing parameters corresponding to each candidate data processing pipeline stage and target processing parameters of each data block to be processed so as to process data based on the target data processing pipeline stage, thereby achieving the purpose of dynamically distributing the hardware resources in pipeline stage based on data processing requirements and improving the flexibility of hardware resource utilization and the accuracy of data processing results. In a first aspect, an embodiment of the present application provides a data processing method based on hardware resource reconstruction, where the method includes: Acquiring hardware configuration parameters and processing types of data to be processed, and determining operator types and a running water level processing sequence corresponding to the processing types; according to the hardware configuration parameters and the operator types, carrying out data splitting on the data to be processed to obtain a plurality of data blocks to be processed; Determining a plurality of candidate data processing pipeline stages corresponding to operator types in a preset data processing pipeline stage list, and determining a plurality of target data processing pipeline stages corresponding to each data block to be processed according to available processing parameters corresponding to each candidate data processing pipeline stage and target processing parameters of each data block to be processed; And processing the corresponding data blocks to be processed according to the serial water level processing sequence based on the multiple target data processing serial water levels. Further, according to the hardware configuration parameters and the operator types, splitting the data to be processed to obtain a plurality of data blocks to be processed, including: Determining a basic splitting length according to hardware configuration parameters, identifying a data format of data to be processed, and determining an association splitting length according to the data format and operator types; and performing length bidirectional expansion on the basic splitting length according to the associated splitting length to obtain a target splitting length, and performing data splitting on the data to be processed according to the target splitting length to obtain a plurality of data blocks to be processed. Further, the number of operator types is a plurality; Determining the association splitting length according to the data format and the operator type comprises the following steps: under the condition that the data format is a structured data format, determining the data processing association demand length corresponding to each operator type, comparing the magnitude relation of a plurality of data processing association demand lengths, and taking the maximum data processing association demand length in the magnitude relation comparison result as an association splitting length; Under the condition that the data format is not the structured data format, determining a target similarity algorithm corresponding to each operator type, respectively calculating actual similarity between adjacent data to be processed in the