CN-121480586-B - Data processing method and system based on AI chip

CN121480586BCN 121480586 BCN121480586 BCN 121480586BCN-121480586-B

Abstract

The invention provides a data processing method and a system based on an AI chip, which relate to the technical field of electric digital data processing, wherein the method comprises the steps of obtaining a current data unit in a target data stream; the method comprises the steps of calculating data similarity C between the data similarity C and a previous data unit, calculating entropy attenuation indexes D {N} between adjacent calculation layers in layer-by-layer processing of a neural network, wherein the entropy attenuation indexes D {N} are used for quantifying the reduction degree of information uncertainty, carrying out joint judgment on the data similarity C and the entropy attenuation indexes D {N} , and interrupting the processing of a subsequent calculation layer and rejecting the characteristic data of the current level as redundant data when the data similarity C is higher than a similarity threshold and the data similarity D {N} is lower than an attenuation threshold. According to the invention, by constructing the dual verification criteria of the input similarity and the entropy change of the internal processing of the model, the dynamic pruning of the calculation path is realized, the redundant calculation amount can be reduced on the premise of not sacrificing the risk identification precision, and the processing efficiency and the energy efficiency ratio of the AI chip are improved.

Inventors

ZHAN ZHU
LIU JUN
SU HAIBIN
Shen Lurong
PENG LIJUN
WANG QUANBAO
QIU QI
TANG CHUANGUO

Assignees

方心科技股份有限公司

Dates

Publication Date: 20260508
Application Date: 20260108

Claims (9)

1. The data processing method based on the AI chip is characterized by comprising the following specific steps of acquiring a target data stream comprising a plurality of data units; for the current data unit, calculating the data similarity C between the current data unit and the previous data unit in the target data stream; Performing layer-by-layer forward propagation processing on the current data unit through a plurality of calculation layers of the neural network model; before performing the layer-by-layer forward propagation processing on the current data unit, the method further comprises: Calculating a space-time hash value of the current data unit, wherein the space-time hash value is generated based on the space characteristics of the current data unit and the time context of the previous data unit in the target data stream; The method comprises the steps of inquiring in a hardware cache, wherein aiming at a space-time hash value, if inquiring hits, directly acquiring a group of cache data prestored in the current hardware cache, rejecting the current level characteristic data as redundant data, marking a final processing result as a conventional security tag, and skipping a layer-by-layer forward propagation processing and a subsequent joint judgment step; After the forward propagation processing is performed to the nth calculation layer, calculating an information entropy H {N} of a set of current-level feature data generated by the nth calculation layer for the input data, and acquiring an information entropy H {N-1} of a set of previous-level feature data generated by the N-1 calculation layer as the input of the nth calculation layer; Based on H {N} and H {N-1} , calculating an entropy attenuation index D {N} , wherein the entropy attenuation index D {N} is used for quantifying the degree of information uncertainty reduction in the processing from the N-1 layer to the N layer; And carrying out joint judgment on the entropy attenuation index D {N} and the data similarity C, when the entropy attenuation index D {N} is lower than a preset attenuation threshold value and the data similarity C is higher than the preset similarity threshold value, interrupting the processing of a subsequent calculation layer, eliminating the current-level characteristic data as redundant data, marking a final processing result as a conventional security tag, otherwise, retaining the current-level characteristic data, and marking the final processing result as a change risk tag.
2. The AI chip-based data processing method of claim 1, wherein the step of calculating the data similarity C is specifically configured to: acquiring a first pixel matrix of a current data unit and a second pixel matrix of a previous data unit; Based on the first pixel matrix and the second pixel matrix, respectively calculating the mean value mu and the standard deviation sigma of the first pixel matrix and the second pixel matrix, and calculating the covariance sigma xy of the first pixel matrix and the second pixel matrix, so as to construct a statistical feature set containing brightness, contrast and structural information; Then, carrying out numerical stability treatment on the statistical feature set by adopting a stabilization constant injection method, and respectively carrying out formulation combination on the mean value mu, the standard deviation sigma and the covariance sigma xy and a preset stabilization constant w 1 、w 2 、w 3 to generate a brightness component, a contrast component and a structural component with numerical robustness; And finally, performing multiplicative fusion operation on the brightness component, the contrast component and the structural component subjected to stability processing, multiplying the calculation results of the three components item by item to obtain scalar output, and taking the scalar output as data similarity C representing the content consistency degree of the two data units.
3. The AI chip-based data processing method according to claim 1, wherein the neural network model is constructed by setting a network topology including a plurality of calculation layers, initializing a learnable parameter in the network topology, performing iterative training on the network topology by using a preset training data set related to a target processing task, terminating training after a performance index of the neural network model reaches a preset convergence criterion, and deploying the trained neural network model including the cured learnable parameter into an AI chip for performing subsequent layer-by-layer forward propagation processing.
4. The AI chip-based data processing method of claim 1, wherein the entropy decay index D {N} is obtained by calculating an absolute amount of entropy decay of information from layer N-1 to layer N, the absolute amount of entropy decay being equal to a difference between an information entropy H {N-1} of the previous-level feature data and an information entropy H {N} of the current-level feature data; normalizing the absolute attenuation amount of the information entropy, namely dividing the absolute attenuation amount by the information entropy H {N} of the prior-level characteristic data serving as a reference to obtain a dimensionless relative attenuation rate; and (5) assigning the relative attenuation rate obtained after normalization processing as an entropy attenuation index D {N} .
5. The AI chip-based data processing method according to claim 1, wherein a state significance coefficient S for characterizing a target data stream processing task significance level is acquired before joint judgment is made; Presetting a significance threshold, and when the state significance coefficient S is higher than the significance threshold, disregarding the judging result of the entropy attenuation index D {N} and the data similarity C, and forcibly not interrupting the processing of a subsequent calculation layer; If the state significance coefficient S is not lower than the significance threshold, judging that the processing interruption decision is false, and continuously executing the processing of the subsequent calculation layer; If the state significance coefficient S is lower than the significance threshold, further determining whether the data similarity C is higher than the similarity threshold and the entropy decay index H {N} is lower than the decay threshold, and when both are satisfied, determining that the process interruption decision is true.
6. The AI-chip-based data processing method according to claim 5, wherein the acquiring means of the state saliency coefficient S before the joint judgment is performed includes: Presetting a concerned content list, wherein the concerned content list is used for establishing a mapping relation between a content identifier and a basic significance score, and the content identifier at least comprises one or more of specific personnel, smog, access control opened abnormally or unauthorized equipment; performing a content parsing process on the target data stream to identify and extract one or more current content objects contained in the data stream; matching and inquiring the current content object and the attention content list to judge whether the current content object hits any content identifier in the list; Determining a state significance coefficient S according to the result of the matching query, wherein the output score of the state significance coefficient S is a normalized [0,1] interval: if the query hits, the basic significance score corresponding to the hit content identifier is obtained and used as a state significance coefficient S; if the query is not hit, a preset default significance score is adopted as a state significance coefficient S.
7. The AI chip-based data processing method of claim 1, wherein the generating of the spatio-temporal hash value specifically includes: calculating a spatial hash value of the current data unit by using the local sensitive hash function; and carrying out preset bit operation on the spatial hash value and the stored historical hash value corresponding to the previous data unit so as to generate a space-time hash value.
8. The AI chip-based data processing method as set forth in claim 1, wherein the query step specifically includes calculating a Hamming distance between a space-time hash value of the current data unit and each space-time hash value stored in the hardware cache, determining a query hit when any Hamming distance is lower than a predetermined distance threshold, acquiring a pre-stored calculation result corresponding to a space-time hash value with a smallest Hamming distance, determining a query miss when all Hamming distances are not lower than the predetermined distance threshold, and entering a layer-by-layer forward propagation process and a subsequent joint judgment step.
9. An AI chip-based data processing system, characterized in that the system is adapted to perform the AI chip-based data processing method of any of claims 1-8, comprising: a data stream acquisition module configured to acquire a target data stream including a plurality of data units; a data similarity calculation module configured to calculate, for a current data unit, a data similarity C between the current data unit and a previous data unit in the target data stream; The neural network processing module is configured to perform layer-by-layer forward propagation processing on the current data unit through a plurality of calculation layers in the neural network processing module; The dynamic monitoring and entropy calculating module is connected with the neural network processing module and is configured to calculate the information entropy H {N} of the current-level characteristic data generated by the N-1 layer calculating layer in real time, acquire the information entropy H {N-1} of the previous-level characteristic data generated by the N-1 layer calculating layer and calculate an entropy attenuation index D { N }; The intelligent decision and control module is respectively connected with the data similarity calculation module and the dynamic monitoring and entropy calculation module, and is configured to perform joint judgment on the entropy attenuation index D {N} and the data similarity C and generate a processing control instruction based on a judgment result; The data management module receives the processing control instruction and is configured to execute one of the following operations: when the processing control instruction is interrupt processing, interrupting the processing of a subsequent calculation layer, eliminating the current-level characteristic data as redundant data, and marking a final processing result as a conventional security tag; when the processing control instruction is to continue processing, the current-level characteristic data is reserved, the processing of a subsequent calculation layer is continued, and the final processing result is marked as a change risk label.

Description

Data processing method and system based on AI chip Technical Field The invention relates to the technical field of electric digital data processing, in particular to a data processing method and system based on an AI chip. Background With the rapid development of smart cities and ubiquitous sensor networks, video monitoring systems have become indispensable nerve endings in social security, industrial production and urban management. The Artificial Intelligence (AI) technology with the deep neural network as the core is behind the technology, and the depth and the breadth which are not available before are endowed with the capability of 'understanding' and 'understanding' for massive video data. AI chips act as a powerful cornerstone of this kind, driving a industry revolution from "clean" to "understandable". However, under this powerful feast, a great challenge is also followed. Video data streams have natural temporal-spatial continuity, which means that there is a huge amount of redundant information contained therein. Often, the monitoring camera captures a static scene or small periodic changes over a long period of time, such as a street that is open at midnight, a tree shadow that is jogged with the wind, or a silent corridor in an office building. These "silence moments" that account for a significant portion of the total amount of data constitute turbulent "digital floods". In the existing processing paradigm, a 'one-view' brute force calculation strategy is mostly adopted. Whether the picture is calm or surging, the AI chip needs to perform a complete, complex neural network forward propagation for each frame of data. This "computational resource non-adaptation" approach, which applies high-intensity depth computation to indiscriminate temporal data, results in dramatic energy waste, high storage costs, and unnecessary hardware load. On the other hand, the conventional simple methods such as motion detection can filter part of the still picture, but cannot cope with pseudo-changes such as light gradual change and shadow movement, and further cannot understand the deep semantics of the scene, so that the real and fine risk precursors and meaningless background noise are easily mixed together. Therefore, there is a need in the art for a dynamic, adaptive computing mechanism that can intelligently "brake" or "accelerate" according to the information evolution of the data content itself, thereby promoting optimization of the energy efficiency and response speed of the system while ensuring that critical information is not missed. The above information disclosed in the above background section is only for enhancement of understanding of the background of the disclosure and therefore it may include information that does not form the prior art that is already known to those of ordinary skill in the art. Disclosure of Invention The invention aims to provide a data processing method and system based on an AI chip, so as to solve the problems in the background art. The data processing method based on AI chip includes the specific steps of obtaining target data stream comprising a plurality of data units; for the current data unit, calculating the data similarity C between the current data unit and the previous data unit in the target data stream; Performing layer-by-layer forward propagation processing on the current data unit through a plurality of calculation layers of the neural network model; After the forward propagation processing is performed to the nth calculation layer, calculating an information entropy H {N} of a set of current-level feature data generated by the nth calculation layer for the input data, and acquiring an information entropy H {N-1} of a set of previous-level feature data generated by the N-1 calculation layer as the input of the nth calculation layer; Based on H {N} and H {N-1}, calculating an entropy attenuation index D {N}, wherein the entropy attenuation index D {N} is used for quantifying the degree of information uncertainty reduction in the processing from the N-1 layer to the N layer; And carrying out joint judgment on the entropy attenuation index D {N} and the data similarity C, when the entropy attenuation index D {N} is lower than a preset attenuation threshold value and the data similarity C is higher than the preset similarity threshold value, interrupting the processing of a subsequent calculation layer, eliminating the current-level characteristic data as redundant data, marking a final processing result as a conventional security tag, otherwise, retaining the current-level characteristic data, and marking the final processing result as a change risk tag. An AI chip-based data processing system for performing the AI chip-based data processing method, comprising: a data stream acquisition module configured to acquire a target data stream including a plurality of data units; a data similarity calculation module configured to calculate, for a current data unit, a dat