CN-121998582-A - Material intelligent reporting and early warning system and method based on big data

CN121998582ACN 121998582 ACN121998582 ACN 121998582ACN-121998582-A

Abstract

The invention discloses a material intelligent reporting early warning system and method based on big data, wherein the system comprises a data acquisition synchronization module, a data cleaning and preprocessing module, an algorithm calculation module, an early warning generation module and a user interaction module, wherein the data acquisition synchronization module is used for realizing multi-source data real-time synchronization by adopting a distributed architecture, the data cleaning and preprocessing module is used for completing data standardization processing through a multi-stage assembly line, the early warning generation module is used for generating a multi-dimensional early warning rule by combining a micro-service architecture and a knowledge graph dynamic reasoning engine, and the user interaction module is used for providing a visual decision support interface. The system adopts the technologies of edge calculation, asynchronous transmission, two-dimensional verification and the like to ensure the reliability of data synchronization.

Inventors

WEI JIANSHE
MAO SHIWEI
CUI QINGBO
ZHANG FANG

Assignees

天津江天智云科技有限公司

Dates

Publication Date: 20260508
Application Date: 20260107

Claims (10)

1. Big data-based intelligent material reporting and early warning system is characterized by comprising: The system comprises a data acquisition synchronization module, a data cleaning and preprocessing module, an algorithm calculation module, an early warning generation module and a user interaction module, wherein the data acquisition synchronization module is connected with the data cleaning and preprocessing module through a data line and is used for acquiring and synchronizing structured data and ERP data distributed in different systems to a large data platform in real time; The data cleaning and preprocessing module is used for cleaning and preprocessing data in the synchronization process, and comprises the steps of removing repeated records, correcting error data and filling missing values, and performing standardized processing; the algorithm calculation module is used for analyzing and modeling the cleaned data by utilizing a machine learning algorithm and generating parameter factors, the algorithm calculation module is connected with the preprocessing module through a data line and data cleaning, and a hybrid integrated learning model is integrated in the algorithm calculation module and is provided with a random forest, XGBoost and a deep neural network; The early warning generation module is used for generating early warning information according to the parameter factors and converting the early warning information into a text format through a natural language processing technology, a big data model application sub-module is arranged in the early warning generation module and is used for inputting constraint conditions of the generated text format into a big data model, calculating and analyzing the constraint conditions and generating a prediction result; The system comprises a user interaction module, an early warning generation module and a user interaction module, wherein the user interaction module is used for feeding early warning information back to a user and providing an interaction interface for the user to refer to decisions, a dynamic early warning reasoning engine of a knowledge graph is integrated in the early warning generation module, and the dynamic early warning reasoning engine of the knowledge graph is used for constructing an industry knowledge graph and correlating data of materials, supply chains and market trends.
2. The intelligent material reporting and early warning system based on big data as set forth in claim 1, wherein the data acquisition synchronization module is designed with a distributed architecture, the distributed architecture comprises data acquisition nodes and a central scheduling server, the acquisition nodes are deployed in a network area where a data source system is located and are kept connected with the central scheduling server through an asynchronous communication mechanism, the central scheduling server dynamically distributes data acquisition tasks by adopting a load balancing strategy, and a breakpoint continuous transmission mechanism is further built in the data acquisition synchronization module.
3. The intelligent material declaration early warning system based on big data according to claim 1, wherein the data cleaning and preprocessing module adopts a multi-stage pipeline processing architecture, the multi-stage pipeline processing architecture sequentially comprises a data quality detection layer, an abnormal data processing layer and a standardized conversion layer, the data quality detection layer automatically identifies null values, abnormal values and format errors in data through a rule engine and a pattern recognition technology, the abnormal data processing layer is configured with an intelligent repair strategy library, and the standardized conversion layer is internally provided with an industry standard data dictionary.
4. The intelligent material declaration early warning system based on big data according to claim 1, wherein the big data model application submodule in the early warning generation module adopts a micro-service architecture design, the micro-service architecture design comprises a model loading service, a data preprocessing service and a prediction execution service, the model loading service is used for managing deployment and hot switching of early warning models of different versions, the data preprocessing service is used for converting input parameter factors into tensor formats required by the models, and the prediction execution service is used for realizing high-concurrency prediction request processing through a distributed computing framework.
5. The intelligent material declaration and early warning system based on big data according to claim 1, wherein the dynamic early warning and reasoning engine of the knowledge graph stores industry knowledge data by adopting a graph database, a knowledge network containing multidimensional relations of material entities, provider nodes and market indexes is constructed, the dynamic early warning and reasoning engine of the knowledge graph is internally provided with a rule reasoning and semantic reasoning double engine, the rule reasoning engine is used for executing logic judgment based on business rules, and the semantic reasoning engine is used for mining potential association rules through a graph neural network.
6. The intelligent material reporting and early warning system based on big data as set forth in claim 1, wherein the user interaction module adopts a responsive design, and an interaction interface of the user interaction module comprises an early warning information dashboard, a decision-making auxiliary tool and a feedback collection component, wherein the early warning information dashboard dynamically displays early warning level distribution and trend change through a visual chart, the decision-making auxiliary tool is used for providing historical case query and treatment suggestion generation functions, and the feedback collection component is used for recording treatment comments of a user on early warning information.
7. The intelligent material reporting and early warning method based on big data is based on the intelligent material reporting and early warning system based on big data as set forth in any one of claims 1 to 6, and is characterized by comprising the following steps: S1, data acquisition and synchronization: The method comprises the steps of collecting structured data and ERP data scattered in different systems in real time through a distributed architecture, and synchronizing the structured data and ERP data to a big data platform; S2, intelligent data cleaning and pretreatment: the synchronous data is cleaned through a multi-stage assembly line, null values, abnormal values and format errors in the data are detected through a rule engine and a pattern recognition technology, then an intelligent repair strategy library is called to automatically repair the abnormal data, and finally data standardized conversion is completed through a built-in industry standard data dictionary; s3, analyzing and calculating a hybrid integrated model: Adopting a hybrid integrated learning model comprising a random forest, XGBoost and a deep neural network, carrying out multidimensional analysis on the data preprocessed in the step S2, wherein the random forest is responsible for feature primary screening, XGBoost carries out feature optimization weighting, the deep neural network processes unstructured data features, and finally, a comprehensive parameter factor comprising key indexes of stock state and consumption trend is generated through a weighted voting mechanism; s4, intelligent early warning generation with knowledge enhancement: Inputting the parameter factors generated in the step S3 into a big data model application submodule based on a micro-service architecture, converting a prediction result into early warning information in a text format through a natural language processing technology, and simultaneously, associating materials, a supply chain and market trend data based on a knowledge graph dynamic reasoning engine to generate a multidimensional early warning rule; s5, interactive early warning decision support: And displaying the early warning information instrument panel, the decision-making auxiliary tool and the feedback collection component through a responsive interactive interface, dynamically visualizing early warning level distribution, providing historical case inquiry and treatment suggestions, and recording user treatment suggestions to optimize the model.
8. The intelligent material reporting and early warning method based on big data as set forth in claim 7, wherein in the step S1, in the data acquisition and synchronization process, an edge computing node is adopted to construct a distributed acquisition network, a lightweight data agent module is deployed at an outlet of a service system to realize non-invasive data acquisition, an asynchronous transmission buffer is realized by combining a message queue, and a two-dimensional data synchronous verification mechanism is constructed based on a time stamp and a service serial number.
9. The intelligent material declaration and early warning method based on big data according to claim 7, wherein the multistage pipeline in the step S2 adopts a dynamic extensible architecture design, and comprises three parallel processing channels, wherein a first channel in the parallel processing channels is configured with a rule engine based on reinforcement learning, a decision tree is constructed, null types are automatically identified, filling strategies are generated, a convolutional neural network is integrated in a second channel in the parallel processing channels, an abnormal expression mode in a text field is positioned by adopting an attention mechanism, an countermeasure generation network is deployed in a third channel in the parallel processing channels, and synthetic data samples conforming to business rules are generated by generating countermeasure training.
10. The intelligent material declaration and early warning method based on big data as set forth in claim 7, wherein the random forest model in the step S3 adopts a dynamic feature importance assessment mechanism, the feature weight is updated every iteration period through a replacement importance algorithm, core business indexes of inventory turnover rate and supplier delivery time rate are screened out in a key mode, the XGBoost is constructed by introducing SHAP value explanatory constraint, a special time sequence processing branch is set in a deep neural network, and a multi-head attention mechanism of a Transformer architecture is adopted to analyze unstructured data of purchase contract text and logistics track.

Description

Material intelligent reporting and early warning system and method based on big data Technical Field The invention relates to the field of enterprise material reporting management application, in particular to a material intelligent reporting early warning system and method based on big data. Background The material reporting early warning system belongs to an enterprise intelligent manufacturing and supply chain management flow, and because the production flow of a steel enterprise is complex, a plurality of links such as raw material purchasing, production scheduling, inventory management and the like are involved, and the traditional reporting mode relies on manual experience judgment, so that the system is difficult to adapt to the requirements of dynamic market change and fine management. How to efficiently integrate scattered data and realize intelligent early warning becomes a key problem for improving enterprise reporting accuracy and decision-making efficiency. The common material reporting system of the iron and steel enterprises has a structural technical bottleneck generally that data of each business link are stored in a production execution system, a warehouse management system and a supply chain platform in a scattered manner, and a unified data acquisition and synchronization mechanism is lacked, so that information required by reporting decisions presents fragmentation characteristics. The missing or insufficient processing capacity of the data cleaning and preprocessing links causes the problems of repeated data, abnormal values, format differences and the like to exist for a long time, and seriously weakens the reliability of the basic data. In the analysis modeling layer, complex association relation between material demand and supply chain fluctuation cannot be effectively captured by relying on manual experience or a reporting threshold setting mode of a single algorithm, and reporting rationality is difficult to verify due to the lack of quantitative evaluation means. The existing early warning mechanism mostly adopts a post-event statistical mode, so that prospective identification of risks cannot be realized, an interpretable decision basis is difficult to provide, reporting abnormality is delayed, the correction cost is high, the working requirements of enterprise material reporting management application cannot be met, and therefore, the intelligent material reporting early warning system and method based on big data are provided. Disclosure of Invention The invention provides a technical scheme that the intelligent material reporting and early warning system based on big data comprises: The system comprises a data acquisition synchronization module, a data cleaning and preprocessing module, an algorithm calculation module, an early warning generation module and a user interaction module, wherein the data acquisition synchronization module is connected with the data cleaning and preprocessing module through a data line and is used for acquiring and synchronizing structured data and ERP data distributed in different systems to a large data platform in real time; The data cleaning and preprocessing module is used for cleaning and preprocessing data in the synchronization process, and comprises the steps of removing repeated records, correcting error data and filling missing values, and performing standardized processing; the algorithm calculation module is used for analyzing and modeling the cleaned data by utilizing a machine learning algorithm and generating parameter factors, the algorithm calculation module is connected with the preprocessing module through a data line and data cleaning, and a hybrid integrated learning model is integrated in the algorithm calculation module and is provided with a random forest, XGBoost and a deep neural network; The early warning generation module is used for generating early warning information according to the parameter factors and converting the early warning information into a text format through a natural language processing technology, a big data model application sub-module is arranged in the early warning generation module and is used for inputting constraint conditions of the generated text format into a big data model, calculating and analyzing the constraint conditions and generating a prediction result; The system comprises a user interaction module, an early warning generation module and a user interaction module, wherein the user interaction module is used for feeding early warning information back to a user and providing an interaction interface for the user to refer to decisions, a dynamic early warning reasoning engine of a knowledge graph is integrated in the early warning generation module, and the dynamic early warning reasoning engine of the knowledge graph is used for constructing an industry knowledge graph and correlating data of materials, supply chains and market trends. The invention provides a material intelligent re