Search

CN-121996975-A - Intelligent data analysis system and method based on artificial intelligence

CN121996975ACN 121996975 ACN121996975 ACN 121996975ACN-121996975-A

Abstract

The invention discloses an intelligent data analysis system and method based on artificial intelligence, and relates to the technical field of artificial intelligence and data processing. The system comprises a data acquisition module, a data preprocessing module, a characteristic engineering module, a model training optimization module, a result evaluation deployment module and a visual interaction module. The realization method sequentially carries out data source selection and data collection arrangement, data preprocessing, data characteristic engineering, model selection training optimization, result evaluation and deployment, and presents analysis results through a visualization technology. The invention solves the problems of low data quality, poor algorithm selection adaptability, insufficient computing resources and the like in the existing data analysis process, can efficiently process large-scale data, accurately excavates data rules and trends, provides reliable support for decision making of various industries, and can be widely applied to the fields of electronic commerce, finance, medical treatment, transportation and the like.

Inventors

  • GE DINGJIA

Assignees

  • 上海精鲲计算机科技有限公司

Dates

Publication Date
20260508
Application Date
20260130

Claims (10)

  1. 1. The intelligent data analysis system based on the artificial intelligence is characterized by comprising a data acquisition module, a data preprocessing module, a characteristic engineering module, a model training optimization module, a result evaluation deployment module and a visual interaction module; The data preprocessing module is used for processing the acquired data so as to improve the data quality; The feature engineering module is used for carrying out feature related operation on the preprocessed data, the model training optimization module is used for selecting a proper model and carrying out training and parameter adjustment, the result evaluation deployment module is used for evaluating the performance of the model and making a deployment scheme and carrying out monitoring optimization, and the visual interaction module is used for displaying data analysis results in various forms and supporting user interaction operation.
  2. 2. The intelligent data analysis system based on artificial intelligence of claim 1, wherein the data sources acquired by the data acquisition module comprise at least one of databases, API interfaces, social media platforms, sensor devices, web crawlers acquired network data.
  3. 3. The intelligent data analysis system according to claim 1, wherein the processing operations of the data preprocessing module include data cleansing for removing duplicate, invalid and erroneous data, data conversion for converting data from one format or structure to another, and data normalization for scaling data to a uniform range of values.
  4. 4. The intelligent data analysis system based on artificial intelligence according to claim 1, wherein the feature related operations of the feature engineering module comprise feature extraction, feature selection, feature conversion, feature dimension reduction, wherein the feature extraction is used for extracting features related to analysis targets from raw data.
  5. 5. The intelligent data analysis system based on artificial intelligence according to claim 1, wherein the model selected by the model training optimization module comprises a machine learning model and a deep learning model, and the machine learning model comprises a classification algorithm model, a clustering algorithm model and a regression algorithm model.
  6. 6. The intelligent data analysis system based on artificial intelligence according to claim 1, wherein the model performance evaluation of the result evaluation deployment module is performed by using a test data set, the deployment scheme comprises a hardware configuration scheme and a software environment configuration scheme, and the monitoring optimization is used for monitoring the running state of the deployed model in real time and adjusting the model parameters according to the monitoring result.
  7. 7. The intelligent data analysis system based on artificial intelligence according to claim 1, wherein the presentation form of the visual interaction module comprises a chart, a visual dashboard, an interactive visual interface, a visual large screen.
  8. 8. The method for artificial intelligence based intelligent data analysis according to any one of claims 1 to 7, comprising the steps of: s1, selecting a data source and collecting and sorting data, determining the data source and collecting data, and primarily sorting the collected data; S2, data preprocessing, namely performing data cleaning, data conversion and data normalization on the data after finishing; s3, carrying out data feature engineering, namely carrying out feature extraction, feature selection, feature conversion and feature dimension reduction on the preprocessed data; S4, selecting, training and optimizing a model, selecting a proper model according to data characteristics and service requirements, training the model by using a training data set, and adjusting model parameters by a cross verification technology; S5, evaluating and deploying results, evaluating the performance of the model by using a test data set, making a model deployment scheme, deploying, and monitoring and optimizing the deployed model in real time; And S6, visually displaying the data analysis result, displaying the data analysis result in various forms through a visual interaction module and supporting user interaction.
  9. 9. The method of intelligent data analysis based on artificial intelligence of claim 8, wherein in step S1, the data collection and sorting further comprises sorting the collected data.
  10. 10. The method according to claim 9, wherein in step S5, the monitoring and optimizing after the model deployment includes periodically collecting model operation data, analyzing model performance variation trend, and re-performing model training or parameter adjustment when the model performance degradation exceeds a preset threshold.

Description

Intelligent data analysis system and method based on artificial intelligence Technical Field The invention relates to the technical field of artificial intelligence and data processing, in particular to an intelligent data analysis system and method based on artificial intelligence. Background With the rapid development of information technology, the data volume generated by each industry is increased in an explosive manner, and how to mine valuable information from massive data provides support for decision making, so that the method becomes a current problem to be solved urgently. The traditional data analysis method often depends on manual operation, has low processing efficiency, is difficult to deal with large-scale data, and has the accuracy and reliability of analysis results greatly influenced by artificial factors. Although some data analysis techniques have been introduced into computer technology, there are still many problems in practical application. In terms of data quality, the acquired data often contains a large amount of repeated, invalid and error information, if effective processing is not performed, the deviation of a subsequent analysis result is caused, in terms of algorithm selection, the prior art lacks scientific analysis on data characteristics and algorithm suitability, the most suitable algorithm is difficult to select for data analysis, the analysis effect is influenced, in terms of calculation resources, large-scale data processing needs to consume a large amount of calculation resources, the prior art has the defect of low calculation resource utilization efficiency, and the condition that the analysis speed is slow or even the analysis cannot be completed easily occurs. In addition, the display form of the existing data analysis results is single, so that the user is difficult to intuitively and conveniently acquire key information, and is also incapable of performing interactive exploration, and the value of the data analysis results is not fully exerted. Accordingly, one skilled in the art would be able to provide intelligent data analysis systems and methods based on artificial intelligence to solve the problems set forth in the background above. Disclosure of Invention The invention provides an intelligent data analysis system based on artificial intelligence, which comprises a data acquisition module, a data preprocessing module, a characteristic engineering module, a model training optimization module, a result evaluation deployment module and a visual interaction module, wherein the data acquisition module is used for acquiring data from different sources, the data preprocessing module is used for processing the acquired data to improve the data quality, the characteristic engineering module is used for carrying out characteristic related operation on the preprocessed data, the model training optimization module is used for selecting a proper model and carrying out training and parameter adjustment, the result evaluation deployment module is used for evaluating the performance of the model and making a deployment scheme and carrying out monitoring optimization, and the visual interaction module is used for displaying data analysis results in various forms and supporting user interaction operation. Preferably, the data source acquired by the data acquisition module comprises at least one of a database, an API interface, a social media platform, sensor equipment and network data acquired by a webpage crawler. Preferably, the processing operation of the data preprocessing module comprises data cleaning, data conversion and data normalization, wherein the data cleaning is used for removing repeated data, invalid data and error data, the data conversion is used for converting data from one format or structure to another format or structure, and the data normalization is used for scaling the data to a uniform numerical range. Preferably, the feature related operation of the feature engineering module comprises feature extraction, feature selection, feature conversion and feature dimension reduction, wherein the feature extraction is used for extracting features related to an analysis target from original data. Preferably, the model training optimization module selects a model comprising a machine learning model and a deep learning model, and the machine learning model comprises a classification algorithm model, a clustering algorithm model and a regression algorithm model. The model performance evaluation of the result evaluation deployment module is preferably carried out by adopting a test data set, the deployment scheme comprises a hardware configuration scheme and a software environment configuration scheme, and the monitoring optimization is used for monitoring the running state of the deployed model in real time and adjusting model parameters according to the monitoring result. Preferably, the display form of the visual interaction module comprises a chart, a visual i