CN-121998699-A - Mall data analysis system

CN121998699ACN 121998699 ACN121998699 ACN 121998699ACN-121998699-A

Abstract

The invention discloses a mall data analysis system, which relates to the technical field of electronic commerce data processing, and comprises a data access and preprocessing module, a flow batch integrated calculation engine, a dynamic mode identification and prediction module, an intelligent decision support module and a visual interaction platform, wherein the flow batch integrated calculation and increment learning mechanism is used for realizing high throughput and low delay data processing and dynamic mode identification, and the real-time performance, the self-adaption capability and the decision efficiency of the mall data analysis are improved by combining a multi-objective optimization and asynchronous rendering technology. The invention aims to solve the problems of insufficient real-time performance of data processing, high calculation delay and poor adaptability to dynamic user behaviors.

Inventors

CUI YONGZHI
SHEN XIAO
ZHOU LIANG
WANG QIANG
YE TIAN
YANG DEHAO

Assignees

微点(杭州)网络科技有限公司

Dates

Publication Date: 20260508
Application Date: 20260121

Claims (10)

1. A mall data analysis system, comprising: The data access and preprocessing module is used for interfacing multi-source heterogeneous data input, performing data cleaning, format standardization and preliminary feature extraction operations, and a data quality check unit and a real-time data distribution channel are arranged in the data access and preprocessing module; The flow batch integrated computing engine adopts a unified architecture to simultaneously support real-time flow data processing and offline batch computing tasks, and comprises a flow processing sub-engine and a batch processing sub-engine which share the same set of memory resource management and task scheduling strategies and keep the consistency of computing contexts through a state synchronization mechanism; The dynamic mode recognition and prediction module is constructed based on an incremental learning algorithm, continuously receives output data from a flow batch integrated calculation engine, automatically detects mutation of a user behavior mode and abnormal sales trend, and generates a short-term prediction index; the intelligent decision support module receives the analysis result of the dynamic mode identification and prediction module, combines a preset business rule and an optimization target, and generates an inventory allocation suggestion, a marketing strategy adjustment scheme and a risk early warning signal; The visual interaction platform provides data display and query functions based on a Web graphical interface, supports multi-dimensional data drilling, real-time dashboard refreshing and interactive report generation, and a front-end rendering engine adopts an asynchronous loading and data slicing technology.
2. The mall data analysis system of claim 1, wherein the data access and pre-processing module receives input data streams from the user terminal, the transaction server, the logistics database and the third party data interface through the distributed message middleware; the data cleaning unit automatically identifies and filters missing fields, repeated records and format error data by using a rule engine and an abnormal value detection algorithm, the format standardization unit converts the heterogeneous data into a unified JSON-LD structured format, and adds a time stamp and a data source identifier to each data record, and the preliminary feature extraction unit extracts key attributes from original data based on a preset feature template, wherein the key attributes comprise user session duration, commodity click sequence, transaction amount distribution and geographic position information.
3. The mall data analysis system according to claim 2, wherein the data quality verification unit dynamically evaluates the input data quality by calculating data integrity rate, consistency index and timeliness score, and triggers data retransmission or alarm flow, and the real-time data distribution channel adopts a publish-subscribe mode to push the preprocessed data to the input buffer area of the stream batch integrated calculation engine in parallel.
4. The mall data analysis system according to claim 1, wherein the stream processing sub-engine processes real-time data stream based on event time semantics, and internally implements a sliding window aggregation operator and a state snapshot mechanism, and can complete user behavior funnel analysis and real-time transaction statistics in any time window, and the batch processing sub-engine periodically performs full-scale data calculation tasks including historical sales trend modeling, user portrait update and inventory turnover rate analysis.
5. The mall data analysis system according to claim 4, wherein the memory resource management unit adopts a dynamic partition allocation strategy to automatically adjust allocation proportion and recovery threshold of memory blocks according to real-time load conditions of stream processing and batch processing tasks, and the task scheduling strategy allocates more computing resources for the tasks with high real-time performance based on a priority queue and a resource estimation model, and simultaneously ensures that progress of batch tasks is not affected.
6. The mall data analysis system of claim 5, wherein the state synchronization mechanism records intermediate states of the stream processing sub-engines through the distributed transaction log and resumes the consistency view at start-up of the batch processing sub-engines.
7. The mall data analysis system according to claim 1, wherein the incremental learning algorithm employs an online gradient descent optimization method, updates model weights each time a batch of new data is received without retraining a full amount of historical data, the model drift detection unit automatically identifies a model transition point of a user behavior or sales trend by monitoring a statistical distribution change of a prediction error sequence, and when a significant drift is detected, the unit transmits a model reconstruction instruction to the adaptive model update unit.
8. The mall data analysis system according to claim 7, wherein the adaptive model updating unit selects a local parameter fine adjustment or global structure reconstruction strategy according to the magnitude and direction of the pattern drift, adjusts only the weight parameters of the model output layer for the slight drift, triggers feature selection reorganization and hidden layer node deletion operation for the severe drift, and the short-term prediction index comprises sales volume estimation, popular commodity ranking change probability and abnormal transaction risk index for 24 hours in the future, and is refreshed every 5 minutes and pushed to the intelligent decision support module.
9. The mall data analysis system of claim 1, wherein the multi-objective optimization solver constructs a constrained mathematical programming model with parallel objectives of maximizing sales, minimizing inventory backlog, and optimizing customer satisfaction; The solver outputs a pareto optimal solution set in millisecond time by adopting a genetic algorithm and linear programming mixed solving strategy, the inventory allocation proposal calculates allocation quantity and replenishment time among all warehouses based on real-time sales prediction and current inventory level, the marketing strategy adjustment scheme dynamically generates a personalized coupon putting strategy and advertisement space content updating instruction according to user behavior mode change, and the risk early warning signal triggers a manual auditing process or automatically freezes suspicious account operation according to the detected abnormal transaction mode.
10. The mall data analysis system of claim 1, wherein the Web graphical interface of the visual interaction platform is developed based on a componentized architecture, supports drag layout customization and theme style switching, the data drill function allows a user to drill down from a summary view to detail data layer by layer, the drill path is automatically constructed based on a dimension hierarchical relationship, the real-time instrument panel is kept synchronous with a back-end data service through WebSocket long connection, the interactive report generator provides various chart templates and filtering conditions, and the user rapidly generates a customized report through checking dimensions and index items.

Description

Mall data analysis system Technical Field The invention relates to the technical field of electronic commerce data processing, in particular to a mall data analysis system. Background In the technical field of electronic commerce, the importance of a mall data analysis system as a core tool for supporting business decisions and optimizing operation efficiency is increasingly highlighted. Such systems typically involve the collection, storage, processing and analysis of massive amounts of user behavior data, transaction data and merchandise data to mine potential commercial value and market trends. The key task of the mall data analysis system is to convert the original data into insight information for decision reference through an efficient data processing flow. The basic aim is to assist the manager to carry out accurate marketing, inventory optimization and service improvement through monitoring and analysis of multidimensional indexes such as sales trend, user preference, inventory condition and the like. In the prior art, a mall data analysis system generally adopts a traditional data warehouse and batch processing computing framework, and is difficult to meet the requirement of real-time or near real-time data analysis. When the system processes high-concurrency multi-source heterogeneous data, the system often faces the problems of low data integration efficiency and high calculation delay, and the analysis result is delayed from service change. In addition, existing analytical models have insufficient adaptability to dynamically changing patterns of user behavior, are difficult to quickly identify sudden sales trends or abnormal transaction behaviors, and are prone to response bottlenecks during promotional campaigns or traffic peaks. Meanwhile, the performance of the data visualization and interactive query functions is limited, and the decision-making efficiency and user experience of management staff are affected. Disclosure of Invention The invention aims to provide a mall data analysis system which is used for solving the problems of insufficient real-time performance, high calculation delay, poor adaptability to dynamic user behavior modes and obvious response bottleneck during traffic peaks in the prior art. In order to solve the technical problems, the invention provides the following technical scheme: the system comprises a data access and preprocessing module, a flow batch integrated calculation engine, a dynamic mode identification and prediction module, an intelligent decision support module and a visual interaction platform. The data access and preprocessing module is responsible for interfacing multi-source heterogeneous data input, performing data cleaning, format standardization and preliminary feature extraction operations, and is internally provided with a data quality check unit and a real-time data distribution channel, so that the original data is ensured to finish standardization processing within millisecond delay and is transmitted to a downstream computing node. The integrated flow-batch computing engine adopts a unified architecture to simultaneously support real-time flow data processing and offline batch computing tasks, and comprises a flow processing sub-engine and a batch processing sub-engine which share the same set of memory resource management and task scheduling strategies and keep consistency of computing context through a state synchronization mechanism, so that high throughput and low-delay data processing capacity are realized. The dynamic mode recognition and prediction module is constructed based on an incremental learning algorithm, continuously receives output data from a flow batch integrated calculation engine, automatically detects mutation of a user behavior mode and abnormal sales trend, generates a short-term prediction index, and is integrated with a mode drift detection unit and a self-adaptive model updating unit, so that parameters and structures of a recognition model can be dynamically adjusted under the condition of no service interruption. The intelligent decision support module receives the analysis result of the dynamic mode identification and prediction module, combines a preset business rule and an optimization target to generate an inventory allocation suggestion, a marketing strategy adjustment scheme and a risk early warning signal, and is internally provided with a multi-target optimization solver for solving the optimal decision combination under the condition of resource constraint. The visual interaction platform provides data display and query functions based on a Web graphical interface, supports multi-dimensional data drilling, real-time dashboard refreshing and interactive report generation, and ensures smooth user experience under a high concurrent access scene by adopting an asynchronous loading and data slicing technology through a front-end rendering engine. The data access and preprocessing module receives input data strea