CN-121680967-B - Tree structure big data loading method and device based on double-engine cooperation

CN121680967BCN 121680967 BCN121680967 BCN 121680967BCN-121680967-B

Abstract

The invention relates to a tree structure big data loading method and device based on double-engine cooperation, and belongs to the field of computers. The method comprises the steps of preprocessing original tree data, initializing and starting monitoring, enabling a prediction preloading engine to complete interaction and prediction based on an operation probability model, enabling a progressive rendering engine to complete frame-division rendering based on a first rendering load balancing model, enabling double engines to feed back each other, and adjusting an unfolding probability threshold value and a rendering load threshold value. The device comprises a data preprocessing module, an initializing and monitoring module, an interaction and prediction module, a framing rendering module and a feedback adjusting module. According to the invention, the data loading flow of the tree structure is reconstructed through the closed loop cooperative architecture of the prediction preloading engine and the progressive rendering engine, so that the data loading flow is dynamically adapted to the rendering performance requirement, and the interaction fluency of the node scale above hundred thousand levels is satisfied.

Inventors

XIE ZHUOWEI
WU XIAN
LUO KAI
JIANG YONGJIE
CHEN YUNFENG
LIU YIFAN

Assignees

成都云祺科技有限公司

Dates

Publication Date: 20260505
Application Date: 20260211

Claims (10)

1. The method for loading the big data of the tree structure based on the cooperation of the double engines is characterized by comprising a prediction preloading engine and a progressive rendering engine which run in parallel and are resident, and comprises the following steps: receiving original tree data, constructing a flattened index, and loading configuration parameters; creating a rendering container, starting state monitoring, instantiating a double engine and establishing a shared parameter channel; Monitoring rolling behavior, and predicting a preloading engine to calculate and share the unfolding probability of each node based on an operation probability model by combining with each frame of rendering time shared by a progressive rendering engine, and judging whether to trigger asynchronous loading, wherein the operation probability model is as follows: Wherein P load is the expansion probability of each node, sigma and beta are weight coefficients, alpha is a reference constant term, In order for the rolling acceleration to be the same, In order to achieve a scrolling speed, For the minimum distance of a node from the edge of the viewable area, For the total height of the scrolling container, γ (time_1) is a rendering capability adaptation coefficient based on the rendering Time per frame, and time_1 is the rendering Time per frame in the standard case; Responding to the unfolding operation instruction, the progressive rendering engine reads the unrendered sub-node list, calculates and shares the rendering time length of each frame and the number of the rendering nodes of each frame based on a first rendering load balancing model, and then executes framing rendering, wherein the first rendering load balancing model is as follows: Where Time _1 is the rendering Time per frame under standard conditions, The time is estimated for the total rendering and, For the total number of rendering frames, For the current node depth, For the maximum depth of the tree, For the current CPU availability rate, As a coefficient of the complexity of the node, Is a rolling state coefficient; The prediction preloading engine and the progressive rendering engine feed back each other through a shared parameter channel, and the unfolding probability threshold and the rendering load threshold are adjusted.
2. The dual-engine collaboration-based tree structure big data loading method according to claim 1, wherein the steps of receiving the original tree data, constructing a flattened index, and loading configuration parameters further comprise: constructing a collaborative state pool in a memory, wherein the collaborative state pool space is logically divided into a shared data access layer and a global configuration space; receiving original tree data, converting the original tree data into node data with a unified structure through a data standardization function, storing the unified node data into a shared data access layer for dual-engine real-time operation and updating, wherein each node at least comprises a node ID field, a father node ID field, a display tag field, a node depth field, a node type field, a child node field, an expanded field and a rendered field; Constructing a flattened node index table according to the original tree data, wherein the flattened node index table comprises a parent-child node mapping hash table and a node searching hash table, and further records unique identification, depth, parent node reference and visibility state of each node; And receiving a configuration parameter set transmitted by the service layer, wherein the configuration parameter set is loaded into the global configuration space for the double engine to execute read-only operation at one time when the system is initialized, and at least comprises a tree-shaped maximum depth threshold value, single-node base rendering time consumption and a node complexity mapping table.
3. The dual engine collaboration-based tree structured big data loading method of claim 1 or 2, wherein the steps of creating a rendering container, starting state monitoring, instantiating dual engines and establishing a shared parameter channel further comprise: Creating a tree structure rolling container with a fixed viewport height and a fixed rolling pattern, rendering a root node to the container, and building a mapping index table of a node ID and DOM elements in a memory for reading node position information by a double engine; The instantiation rolling history manager monitors the rolling event and records displacement, time sequence and direction to the annular buffer zone; creating a visual state monitor instance, binding root node parameters of the visual state monitor instance to a tree-shaped rolling container, and configuring a multi-threshold monitoring node visual state change; binding node click events on the tree-shaped rolling container through an event entrusting mechanism, and capturing user unfolding/folding operation; Starting a progressive rendering engine and a predictive preloading engine in parallel, respectively reading a rendering reference parameter and a performance target parameter in shared configuration, and initializing a queue to be rendered and a preloading task queue; And establishing a memory communication channel, and completing state synchronization and instruction transmission by the double engines through reading and writing real-time load parameters, task queues and node state indexes in the shared state object.
4. The method for loading big data based on a tree structure with cooperation of two engines according to claim 1 or 2, wherein the steps of monitoring rolling behavior, predicting the preloading engine, calculating and sharing the unfolding probability of each node by combining the rendering time of each frame shared by the progressive rendering engine based on an operation probability model, and judging whether to trigger asynchronous loading further comprise: When the rolling event is triggered, executing a rolling event callback, recording a rolling position and a time stamp by a rolling history manager, calculating the current rolling speed and acceleration, and simultaneously updating a visible node list in real time by a visible state monitor; The prediction preloading engine calculates the unfolding probability of each node based on the operation probability model by combining with reading the rendering time of each frame output by the progressive rendering engine; Judging whether the expansion probability exceeds a preset expansion probability critical value and whether the target node is within the preloading distance according to the expansion probability result, and if so, triggering the node data which are not loaded near the asynchronous loading visible area; and if not, releasing all the logic invisible node data and resources, wherein the logic invisible node comprises nodes beyond the visible area and child nodes folded and invisible in the visible area.
5. The method for loading big data of tree structure based on the cooperation of two engines according to claim 2, wherein the step of responding to the unfolding operation instruction, the progressive rendering engine reads the unrendered sub-node list, calculates and shares the rendering time length and the number of rendering nodes per frame based on the first rendering load balancing model, and then performs the frame-by-frame rendering further comprises: responding to the unfolding operation instruction, and reading an unrendered child node list by the progressive rendering engine and simultaneously reading sharing state parameters; calculating the rendering time length of each frame and the number of rendering nodes of each frame corresponding to the current rendering batch based on the first rendering load balancing model; writing rendering time length of each frame into a shared data access layer for a prediction preloading engine to read; arranging rendering tasks through a frame scheduling interface, traversing unrendered child nodes according to the number of rendering nodes of each frame, executing a node rendering function to create DOM elements, binding rolling events and adding the DOM elements to a parent node container of the DOM elements, and updating a node rendering state mark; And the progressive rendering engine submits the node tasks to be rendered to the callback function of the request animation frame in batches, detects the number of the remaining nodes after each frame of rendering is completed, and continues to submit the next batch if the number of the remaining nodes is greater than zero until all the tasks are processed.
6. The dual-engine collaboration-based tree structure big data loading method according to claim 5, wherein the predicting preloading engine and the progressive rendering engine feed back to each other through a shared parameter channel, and the step of adjusting the deployment probability threshold and the rendering load threshold further comprises: after each frame of the progressive rendering engine is rendered, the rendering performance index is written into the shared data access layer, the closed-loop coupler reads the rendering performance index in real time and compares the rendering performance index with a preset rendering load critical value, and if the rendering performance index is judged to be lower than the preset rendering load critical value, a negative regulation signal is broadcast to the prediction preloading engine through the shared parameter channel, and the unfolding probability threshold value is automatically adjusted downwards; When the preloading engine is predicted to execute an asynchronous loading task, the preloading progress is written into the shared data access layer, and when the closed-loop coupler monitors that the rendering load threshold has a recovery allowance and the preloading progress is lagged, a forward regulating signal is sent to the progressive rendering engine, so that the rendering engine is allowed to moderately lift the rendering load threshold.
7. The method for loading big data of tree structure based on the cooperation of two engines according to claim 1, wherein in the step of responding to the unfolding operation instruction, the progressive rendering engine reads the list of unrendered child nodes, calculates and shares the rendering time length and the rendering node number of each frame based on a first rendering load balancing model, and then executes the frame-division rendering, the method further comprises the steps of replacing the first rendering load balancing model with a second rendering load balancing model, calculating the rendering time length and the rendering node number of each frame corresponding to the current rendering batch, and increasing the calculation factor window density based on the first rendering load balancing model by the second rendering load balancing model The second rendering load balancing model specifically comprises the following steps: Where Time_2 is the rendering Time per frame at the current window density, , The number of nodes that have been rendered for the current viewable area, Is the window height.
8. A dual-engine collaboration-based tree-structured big data loading device, characterized in that a predictive preloading engine and a progressive rendering engine running in parallel and residing are provided, the device comprising: the data preprocessing module is used for receiving the original tree data, constructing a flattened index and loading configuration parameters; The initialization and monitoring module is used for creating a rendering container, starting state monitoring, instantiating a double engine and establishing a shared parameter channel; The interaction and prediction module is used for monitoring the rolling behavior, predicting the pre-loading engine to calculate and share the unfolding probability of each node by combining the rendering time of each frame shared by the progressive rendering engine based on the operation probability model, and judging whether to trigger asynchronous loading, wherein the operation probability model is as follows: Wherein P load is the expansion probability of each node, sigma and beta are weight coefficients, alpha is a reference constant term, In order for the rolling acceleration to be the same, In order to achieve a scrolling speed, For the minimum distance of a node from the edge of the viewable area, For the total height of the scrolling container, γ (time_1) is a rendering capability adaptation coefficient based on the rendering Time per frame, and time_1 is the rendering Time per frame in the standard case; the frame-dividing rendering module is used for responding to the unfolding operation instruction, the progressive rendering engine reads the unrendered sub-node list, calculates and shares the rendering time length of each frame and the number of the rendering nodes of each frame based on a first rendering load balancing model, and then executes frame-dividing rendering, wherein the first rendering load balancing model is as follows: Where Time _1 is the rendering Time per frame under standard conditions, The time is estimated for the total rendering and, For the total number of rendering frames, For the current node depth, For the maximum depth of the tree, For the current CPU availability rate, As a coefficient of the complexity of the node, Is a rolling state coefficient; and the feedback adjusting module is used for predicting mutual feedback of the preloading engine and the progressive rendering engine through the shared parameter channel and adjusting the unfolding probability threshold and the rendering load threshold.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the dual engine collaboration based tree structured big data loading method as claimed in any of claims 1 to 7 when executing the program.
10. A computer-readable storage medium, on which a computer program is stored, characterized in that the program, when executed by a processor, implements the dual-engine collaboration-based tree-structured big data loading method according to any of claims 1 to 7.

Description

Tree structure big data loading method and device based on double-engine cooperation Technical Field The invention belongs to the field of computers, and relates to a tree structure big data loading method and device based on double-engine cooperation, computer equipment and a computer storage medium. Background The Tree Structure (Tree Structure) is used as the most basic data organization form in the application system, realizes hierarchical expression and hierarchical navigation of information through the nesting relationship of father and son nodes, and is widely applied to core business scenes such as authority management, organization architecture, file system and the like. The large data loading is to realize the full-flow processing of dynamic request, memory management, visual rendering and interactive response of node data by indicating that a single tree instance contains super-large scale data sets of 10-100 ten thousand magnitude nodes. Along with the deep advancement of enterprise-level digital transformation, the node scale of a single tree structure is evolving from the traditional thousand level to the hundred thousand level or even millions of levels, and subversion challenges are provided for rendering performance, interaction smoothness and resource consumption during front-end operation. To alleviate the rendering pressure of large-scale tree data, the current mainstream front-end framework commonly adopts virtual scrolling (Virtual Scrolling) technology as a standard solution. It is to be understood objectively that the virtual scrolling technique is not a single rendering means, but is a comprehensive optimization established on a specific data loading mode, namely, after a browser monitors a scrolling event, calculating the coordinate range of the current visible area, requesting a corresponding data subset according to the requirement, and finally rendering the returned data as DOM nodes. The process works well in a uniform-speed rolling scene of the linear list, and can realize the paging loading and smooth rolling of the foundation. However, when this mechanism is applied to the expansion/contraction operation of the tree structure, the inherent workflow exposes the following performance drawbacks: 1. Rendering and cutting are only carried out on nodes in a visible area, but when a user expands deep nodes, DOM tree depth suddenly increases to forcedly trigger a complete rendering pipeline of a browser from layout calculation to style redrawing to layer synthesis, so that a main thread is blocked for a long time, and the blocking is obvious; 2. before loading, the visibility is judged by synchronously traversing the full quantity of nodes, and the time consumption of single calculation in 10-ten thousand-level node scenes can far exceed the fluency threshold of the browser, so that the rolling frame rate dip and interaction delay are directly caused; 3. The virtual scrolling adopts a 'first request and then rendering' time sequence, a user needs to wait for the network to return to render after clicking the unfolding icon, and an obvious loading blank period exists in the time sequence, so that operation feedback delay and visual incoherence are caused, and the harsh requirements of enterprise-level applications on smooth interaction are difficult to meet. In summary, the performance bottleneck of the prior art under the tree structure is essentially mismatching between the data loading flow and the tree dynamic interaction characteristic, so how to reconstruct the data loading flow of the tree structure in a scene facing to more than one hundred thousand nodes so as to adapt to the rendering performance requirement and realize the interaction fluency is a technical problem to be solved urgently. Disclosure of Invention The invention aims to solve the technical problems in the background art and provides a tree structure big data loading method, a device, computer equipment and a computer storage medium based on double-engine cooperation. The technical scheme for solving the technical problems is as follows: In a first aspect, a dual-engine collaboration-based tree structure big data loading method is provided, wherein a prediction preloading engine and a progressive rendering engine which run in parallel and are resident are provided, and the method comprises the following steps: receiving original tree data, constructing a flattened index, and loading configuration parameters; creating a rendering container, starting state monitoring, instantiating a double engine and establishing a shared parameter channel; Monitoring rolling behavior, and predicting a preloading engine to calculate and share the unfolding probability of each node based on an operation probability model by combining with each frame of rendering time shared by a progressive rendering engine, and judging whether to trigger asynchronous loading, wherein the operation probability model is as follows: Wherein sigma a