CN-122001781-A - Data storage method, device, equipment and storage medium

CN122001781ACN 122001781 ACN122001781 ACN 122001781ACN-122001781-A

Abstract

The present application relates to the field of data processing technologies, and in particular, to a data storage method, apparatus, device, and storage medium. The data storage method comprises the steps of monitoring a monitoring index of a current node by an auxiliary monitoring system of the current node if the current node is an auxiliary node, acquiring monitoring data through monitoring, storing the monitoring data in a local data unit corresponding to the current node, setting a preset index, screening the monitoring data corresponding to the preset index in the monitoring data as the monitoring auxiliary data of the current node, transmitting the monitoring auxiliary data to a control node connected with the auxiliary node, and acquiring the monitoring auxiliary data of the auxiliary node through a control monitoring system of the control node. The method solves the technical problem that the input and output pressure of the disk data is overlarge due to centralized storage of the monitoring system in a large-scale scene, and achieves the technical effects of improving the monitoring scale under the same hardware condition and further reducing the load of the control node.

Inventors

YAN DONGDONG
WU JINGHE

Assignees

济南浪潮数据技术有限公司

Dates

Publication Date: 20260508
Application Date: 20260312

Claims (10)

1. A data storage method, the data storage method comprising: responding to the current node as an auxiliary node, and monitoring a monitoring index of the current node by an auxiliary monitoring system of the current node to obtain monitoring data; storing the monitoring data in a local data unit of the current node; Screening monitoring data corresponding to preset indexes in the monitoring data to serve as monitoring auxiliary data of the current node; and transmitting the monitoring auxiliary data to a control node connected with the auxiliary node so that a control monitoring system of the control node can acquire the monitoring auxiliary data of the auxiliary node.
2. The data storage method of claim 1, wherein the affiliated monitoring system comprises a proxy service, a monitoring unit, and the local data unit, wherein the monitoring unit is configured to monitor the current node; the transmitting the monitoring auxiliary data to the control node connected with the auxiliary node comprises: The proxy service responds to the transmission requirement, accesses the local data unit storing the monitoring data, acquires the monitoring auxiliary data meeting the transmission requirement or takes the real-time monitoring data meeting the transmission requirement as the monitoring auxiliary data; the proxy service transmits the monitoring adjunct data to the management component of the control monitoring system to which it is connected.
3. The data storage method according to claim 1 or 2, wherein the screening the monitoring data corresponding to the preset index in the monitoring data as the monitoring auxiliary data of the current node includes: Acquiring an index type identifier of the monitoring index; Identifying whether the indicator type identifier is a first preset field; judging whether the monitoring index belongs to the preset index or not according to the index type identifier, wherein the index type identifier is a first preset field; wherein the frequency of the monitoring data transmitted to the local data unit is higher than the frequency of the monitoring auxiliary data transmitted to the control node.
4. The data storage method of claim 1, wherein the storing the monitoring data in the local data unit of the current node comprises: Determining whether a local monitoring data receiving function of the current node is configured in an on state, determining to store the monitoring data to the local data unit in response to the local receiving function of the current node being configured in an on state, and/or, The transmitting the monitoring auxiliary data to the control node connected with the auxiliary node comprises: acquiring a receiving function state of the control node; judging whether the receiving function state characterizes the monitoring auxiliary data of the auxiliary node of the control node; responsive to the receiving functional status characterizing that the control node is permitted to receive monitored ancillary data of its ancillary node, determining to transmit the monitored ancillary data to the control node to which the ancillary node is connected.
5. The data storage method of claim 4, the data storage method is characterized by further comprising the following steps: responding to the current node as a control node or an auxiliary node, acquiring a data input request by the control monitoring system or the auxiliary monitoring system, and inquiring a data source of data to be input carried by the data input request; and receiving the data to be input in response to the data source conforming to the receiving function state of the current node.
6. The data storage method according to claim 1 or 4, wherein the acquiring the monitoring data comprises: acquiring a preset monitoring data transmission mode of the current node; Responding to the preset transmission mode of the current node as a first transmission mode, and acquiring a data scheduling mode of the current node; The method comprises the steps of responding to the data scheduling mode to be a serial scheduling mode, obtaining a transmission list, constructing a transmission connection channel between the transmission list and the control node, transmitting the monitoring data of the current node through the transmission connection channel, responding to the completion of transmitting the monitoring data of the current node, and interrupting the transmission connection channel; Responding to the data scheduling mode as a parallel scheduling mode, constructing a plurality of parallel transmission connection channels and forming a connection pool, adding the monitoring data into a sending buffer pool, and transmitting the monitoring data in the sending buffer pool in parallel through the plurality of transmission connection channels in the connection pool; and constructing a monitoring data packet in response to the second transmission mode of the monitoring data transmission mode, and simultaneously transmitting the monitoring data packet to each control node, wherein the monitoring data packet stores the monitoring auxiliary data.
7. The data storage method according to claim 1, wherein the acquiring the monitoring data includes, after: Responding to the node state as abnormal node state, and feeding back abnormal node state information to the control node so as to enable the control node to perform abnormal processing on the current node in the abnormal node state; and/or the number of the groups of groups, The storing the monitoring data in the local data unit of the current node comprises: Taking the monitoring data stored in the local data unit as local data; And responding to the data state of the local data as an abnormal data state and/or the static time length of the local data reaches a static threshold value, judging the local data as abnormal data and deleting the abnormal data, wherein the static time length is the time length of updating the local data from the adjacent previous time.
8. A data storage device, comprising: the monitoring module is used for responding to the current node as an auxiliary node, and an auxiliary monitoring system of the current node monitors the monitoring index of the current node to acquire monitoring data; the storage module is used for storing the monitoring data in a local data unit of the current node; The screening module is used for screening monitoring data corresponding to preset indexes in the monitoring data to serve as monitoring auxiliary data of the current node; And the transmission module is used for transmitting the monitoring auxiliary data to the control node connected with the auxiliary node so as to enable the control monitoring system of the control node to acquire the monitoring auxiliary data of the auxiliary node.
9. An electronic device, comprising: A memory for storing a computer program; A processor for implementing the steps of the data storage method according to any one of claims 1 to 7 when executing said computer program.
10. A computer readable storage medium, characterized in that a computer program is stored in the computer readable storage medium, wherein the computer program, when being executed by a processor, implements the steps of the data storage method according to any of claims 1 to 7.

Description

Data storage method, device, equipment and storage medium Technical Field The present application relates to the field of data processing technologies, and in particular, to a data storage method, a data storage device, an electronic device, and a computer readable storage medium. Background Cluster monitoring tools are widely used in private cloud scenarios due to their lightweight and ease of scalability, but they rely on ring databases as a design for centralized storage, which presents an inherent bottleneck. Each monitoring item needs to be independently written into the annular database file, so that when the monitoring scale is enlarged, for example, thousands of nodes and hundreds of thousands of monitoring indexes, batch write operations at fixed intervals can cause the rapid increase of disk data input and output operations. Conventional solutions are still limited by single-node storage performance and hardware cost under a very large scale scene (such as ten thousand nodes), and cannot solve the problem of system performance attenuation caused by disk data input and output pressure. Disclosure of Invention The application provides a data storage method, a data storage device, electronic equipment and a computer readable storage medium, which at least solve the technical problem that the input and output pressure of magnetic disk data is overlarge due to centralized storage in a large-scale scene of a monitoring system. The application provides a data storage method, which comprises the steps of responding to the current node as an auxiliary node, monitoring a monitoring index of the current node by an auxiliary monitoring system of the current node to obtain monitoring data, storing the monitoring data in a local data unit of the current node, screening the monitoring data corresponding to a preset index in the monitoring data to serve as monitoring auxiliary data of the current node, and transmitting the monitoring auxiliary data to a control node connected with the auxiliary node so as to enable a control monitoring system of the control node to obtain the monitoring auxiliary data of the auxiliary node. The application further provides a data storage device, which comprises a monitoring module, a storage module, a screening module and a transmission module, wherein the monitoring module is used for responding to the current node as an auxiliary node, an auxiliary monitoring system of the current node monitors monitoring indexes of the current node to obtain monitoring data, the storage module is used for storing the monitoring data in a local data unit of the current node, the screening module is used for screening the monitoring data corresponding to preset indexes in the monitoring data to serve as monitoring auxiliary data of the current node, and the transmission module is used for transmitting the monitoring auxiliary data to a control node connected with the auxiliary node so that a control monitoring system of the control node can obtain the monitoring auxiliary data of the auxiliary node. The application also provides electronic equipment which comprises a memory and a processor, wherein the memory is used for storing a computer program, and the processor is used for realizing the steps of any data storage method when executing the computer program. The present application also provides a computer readable storage medium having a computer program stored therein, wherein the computer program when executed by a processor implements the steps of any of the data storage methods described above. According to the application, all monitoring data are required to be stored locally by the auxiliary node, and only the monitoring auxiliary data corresponding to the screened preset index are transmitted to the control node, and the data storage mechanism changes the data circulation mode of the traditional monitoring architecture. Because the auxiliary node itself bears the local storage responsibility of the full data, the control node does not need to receive and process all original monitoring data, and only needs to receive the screened key data subset, namely the monitoring auxiliary data, thereby obviously reducing the data input and output pressure of the control node. Meanwhile, the integrity and real-time accessibility of the monitoring data are ensured by the local full-quantity storage, focusing key information can be selected by screening transmission of preset indexes, and network bandwidth occupation and centralized storage load are effectively reduced on the premise of guaranteeing a core monitoring function, so that the expansibility and stability of the system in a large-scale node deployment scene are improved. Therefore, the method can solve the technical problem that the input and output pressure of the disk data is overlarge due to centralized storage of the monitoring system in a large-scale scene, and achieves the technical effects of improving the monitoring scale under th