CN-120011455-B - InfluxDB high availability system and working method thereof

CN120011455BCN 120011455 BCN120011455 BCN 120011455BCN-120011455-B

Abstract

The invention belongs to the field of computer data processing, and provides an InfluxDB high availability system and a working method thereof for solving the problem that the existing InfluxDB lacks a mature high availability scheme, wherein the InfluxDB high availability system comprises a client and an InfluxDB node, wherein in the InfluxDB node, a received data query request is forwarded to a storage node assembly through a service interface assembly, the storage node assembly packages the current query request into a query request supported by a query interface of the InfluxDB, performs the data query of the InfluxDB, and returns a query result to the client; and the InfluxDB nodes are arranged in pairs, each pair of InfluxDB nodes are mutually active and standby, and when data writing exists, the data synchronization of the active and standby nodes is performed through the file synchronization component.

Inventors

ZHANG HUI
JIANG JIAN
WU JINGJING
AN ZHANG
SONG WANPENG

Assignees

山东国数发展股份有限公司

Dates

Publication Date: 20260508
Application Date: 20250418

Claims (5)

1. An InfluxDB high availability system, comprising: The client is used for generating a data query or write request based on user requirements and issuing the data query or write request to the InfluxDB node; an InfluxDB node comprising an intermediate layer and an InfluxDB, wherein: The middle layer is used for forwarding a received data query request to the storage node component through the service interface component, packaging the current query request into a query request supported by a query interface of InfluxDB through the storage node component, carrying out data query of InfluxDB and returning a query result to the client; for the received data writing request, forwarding the data to be written to a file storage component through a service interface component to write the data of the local file, and simultaneously, writing the data to be written into the InfluxDB through a storage node component and returning to a writing state; Generating a data query or write request based on user requirements, wherein the data query or write request adopts a unified destination address to query or write data, and a Keepalive tool is used for monitoring the running state of a main node and a standby node, so that the virtual IP of the address is automatically switched to an InfluxDB node capable of working normally; The service interface component comprises a data query interface, a Ping interface, a data writing-in access and a state notification interface, and specifically comprises the following processing procedures that the Ping interface is called through a virtual IP routing service in a Keepalive tool, whether the current InfluxDB node operates normally or not is determined, if the operation is normal, the current InfluxDB node is selected to execute data query or writing, and if the operation is abnormal, a standby node of the current InfluxDB node is selected to execute data query or writing; The file storage component specifically performs the following processing procedures that when receiving newly written data, an offset address when finishing the latest synchronization is inquired, and the offset address is used as a writing position of the data to write a local file; when receiving a data query request, encapsulating the request into a query request supported by a query interface of InfluxDB, querying the InfluxDB, and encapsulating and returning returned data; when a data writing request is received, the data writing request is analyzed, the data writing request is assembled into a corresponding SQL sentence according to InfluxDB SQL grammar, the data is written into InfluxDB based on the SQL sentence, and the writing state is returned; InfluxDB, which is used for storage of data; The InfluxDB nodes are arranged in pairs, each pair of InfluxDB nodes are main and standby, and when data writing exists, the data synchronization of the main and standby nodes is performed through a file synchronization component; The file synchronization component specifically performs the following processing procedures of automatically reading the local file of the current node, reading file data according to the offset address of the local cache when the latest synchronization is completed, establishing TCP connection between the current node and the corresponding standby node, and realizing the synchronization of the local files of the two nodes based on TCP transmission.
2. The InfluxDB high availability system of claim 1, wherein after service initialization, if one InfluxDB node is selected as a master node, the file synchronization component specifically performs the following processing procedures of automatically reading a file local to the master node, reading in file data according to the locally cached synchronization offset, establishing TCP connection with another InfluxDB node, and synchronizing the file; If a certain InfluxDB node is selected as a slave node, the file synchronization component specifically executes the following processing procedures of creating a TCP client, establishing connection between the TCP client and another InfluxDB node, and receiving data and writing the data into a local file when the data synchronization of the other party is monitored.
3. A method of operating an InfluxDB high availability system, characterized in that it is based on an InfluxDB high availability system as claimed in any of claims 1-2, comprising: based on the destination address in the data inquiry or write request issued by the client, selecting an InfluxDB node capable of normally executing the current request; In the selected InfluxDB node, for a data query request, forwarding the data query request to a storage node assembly through a service interface assembly, packaging the current query request into a query request supported by a query interface of the InfluxDB through the storage node assembly, performing data query of the InfluxDB, and returning a query result to a client, for a data write request, forwarding data to be written to a file storage assembly through the service interface assembly to perform data writing of a local file, and simultaneously, writing the data to be written to the InfluxDB through the storage node assembly and returning to a writing state, wherein when the data writing exists, data synchronization of a master node and a slave node is performed through a file synchronization assembly.
4. A working method of an InfluxDB high availability system according to claim 3, wherein a unified destination address is adopted in the data query request or the write request to query or write data, and a Keepalive tool is used to monitor the operation condition of the active/standby node, and automatically switch the virtual IP of the address to the InfluxDB node capable of working normally.
5. The method of claim 3, wherein when a certain InfluxDB node is selected as the data write request, the local file of the current node is automatically read, file data is read according to the offset address of the local buffer when the latest synchronization is completed, TCP connection is established between the current node and the corresponding standby node, and synchronization of the local files of the two nodes is realized based on TCP transmission.

Description

InfluxDB high availability system and working method thereof Technical Field The invention belongs to the technical field of computer data processing, and particularly relates to an InfluxDB high-availability system and a working method thereof. Background The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art. InfluxDB high availability means that through a series of technology and architecture design, the InfluxDB database system can still continuously and stably provide service when facing various faults and abnormal conditions, and the availability of data and the reliability of the system are guaranteed, wherein InfluxDB, namely a time sequence database does not support clusters, currently two InfluxDB high availability schemes exist, one is a Fork version of InfluxDB-Relay which is a high availability scheme provided by an official, and the other is Influx-Proxy which is an improved version of InfluxDB-Relay. The InfluxDB-Relay scheme of the Fork version is externally a HTTPSERVER, after receiving a write request, the InfluxDB-Relay scheme can support writing data into the InfluxDB in an HTTP or UDP mode, and can also support remote writing (HTTP) of Prometaus, but the InfluxDB-Relay does not support a query interface of the InfluxDB and remote reading of the Prometaus, when the request for writing the data reaches the InfluxDB-Relay layer, the InfluxDB-Relay can initiate a write request to the InfluxDB at the back end configured by the InfluxDB-Relay, but the problem of data consistency cannot be solved, if the writing of the InfluxDB fails, only one error message can be simply returned, and the data which fails to be written is lost in the InfluxDB; Secondly, the InfluxDB-Proxy scheme, which is mainly used for Proxy and forwarding requests, acts as an intermediate layer between a client and one or more InfluxDB instances, can process the forwarding of requests in real time, is transparent to the upper layer client, can be used like a single InfluxDB, the Proxy scheme is used for writing the request once, which corresponds to a back-end database and leads to the limitation of the back-end database, in addition, the InfluxDB-Proxy needs additional configuration KEYMAPS, the InfluxDB-Proxy is mainly used for correctly mapping and adapting the incoming request and the InfluxDB service of the back-end, KEYMAPS is essentially a configuration mechanism used for processing possible differences between different client requests and the back-end database, the writing request is randomly routed to the back-end database, which leads to the non-uniform back-end data storage of different databases, and each writing is randomly routed, and the writing data amount cannot be controlled to be the same at each node, which leads to the non-uniform data load. In summary, the existing scheme lacks a mature InfluxDB high availability scheme, the existing scheme does not support a query interface, data consistency cannot be ensured, and data cannot be backed up quickly. Disclosure of Invention The embodiment of the invention provides an InfluxDB high-availability system and a working method thereof, which are used for solving the problems that the traditional scheme does not support a query interface, the consistency of data cannot be ensured, and the data cannot be backed up quickly. According to a first aspect provided by an embodiment of the present invention, there is provided an InfluxDB high availability system, including: The client is used for generating a data query or write request based on user requirements and issuing the data query or write request to the InfluxDB node; an InfluxDB node comprising an intermediate layer and an InfluxDB, wherein: The middle layer is used for forwarding a received data query request to the storage node component through the service interface component, packaging the current query request into a query request supported by a query interface of InfluxDB through the storage node component, carrying out data query of InfluxDB and returning a query result to the client; for the received data writing request, forwarding the data to be written to a file storage component through a service interface component to write the data of the local file, and simultaneously, writing the data to be written into the InfluxDB through a storage node component and returning to a writing state; InfluxDB, which is used for storage of data; The InfluxDB nodes are arranged in pairs, each pair of InfluxDB nodes are active and standby, and when data writing exists, data synchronization of the active and standby nodes is performed through a file synchronization component. Further, the data query or write request is generated based on the user requirement, wherein the data query or write request adopts a unified destination address to query or write data, the operation condition of the active and standby nodes is monito