CN-121979719-A - Root cause positioning method, equipment and medium for memory overflow of database

CN121979719ACN 121979719 ACN121979719 ACN 121979719ACN-121979719-A

Abstract

The application discloses a root cause positioning method, equipment and medium for memory overflow of a database, and relates to the technical field of computers, wherein the method comprises the steps of judging memory abnormal states corresponding to memory index data through abnormal trend analysis and associated feature analysis so as to determine the memory overflow abnormal states of the database; the method comprises the steps of carrying out time axis alignment on hierarchical log data to obtain synchronous log data, collecting corresponding memory blocks in a data memory page according to preset snapshot triggering conditions, carrying out dynamic pile insertion on the memory blocks to obtain synchronous snapshot data, determining a memory overflow mode based on memory overflow abnormal states through memory overflow identification algorithm matching, and carrying out multi-scene abnormal positioning on the memory overflow mode according to the synchronous log data and the synchronous snapshot data to determine root cause nodes corresponding to the memory overflow mode. The method solves the technical problems that the positioning mode of the overflow root cause of the memory of the database is single and the automatic adaptation capability of the database is lacking.

Inventors

MIAO JIAN
DONG XIAO
Feng Mingran
CHEN LI
GAO HUA

Assignees

瀚高基础软件股份有限公司

Dates

Publication Date: 20260505
Application Date: 20260408

Claims (10)

1. The root cause positioning method for the memory overflow of the database is characterized by comprising the following steps: periodically acquiring memory index data of a database, and judging a memory abnormal state corresponding to the memory index data through abnormal trend analysis and associated feature analysis to determine a memory overflow abnormal state of the database; acquiring hierarchical log data, and performing time axis alignment on the hierarchical log data to obtain synchronous log data; The method comprises the steps of collecting corresponding memory blocks in a data memory page according to components through preset snapshot triggering conditions, and dynamically inserting piles to the memory blocks to obtain synchronous snapshot data; based on the memory overflow abnormal state, determining a memory overflow mode through matching of a memory overflow identification algorithm; And according to the synchronous log data and the synchronous snapshot data, carrying out multi-scene abnormal positioning on the memory overflow mode so as to determine a root cause node corresponding to the memory overflow mode.
2. The method for locating the root cause of memory overflow in a database according to claim 1, wherein the determining the memory abnormal state corresponding to the memory index data by means of an abnormal trend analysis and an associated feature analysis, comprises: Performing initial threshold configuration on the memory index data based on database parameters corresponding to the database, and performing baseline dynamic adjustment on the initial threshold to obtain an index threshold corresponding to the memory index data, wherein the database parameters comprise database types, service loads and deployment scenes; according to the index threshold, threshold judgment is carried out on the memory index data, and a first abnormal judgment result is determined; Performing abnormal trend analysis on the memory index data to determine a second abnormal judgment result, wherein the types of the abnormal trend analysis comprise abnormal memory growth rate, abnormal connection number growth rate, abnormal node resource competition rate and abnormal distributed transaction; performing associated feature analysis on the memory index data to determine a third abnormal judgment result, wherein associated indexes of the associated feature analysis comprise the number of unreleased memory blocks, associated session, instantaneous overflow parameters, memory occupation amount and synchronous delay; And judging the abnormal states of the first abnormal judgment result, the second abnormal judgment result and the third abnormal judgment result through a voting mechanism, and determining the memory overflow abnormal state, wherein the memory overflow abnormal state comprises memory leakage, instantaneous overflow, resource leakage, containerization abnormality and distributed abnormality.
3. The method for locating the root cause of memory overflow in a database according to claim 1, wherein the time axis alignment is performed on the hierarchical log data to obtain synchronous log data, specifically comprising: Carrying out standardized processing on the hierarchical log data to obtain full-source log data in a JSON format, wherein the hierarchy of the hierarchical log data comprises a system layer, a database layer, an application layer, a container layer and a distributed layer; And carrying out time axis alignment on the all-source log data to obtain the synchronous log data.
4. The method for locating the root cause of memory overflow in a database according to claim 1, wherein the component collecting the corresponding memory block in the data memory page by the preset snapshot triggering condition specifically comprises: Acquiring memory state data, and carrying out snapshot triggering judgment on the memory state data based on the snapshot triggering condition to determine a snapshot triggering type corresponding to the memory state, wherein the snapshot triggering type comprises early warning triggering, pre-fault triggering and timing triggering; Classifying the memory pages according to the memory types of the database to obtain the memory page types; And obtaining the memory block through component sampling configuration based on the snapshot trigger type and the memory page type.
5. The method for locating the root cause of memory overflow in a database according to claim 1, wherein the dynamic instrumentation is performed on the memory block to obtain synchronous snapshot data, and the method specifically comprises: defining a unique mark corresponding to the memory block, and dynamically inserting piles in a memory application function of the database process to obtain the memory block inserted with the unique mark, wherein the unique mark comprises a fixed identifier, an allocation source code and a memory block ID; and carrying out snapshot synchronization on the memory block inserted with the unique mark to obtain the synchronous snapshot data.
6. The method for locating the root cause of a memory overflow in a database according to claim 1, wherein the determining a memory overflow pattern based on the memory overflow abnormal state by matching a memory overflow identification algorithm comprises: Constructing an overflow feature set of the type of the database corresponding to the memory overflow abnormal state, and carrying out normalization processing on the overflow feature set to obtain an abnormal feature set, wherein the abnormal feature set comprises basic features and scenerising features; Performing score matching on feature conditions corresponding to the abnormal feature set to determine matching scores corresponding to the abnormal features in the abnormal feature set; screening candidate threshold values of the rule traversal for the matching scores to obtain a first memory overflow mode; Carrying out random forest algorithm processing of the whole scene on the first memory overflow mode to obtain a second memory overflow mode; And determining the memory overflow mode according to the first memory overflow mode or the second memory overflow mode through judging the output probability value.
7. The method for positioning the root cause of memory overflow in a database according to claim 1, wherein the multi-scenario exception positioning is performed on the memory overflow mode according to the synchronous log data and the synchronous snapshot data to determine the root cause node corresponding to the memory overflow mode, and the method specifically comprises: Based on the synchronous log data, calculating the memory contribution degree corresponding to each service scene to determine a first root cause node; extracting an abnormal service scene in the synchronous log data, and carrying out abnormal session screening on the abnormal service scene to determine a second root cause node; acquiring execution SQL, and performing cost analysis on the execution SQL to determine a third root cause node; determining a fourth root cause node through module exception analysis according to the synchronous snapshot data; Extracting a memory allocation call stack in the synchronous snapshot data, and carrying out feature analysis on the memory allocation call stack to determine a fifth root cause node, wherein the feature analysis comprises symbol analysis and AST analysis; and determining the root cause node corresponding to the memory overflow mode through container resource analysis and distributed link tracking based on the first root cause node, the second root cause node, the third root cause node, the fourth root cause node and the fifth root cause node.
8. The method for locating the root cause of a memory overflow in a database according to claim 1, wherein after performing multi-scenario exception locating on the memory overflow mode according to the synchronization log data and the synchronization snapshot data to determine the root cause node corresponding to the memory overflow mode, the method further comprises: the root node is arranged into a corresponding root positioning report through a preset Freemarker template engine, and the root positioning report is stored into a preset root library; And updating the memory overflow identification algorithm according to the root cause library.
9. A root cause positioning device for a database memory overflow, the device comprising: At least one processor; and a memory communicatively coupled to the at least one processor; Wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to implement a method of root cause localization of a database memory overflow as claimed in any one of claims 1 to 8.
10. A non-volatile computer storage medium storing computer executable instructions for root cause localization of a database memory overflow, wherein the computer executable instructions when executed implement a method for root cause localization of a database memory overflow as claimed in any one of claims 1 to 8.

Description

Root cause positioning method, equipment and medium for memory overflow of database Technical Field The present application relates to the field of computer technologies, and in particular, to a method, an apparatus, and a medium for locating a root cause of a database memory overflow. Background Along with the deep digital transformation, the application scene of the database extends from the traditional single machine deployment to the cloud protogenesis containerization and the distributed cluster, the carried traffic and data volume increase exponentially, the memory overflow problem is more frequent and complex, and the memory overflow of the database is mainly divided into three types of memory leakage, instantaneous overflow and resource leakage. Meanwhile, the existing production environment has high requirements on the availability of a database, the positioning process is not allowed to modify source codes, interrupt service or cause obvious performance loss, and the root cause positioning of memory overflow is required to be carried out in a dynamic environment. In the prior art, the positioning method for memory overflow is mainly divided into two types. Firstly, the conventional database memory positioning technology is designed to be used for aiming at a single scene, has insufficient suitability for cloud protoclasis or distributed scenes, and has a single positioning mode. Secondly, for the memory overflow positioning technology of the universal container, although the automatic positioning of the containerized application can be realized, the automatic customization can not be performed according to the service characteristics and the kernel module of the database, and the accurate positioning requirement of the database scene can not be directly adapted. Therefore, there is a need for a low-intrusion, automatically-adapted method for locating the root cause of memory overflow in a database. Disclosure of Invention The embodiment of the application provides a method, equipment and medium for positioning the root cause of memory overflow of a database, which solve the technical problems that the method for positioning the root cause of memory overflow of the database is single and the automatic adaptation capability of the database is lacking in the prior art. The embodiment of the application provides a root cause positioning method for memory overflow of a database, which is characterized by comprising the steps of periodically obtaining memory index data of the database, judging a memory abnormal state corresponding to the memory index data through abnormal trend analysis and associated feature analysis to determine the memory overflow abnormal state of the database, obtaining hierarchical log data, performing time axis alignment on the hierarchical log data to obtain synchronous log data, acquiring corresponding memory blocks in memory pages of the data through preset snapshot triggering conditions, performing dynamic pile insertion on the memory blocks to obtain synchronous snapshot data, determining a memory overflow mode through memory overflow identification algorithm matching based on the memory overflow abnormal state, and performing multi-scene abnormal positioning on the memory overflow mode according to the synchronous log data and the synchronous snapshot data to determine root cause nodes corresponding to the memory overflow mode. In one implementation mode of the application, the memory abnormal state corresponding to the memory index data is judged through abnormal trend analysis and associated feature analysis to determine the memory overflow abnormal state of the database, and the method specifically comprises the steps of carrying out initial threshold configuration on the memory index data based on database parameters corresponding to the database, and carrying out baseline dynamic adjustment on the initial threshold to obtain the index threshold corresponding to the memory index data, wherein the database parameters comprise database types, service loads and deployment scenes, carrying out threshold judgment on the memory index data according to the index threshold to determine a first abnormal judgment result, carrying out abnormal trend analysis on the memory index data to determine a second abnormal judgment result, wherein the types of the abnormal trend analysis comprise abnormal memory growth rate, abnormal connection number growth rate, abnormal node resource competition rate and distributed transaction abnormality, carrying out associated feature analysis on the memory index data to determine a third abnormal judgment result, and the associated feature analysis associated indexes comprise unreleased memory block number, associated session, instantaneous overflow parameters and memory occupation amount and synchronous delay, carrying out threshold judgment on the first abnormal judgment result, the second abnormal judgment result and the third abnormal judgment r