CN-121997308-A - Cross-component metadata access control method and device for information creation big data platform

CN121997308ACN 121997308 ACN121997308 ACN 121997308ACN-121997308-A

Abstract

The invention relates to the technical field of big data access control, and discloses a cross-component metadata access control method and a device of a message creation big data platform, which firstly constructs a metadata virtualization layer for mapping a plurality of metadata databases, the metadata virtualization layer maps a plurality of metadata databases, and subsequent cross-database field mapping and result aggregation operations can be realized in the metadata virtualization layer, so that each database does not need to be accessed independently, and the efficient cross-component access operation is realized. The authentication key is designed into the retrieval section and the key section, the retrieval section is only used for retrieving and distinguishing the key, the user side and the metadata database respectively send the respective key sections to the metadata virtualization layer, unified authentication of the user side authority is realized in the virtual mapping layer through matching of the authentication key, and independent authentication between the user side and each metadata database is not needed, so that the efficiency of metadata access control is improved.

Inventors

PAN XIAOMING
MA ZIJUN
WU KUNMING
ZHANG CHENGWEI
ZHU WEIWEI
ZHU YIFAN
GAO HONGBO
DENG WEIKANG
LI KECHONG
ZHENG TAO
DONG WEIWEI

Assignees

华安证券股份有限公司

Dates

Publication Date: 20260508
Application Date: 20260409

Claims (10)

1. A method of cross-component metadata access control for a big data platform, applied to a metadata virtualization layer, the metadata virtualization layer configured to map a plurality of metadata databases, the method comprising: Receiving a user authentication key sent by a user terminal, wherein the user authentication key comprises a user retrieval section and a user key section; The user retrieval sections are respectively sent to all metadata databases to obtain a library key section returned by each target metadata database, wherein the target metadata databases are metadata databases which store library authentication keys corresponding to the user authentication keys, the library authentication keys comprise library retrieval sections and library key sections, the library retrieval sections correspond to the user retrieval sections, and the library key sections correspond to the user key sections; Matching the user key segment with each library key segment, and determining the access right corresponding to the user terminal; And sending at least one first key to the user side to obtain a user authentication key in the user side, and sending at least one second key to all the target metadata databases to obtain a library authentication key in each target metadata database.
2. The method for controlling access to metadata of a big data platform according to claim 1, wherein the matching the user key segment with each library key segment, determining the access right corresponding to the user side, includes: performing bidirectional verification on each library key segment and each user key segment, and generating a temporary access token after the verification is finished, wherein the temporary access token is used for indicating the metadata access authority corresponding to the user side; And, the method further comprises: And receiving a query request sent by a user terminal, and executing cross-library field mapping and result aggregation operations on the metadata virtualization layer according to the query request and the temporary access token to obtain a query result corresponding to the query request.
3. The method for controlling access to metadata of a cross-component of a big data platform according to claim 2, wherein the performing bidirectional verification on each library key segment and the user key segment, and generating a temporary access token after the verification is finished, includes: for each library key segment, carrying out bidirectional verification on the library key segment and the user key segment, if the verification is successful, determining metadata access rights corresponding to the user side in a metadata database corresponding to the library key segment according to a rights field carried by the library key segment, wherein the rights field is a field which is set by the corresponding metadata database for the user side and used for measuring the access rights enjoyed by the user side; and constructing a temporary access token according to all the metadata access rights.
4. The method for controlling cross-component metadata access of a big data platform of claim 3, wherein the permission field is a field which is set by a corresponding metadata base for the user terminal and used for measuring access permission and receiving permission enjoyed by the user terminal; And performing cross-library field mapping and result aggregation operations on the metadata virtualization layer according to the query request and the temporary access token to obtain a query result corresponding to the query request, wherein the method comprises the following steps: Performing cross-library field mapping and result aggregation operations on the metadata virtualization layer according to the query request and the temporary access token to obtain an initial query result corresponding to the query request; and performing desensitization operation on the initial query result according to the temporary access token to obtain a query result corresponding to the query request.
5. The method for cross-component metadata access control for a big data platform of claim 4, further comprising: Receiving a sub-user query request and a sub-user limit token sent by a user side, wherein the sub-user query request is generated by the user side according to a query application of a sub-user of the user side, the sub-user limit token is generated by the user side according to the temporary access token and a sub-user right corresponding to the sub-user in the user side, and the sub-user limit token is used for further limiting metadata access right indicated by the temporary access token and query result receiving right; Generating a temporary access sub-token corresponding to the sub-user according to the sub-user limit token and the temporary access token; Performing cross-library field mapping and result aggregation operations on the metadata virtualization layer according to the sub-user query request and the temporary access sub-token to obtain an initial sub-user query result corresponding to the sub-user query request; and performing desensitization operation on the initial sub-user query result according to the temporary access sub-token to obtain a sub-user query result corresponding to the sub-user query request.
6. The method for cross-component metadata access control for a big data platform of any of claims 1-4, further comprising: The metadata virtualization layer receives a key revocation instruction sent by an accessed metadata base, wherein the key revocation instruction is obtained after the user side directly and independently accesses a corresponding accessed metadata base by using a user authentication key, and the accessed metadata base adds the user authentication key to the revocation instruction; The metadata virtualization layer sends the key revocation instruction to all metadata databases except the accessed metadata database, so that each metadata database uses the key revocation instruction to revoke a corresponding library authentication key.
7. The method of cross-component metadata access control for a big data platform of claim 6, wherein the generating at least one pair of a first key and a second key comprises: and receiving a key distribution request sent by a user terminal, wherein the key distribution request is generated after the user terminal judges that the number of the stored user authentication keys is less than a preset number threshold.
8. The method for cross-component metadata access control for a big data platform of any of claims 1-4, further comprising: responding to an operation instruction of a management terminal, generating a corresponding permission modification instruction by the metadata virtualization layer aiming at a metadata database to be operated, wherein the metadata database to be operated is a metadata database which is indicated by the operation instruction and needs to modify the permission of the user side, and the permission modification instruction is used for indicating to modify the access permission of the user side in the corresponding metadata database to be operated; And the metadata virtualization layer sends all the permission modification instructions to all the metadata databases to be operated so that each metadata database to be operated modifies the permission field carried by the library key section corresponding to the user side according to the corresponding permission modification instruction.
9. A cross-component metadata access control apparatus for a big data platform, applied to a metadata virtualization layer for mapping a plurality of metadata databases, the apparatus comprising: The key receiving module is used for receiving a user authentication key sent by the user terminal, wherein the user authentication key comprises a user retrieval section and a user key section; The key retrieval module is used for respectively transmitting the user retrieval sections to all metadata databases to obtain a library key section returned by each target metadata database, wherein the target metadata database is a metadata database which stores a library authentication key corresponding to the user authentication key, the library authentication key comprises a library retrieval section and a library key section, the library retrieval section corresponds to the user retrieval section, and the library key section corresponds to the user key section; The key matching module is used for matching the user key segment with each library key segment and determining the access right corresponding to the user side; And the key distribution module is used for generating at least one pair of first keys and second keys, wherein the first keys and the second keys have corresponding retrieval sections and key sections, transmitting at least one first key to the user side to obtain a user authentication key in the user side, and transmitting at least one second key to all the target metadata databases to obtain a library authentication key in each target metadata database.
10. A cross-component metadata access control system for a big data platform, the system comprising a memory storing executable program code, a processor coupled to the memory, the processor invoking the executable program code stored in the memory to perform the cross-component metadata access control method for a big data platform according to any of claims 1-8.

Description

Cross-component metadata access control method and device for information creation big data platform Technical Field The invention relates to the technical field of big data access control, in particular to a cross-component metadata access control method and device for a message creation big data platform. Background With the deep advancement of information technology application innovation (belief creation) strategy, a belief creation big data platform constructed on the basis of domestic software and hardware has become a core information infrastructure supporting data processing and analysis in key industries and fields. Such platforms typically employ heterogeneous, loosely coupled architectures that integrate multiple autonomous developed or adapted large data components, such as distributed file systems (e.g., HDFS localization alternatives), computing engines (e.g., mapReduce, spark localization versions or alternatives), data warehouses (e.g., open source or self-developed OLAP based systems), data lake management systems, and various types of data service tools, etc., to collectively accomplish the collection, storage, computation, analysis, and servicing of the mass data. In these complex multi-component collaborative environments, metadata is used as data describing data (e.g., schema definition of databases/tables, storage locations, partition information, column statistics, data blood-edges, access rights attributes, etc.) and is a cornerstone for achieving data discovery, understanding, management, administration, and efficient access. However, in the prior art, an SQL query may require sequential access to metadata services to parse library table information, locate the resources of the compute engine, obtain the underlying storage path, and possibly trace back data edges, so it is critical to achieve efficient cross-component access operations. On the other hand, the security and controlled access of the metadata are directly related to the data security, privacy protection capability and compliance level of the whole platform, especially in the key fields of government affairs, finance, energy and the like for processing sensitive data, and if the data from different sources can be uniformly authenticated under a unified view, a better metadata access control method can be provided. Therefore, in the created large data platform, how to realize the high-efficiency cross-component access operation and how to perform unified authentication on data from different sources under a unified view becomes a technical problem to be solved. Disclosure of Invention The invention provides a cross-component metadata access control method and device for a message creation big data platform, which are used for realizing high-efficiency cross-component access operation and carrying out unified authentication on data from different sources under a unified view. The first aspect of the invention discloses a cross-component metadata access control method of a big data platform, which is applied to a metadata virtualization layer, wherein the metadata virtualization layer is used for mapping a plurality of metadata databases, and the method comprises the following steps: Receiving a user authentication key sent by a user terminal, wherein the user authentication key comprises a user retrieval section and a user key section; The user retrieval sections are respectively sent to all metadata databases to obtain a library key section returned by each target metadata database, wherein the target metadata databases are metadata databases which store library authentication keys corresponding to the user authentication keys, the library authentication keys comprise library retrieval sections and library key sections, the library retrieval sections correspond to the user retrieval sections, and the library key sections correspond to the user key sections; Matching the user key segment with each library key segment, and determining the access right corresponding to the user terminal; And sending at least one first key to the user side to obtain a user authentication key in the user side, and sending at least one second key to all the target metadata databases to obtain a library authentication key in each target metadata database. In an optional implementation manner, in a first aspect of the present invention, the matching the user key segment with each of the library key segments, to determine an access right corresponding to the user side, includes: performing bidirectional verification on each library key segment and each user key segment, and generating a temporary access token after the verification is finished, wherein the temporary access token is used for indicating the metadata access authority corresponding to the user side; And, the method further comprises: And receiving a query request sent by a user terminal, and executing cross-library field mapping and result aggregation operations on the metadata virtua