CN-122027317-A - Unified management and safety management and control platform for large model

CN122027317ACN 122027317 ACN122027317 ACN 122027317ACN-122027317-A

Abstract

The invention provides a large model unified management and safety management platform, which comprises an interface service module, a routing control module, a safety control module, a tenant management module, a monitoring audit module and a resource scheduling module, wherein the interface service module is used for providing a standardized large model access entrance, receiving a calling request from an application end and returning a response, the routing control module is used for routing the calling request to a corresponding target large model according to an application identifier carried by the application end and managing access credentials of the target large model, the safety control module is used for carrying out identity verification, authority check, sensitive information identification and interception on the calling request and carrying out safety audit on the calling process, the tenant management module is used for realizing logic isolation of data, configuration and access authority among different tenants based on tenant identifiers, the monitoring audit module is used for monitoring the running states and resource use indexes of the platform and the target large model in real time and storing the calling logs for audit tracing, and the resource scheduling module is used for automatically scheduling and expanding computing resources carrying the large model according to traffic load and resource utilization.

Inventors

WANG YAOWEI
FU CHUNLIN
MA FEIFEI

Assignees

王耀伟

Dates

Publication Date: 20260512
Application Date: 20260306

Claims (10)

1. The large model unified management and safety management platform is characterized by comprising an interface service module, a route control module, a safety control module, a tenant management module, a monitoring audit module and a resource scheduling module, The interface service module is used for providing a standardized large model access entry, receiving a call request from an application end and returning a response; The routing control module stores a mapping relation between the application identifier and the target large model, and is used for routing the call request to the corresponding target large model according to the application identifier carried by the application end and managing access credentials of the target large model; The security control module is used for carrying out identity verification, authority check, sensitive information identification and interception on the call request and carrying out security audit on the call process; the tenant management module is used for realizing the logic isolation of data, configuration and access rights among different tenants based on tenant identification; the monitoring audit module is used for monitoring the running states and resource use indexes of the platform and the target large model in real time, and storing call logs for audit tracing in a lasting manner; the resource scheduling module is used for automatically scheduling and expanding the computing resources of the large bearing model according to the service load and the resource utilization rate.
2. The large model unified management and security management platform according to claim 1, wherein: The interface service module adopts an interface protocol compatible with a preset OPEN standard, and performs standardized encapsulation and abstract output on the capabilities of various large models accessed from the lower layer, wherein the preset OPEN standard is an OPEN AI application programming interface standard.
3. The large model unified management and security management platform according to claim 1, wherein: wherein the application identifier is an application programming interface key that is uniformly distributed and managed by the management and control platform.
4. A large model unified management and security management platform according to claim 1 or 3, wherein: The routing control module can dynamically adjust the mapping relation, the application end can switch and access different large models by updating the mapping relation without modifying the configuration of the application end, access credentials of a target large model are stored and managed in a centralized mode, and the application end does not carry the access credentials of the target large model in a call request.
5. The large model unified management and security management platform according to claim 1, wherein: The security control module comprises a sensitive data interception engine, and the sensitive data interception engine identifies and intercepts sensitive information in the call request and/or the response through a predefined sensitive word stock and/or a regular matching rule and desensitizes outbound data.
6. The large model unified management and security management platform according to claim 1 or 5, wherein: the security control module is also used for controlling the calling frequency, alarming abnormal access behaviors and guaranteeing data transmission security by adopting an encryption transmission channel.
7. The large model unified management and security management platform according to claim 6, wherein: The security management and control module is further used for analyzing the calling mode based on the behavior analysis model and early warning abnormal access behaviors.
8. The large model unified management and security management platform according to claim 1, wherein: the tenant management module supports configuring different accessible large model resource lists and subdivision operation authorities for different tenants, wherein the subdivision operation authorities comprise one or more of calling authorities, parameter adjustment authorities and result reading authorities of a designated large model, and all operation behaviors are associated to corresponding tenant identifications for recording.
9. The large model unified management and security management platform according to claim 1, wherein: The monitoring audit module provides monitoring indexes including one or more of GPU utilization rate, video memory occupation, request response time delay and call success rate, and a visual interface for displaying the monitoring indexes, inquiring call logs and configuring audit strategies.
10. The large model unified management and security management platform according to claim 1, wherein: The resource scheduling module automatically triggers the transverse expansion of the model node when the request concurrency and/or the calculation resource utilization rate exceed the preset threshold value, and automatically reduces the redundant node after the load is reduced, and the management and control platform supports the non-flow calling mode and the flow calling mode at the same time.

Description

Unified management and safety management and control platform for large model Technical Field The invention relates to the technical field of computer software and artificial intelligence, in particular to a large model unified management and safety management and control platform which is suitable for integration and management and control scenes of various large artificial intelligence models at an enterprise level. Background Along with the rapid development of artificial intelligence large model technology, the variety of large models integrated in enterprise-level application is increased, but the interface protocols and calling modes of different large models are different, so that when a business system integrates the models, development adaptation, key management and monitoring operation and maintenance are required to be carried out on each model independently, the coupling degree of the system architecture is high, and the integration and maintenance cost is increased remarkably. In the prior art, products such as an API gateway and an ESB have basic routing and auditing functions, but lack proprietary management capability aiming at a large model. The method has the following defects that 1, a traditional routing mechanism only supports fixed path forwarding, a back-end large model service cannot be dynamically and flexibly selected and switched according to application identities, contents or strategies, 2, core computing resources relied by large model reasoning cannot be deeply monitored, service performance and health state cannot be accurately evaluated, 3, a mechanism for performing special identification, interception and desensitization on sensitive information possibly related in a large model interaction process is lacking, leakage risks exist when data are exported to a third party model service, 4, an isolation mechanism is imperfect, safe, independent and customizable large model service access environments are difficult to provide for different departments, different projects or different external clients in an enterprise, 5, call log records are incomplete, persistent storage and multi-dimensional audit capability of all-link call behaviors are lacking, and industry supervision requirements are difficult to meet. Therefore, the prior art is difficult to meet the comprehensive requirements of enterprise level on unified access, safety control, resource scheduling and compliance audit of a large model, and has limitations. Disclosure of Invention The invention is carried out to solve the problems, and aims to provide a large model unified management and safety control platform. The invention provides a large model unified management and safety management platform, which is characterized by comprising an interface service module, a routing control module, a safety control module, a tenant management module, a monitoring audit module and a resource scheduling module, The interface service module is used for providing a standardized large model access entry, receiving a call request from an application end and returning a response; the routing control module stores a mapping relation between the application identifier and the target large model, and is used for routing the calling request to the corresponding target large model according to the application identifier carried by the application end and managing access credentials of the target large model; The security control module is used for carrying out identity verification, authority check, sensitive information identification and interception on the call request and carrying out security audit on the call process; the tenant management module is used for realizing the logic isolation of data, configuration and access rights among different tenants based on tenant identification; The monitoring audit module is used for monitoring the running states and resource use indexes of the platform and the target large model in real time, and storing call logs for audit tracing; the resource scheduling module is used for automatically scheduling and expanding the computing resources of the large bearing model according to the service load and the resource utilization rate. The large model unified management and safety management platform provided by the invention is also characterized in that the interface service module adopts an interface protocol compatible with a preset OPEN standard, and performs standardized encapsulation and abstract output on the capability of various large models accessed from the lower layer, wherein the preset OPEN standard is an OPEN AI application programming interface standard. In the large-model unified management and security management platform provided by the invention, the application can also be characterized in that the application is identified as an application programming interface key which is uniformly distributed and managed by the management and control platform. The unified management and safety