CN-122029538-A - Identifying Application Programming Interface (API) vulnerabilities using Hierarchical Reinforcement Learning (HRL)

Abstract

The computer-implemented method according to various methods includes causing a predetermined HRL agent to perform execution of a first potentially malicious API call to a predetermined API in a controlled environment. Executing the execution of the first potentially malicious API call to the predetermined API includes causing a first subagent of a first level of architecture of the HRL agent to make a first selection defining a first portion of the first API call. Executing the first potentially malicious API call to the predetermined API further includes causing a second subagent of a second level of the architecture of the HRL agent to make a second selection defining a second portion of the first API call, and issuing the first API call to the predetermined API. The first subagent and the second subagent are provided with a first reward-based feedback.

Inventors

• O. Brind
• O. Y. Bohm
• M. G. muffle

Assignees

• 国际商业机器公司

Dates

Publication Date

20260512

Application Date

20240905

Priority Date

20231012

Claims (20)

1. 1. A computer-implemented method, comprising: Causing a predetermined Hierarchical Reinforcement Learning (HRL) agent to perform execution of a first potentially malicious API call to a predetermined API in a controlled environment, wherein performing execution of the first potentially malicious API call to the predetermined API comprises: causing a first subagent of a first level of architecture of the HRL agent to make a first selection defining a first portion of a first API call, and Causing a second subagent of a second level of the architecture of the HRL agent to make a second selection defining a second portion of the first API call, and Issuing the first API call to the predetermined API, and Providing first reward-based feedback to the first sub-agent and the second sub-agent.
2. 2. The computer-implemented method of claim 1, wherein the first level of the architecture of the HRL agent comprises a plurality of sub-agents, wherein the first portion of the first API call defines a selected method for a first micro-service call.
3. 3. The computer-implemented method of claim 2, wherein the second level of the architecture of the HRL agent comprises a plurality of sub-agents, wherein the second portion of the first API call defines selected parameters for invoking the selected method.
4. 4. The computer-implemented method of claim 3, wherein performing execution of the first potentially malicious API call for the predetermined API comprises causing a third subagent of a third level of the architecture of the HRL agent to make a third selection for defining a third portion of the first API call, wherein the third level of the architecture of the HRL agent comprises a plurality of subagent, wherein the third portion of the first API call defines the first micro-service from a plurality of potential micro-services.
5. 5. The computer-implemented method of claim 3, wherein the selected parameter is selected from the group consisting of video-id, token, user-id, and vehicle-id.
6. 6. The computer-implemented method of claim 3, wherein each of the children of the second level of the architecture of the HRL agent is configured to define a single parameter from a plurality of different parameters.
7. 7. The computer-implemented method of claim 3, wherein the second sub-agent of the second level of the architecture of the HRL agent is configured to define more than one parameter from a plurality of different parameters.
8. 8. The computer-implemented method of claim 1, comprising causing the predetermined HRL agent to simulate execution of a plurality of potentially malicious API calls to the predetermined API in the controlled environment, wherein execution of the first potentially malicious API call to the predetermined API comprises monte carlo simulation.
9. 9. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions being readable and/or executable by a computer to cause the computer to: Causing a predetermined Hierarchical Reinforcement Learning (HRL) agent to perform execution of a first potentially malicious API call to a predetermined API in a controlled environment, wherein performing execution of the first potentially malicious API call to the predetermined API comprises: causing a first subagent of a first level of architecture of the HRL agent to make a first selection defining a first portion of a first API call, and Causing a second subagent of a second level of the architecture of the HRL agent to make a second selection defining a second portion of the first API call, and Issuing the first API call to the predetermined API, and Providing first reward-based feedback to the first sub-agent and the second sub-agent.
10. 10. The computer program product of claim 9, wherein the first level of the architecture of the HRL agent comprises a plurality of sub-agents, wherein the first portion of the first API call defines a selected method for a first micro-service call.
11. 11. The computer program product of claim 10, wherein the second level of the architecture of the HRL agent comprises a plurality of sub-agents, wherein the second portion of the first API call defines selected parameters for invoking a selected method.
12. 12. The computer program product of claim 11, wherein effectuating execution of the first potentially malicious API call for the predetermined API comprises causing a third subagent of a third tier of the architecture of the HRL agent to make a third selection for defining a third portion of the first API call, wherein the third tier of the architecture of the HRL agent comprises a plurality of subagent, wherein the third portion of the first API call defines the first micro-service from a plurality of potential micro-services.
13. 13. The computer program product of claim 11, wherein the selected parameter is selected from the group consisting of video-id, token, user-id, and vehicle-id.
14. 14. The computer program product of claim 11, wherein each of the children of the second level of the architecture of the HRL agent is configured to define a single parameter from a plurality of different parameters.
15. 15. The computer program product of claim 11, wherein the second sub-agent of the second level of the architecture of the HRL agent is configured to define more than one parameter from a plurality of different parameters.
16. 16. The computer program product of claim 9, the program instructions being readable and/or executable by a computer to cause the predetermined HRL agent to simulate execution of a plurality of potential malicious API calls to the predetermined API in the controlled environment, wherein execution of the first potential malicious API call to the predetermined API comprises monte carlo simulation.
17. 17. A system, comprising: Processor, and Logic integrated with, executable by, or integrated with the processor and executable by the processor, the logic configured to: Causing a predetermined Hierarchical Reinforcement Learning (HRL) agent to perform execution of a first potentially malicious API call to a predetermined API in a controlled environment, wherein performing execution of the first potentially malicious API call to the predetermined API comprises: causing a first subagent of a first level of architecture of the HRL agent to make a first selection defining a first portion of a first API call, and Causing a second subagent of a second level of the architecture of the HRL agent to make a second selection defining a second portion of the first API call, and Issuing the first API call to the predetermined API, and Providing first reward-based feedback to the first sub-agent and the second sub-agent.
18. 18. The system of claim 17, wherein the first level of the architecture of the HRL agent comprises a plurality of sub-agents, wherein the first portion of the first API call defines a selected method for a first micro-service call.
19. 19. The system of claim 18, wherein the second level of the architecture of the HRL agent comprises a plurality of sub-agents, wherein the second portion of the first API call defines selected parameters for invoking a selected method.
20. 20. The system of claim 19, wherein performing execution of the first potentially malicious API call for the predetermined API comprises causing a third subagent of a third level of the architecture of the HRL agent to make a third selection for defining a third portion of the first API call, wherein the third level of the architecture of the HRL agent comprises a plurality of subagent, wherein the third portion of the first API call defines the first micro-service from a plurality of potential micro-services.

Description

Identifying Application Programming Interface (API) vulnerabilities using Hierarchical Reinforcement Learning (HRL) Background The present invention relates to Application Programming Interfaces (APIs), and more particularly, to identifying API vulnerabilities. APIs are basic elements in an application driven environment. These APIs are important for modern mobile, software as a service (SaaS), and web applications across various industries, such as retail, transportation, and banking. Further, APIs are used for both client-oriented applications and internal applications, such as internet of things (IoT), autonomous vehicles, and smart cities. APIs provide a relatively powerful interface for developers to access services that an organization must provide. Ensuring that APIs conform to published specifications and are resistant to potentially malicious inputs is critical to the overall security of the organization. APIs act as intermediaries between machines and typically involve clients, servers, and resources. As context, a "resource" may be any content or data, such as text files, videos, etc., that a server may provide to a client. The resources may include sensitive data such as personally identifiable information (Personally Identifiable Information, PII), financial information, business related valuable data, etc., and thus such resources are often targets of an attacker. The importance of ensuring API security has led to the development of previous API testing tools. Disclosure of Invention According to various methods, a computer-implemented method includes causing a predetermined hierarchical reinforcement learning (HIERARCHICAL REINFORCEMENT LEARNING, HRL) agent to perform execution of a first potentially malicious API call to a predetermined API in a controlled environment. Executing the execution of the first potentially malicious API call to the predetermined API includes causing a first subagent of a first level of architecture of the HRL agent to make a first selection defining a first portion of the first API call. Executing the first potentially malicious API call to the predetermined API further includes causing a second subagent of a second level of the architecture of the HRL agent to make a second selection defining a second portion of the first API call, and issuing the first API call to the predetermined API. The first subagent and the second subagent are provided with a first reward-based feedback. The computer program product according to the various methods includes a computer readable storage medium having program instructions embodied therein. The program instructions may be read and/or executed by a computer to cause the computer to perform any combination of the features of the foregoing method. A system according to various methods includes a processor and logic integrated with, executable by, or both with and executable by the processor. The logic is configured to perform any combination of the features of the foregoing methods. Other aspects and methods of the present invention will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, illustrating by way of example the principles of the invention. Drawings FIG. 1 is a diagram of a computing environment in accordance with one method of the present invention. Fig. 2A is a flow chart of a method according to one method of the present invention. FIG. 2B is a flow chart of sub-operations of the flow chart of FIG. 2A in accordance with one method of the present invention. FIG. 2C is a flow chart of sub-operations of the flow chart of FIG. 2A in accordance with one method of the present invention. Fig. 3 is an architecture of an HRL agent according to one method of the invention. Fig. 4 is an architecture of an HRL agent according to one method of the invention. Fig. 5A is an architecture of an HRL agent according to one method of the invention. Fig. 5B is a sample of code generated by the architecture of fig. 5A in accordance with one method of the present invention. Fig. 6 is a graph of a method according to the present invention. Fig. 7 is a graph of a method according to the present invention. Fig. 8 is a graph of a method according to the present invention. Fig. 9 is a flow chart of a method according to one method of the present invention. Detailed Description The following description is made for the purpose of illustrating the general principles of the present invention and is not meant to limit the inventive concepts claimed herein. Furthermore, the specific features described herein can be used in combination with other described features in each of the various possible combinations and permutations. Unless specifically defined otherwise herein, all terms are to be given their broadest possible interpretation, including the meaning implied from the specification as well as the meaning understood by those skilled in the art and/or defined in dictionaries, textbooks,