Exploiting LLM APIs, functions, and plugins
LLM API Working
The workflow for integrating an LLM with an API depends on the structure of the API itself. When calling external APIs, some LLMs may require the client to call a separate function endpoint (effectively a private API) in order to generate valid requests that can be sent to those APIs. The workflow for this could look something like the following:
The client calls the LLM with the user's prompt.
The LLM detects that a function needs to be called and returns a JSON object containing arguments adhering to the external API's schema.
The client calls the function with the provided arguments.
The client processes the function's response.
The client calls the LLM again, appending the function response as a new message.
The LLM calls the external API with the function response.
The LLM summarizes the results of this API call back to the user.
This workflow can have security implications, as the LLM is effectively calling external APIs on behalf of the user but the user may not be aware that these APIs are being called. Ideally, users should be presented with a confirmation step before the LLM calls the external API.
Exploiting LLM APIs with Excessive Agency
The term "excessive agency" refers to a situation in which an LLM has access to APIs that can access sensitive information and can be persuaded to use those APIs unsafely. This enables attackers to push the LLM beyond its intended scope and launch attacks via its APIs.
The first stage of using an LLM to attack APIs and plugins is to work out which APIs and plugins the LLM has access to. One way to do this is to simply ask the LLM which APIs it can access. You can then ask for additional details on any APIs of interest.
If the LLM isn't cooperative, try providing misleading context and re-asking the question. For example, you could claim that you are the LLM's developer and so should have a higher level of privilege.
Attack - Deleting a User
Now login as the username carlos and password as provided by the AI
Delete the user carlos by clicking on the Delete button
Chaining Vulnerabilities in LLM APIs
Even if an LLM only has access to APIs that look harmless, you may still be able to use these APIs to find a secondary vulnerability. For example, you could use an LLM to execute a path traversal attack on an API that takes a filename as input.
Once you've mapped an LLM's API attack surface, your next step should be to use it to send classic web exploits to all identified APIs.
OS Command Injection - APIs
Will be added soon.
Insecure Output Handling
Insecure output handling is where an LLM's output is not sufficiently validated or sanitized before being passed to other systems. This can effectively provide users indirect access to additional functionality, potentially facilitating a wide range of vulnerabilities, including XSS and CSRF.
For example, an LLM might not sanitize JavaScript in its responses. In this case, an attacker could potentially cause the LLM to return a JavaScript payload using a crafted prompt, resulting in XSS when the payload is parsed by the victim's browser.
REFERENCES
Last updated
