Overview
Large language models (LLMs) and AI have become incredibly useful for finding and exploiting vulnerabilities in offensive security and penetration testing as well as for day-to-day usage in improving writing, generating text, and building visuals for presentations. We must also, however, consider the protection of data and our clients’ environments as well as the accuracy and integrity of our work.
This document provides guidance on how LLMs and AI should be used within Volkis and the restrictions that must be applied when using these systems.
Volkis Claude subscription
Offensive security consultants will be provided a Claude subscription in the Volkis account. This subscription will have security settings enabled:
- Inputted data is configured to not be included in model training.
- 90 day data retention policy applied.
When using the Volkis Claude subscription, please be aware that admins have access to the data you put in up to the retention policy. The data is also shared with a third party (Anthropic) who will have access to the data submitted. Because of this:
- Avoid putting in your own personal queries or information.
- Never submit Personally Identifiable Information (PII). If you encounter PII during a test, ensure it is removed prior to using the data in an LLM.
- Never use routers or proxy layers that could have the capability of manipulating data or sharing the data with a third party.
Using client data in the Volkis Claude subscription
Volkis runs an “opt-out” policy for the usage of LLMs on client engagements. By default, using the Volkis Claude subscription with client data is acceptable, however the client can ask the consultant to restrain from using LLMs with client data during the engagement. This should be established during the scoping stage of the engagement so that additional time can be provided to the consultant.
Look at the Trello card to see whether using LLMs is OK.
Using Claude to assist with exploitation and processing vulnerability data
Even if using LLMs is ok for the engagement, you should still explicitly ask the client prior to using LLMs for:
- Developing exploits specific to the client environment or application, where the information provided to the LLM could have material risk to the client.
- Using the LLM with vulnerability data, access tokens, or client passwords.
If possible, strip sensitive information such as access tokens or passwords from data inputted into the LLM.
Using Claude to assist with reporting
The general rule for Volkis reports is that if you have your name as the author of the report then you should be willing to stand by that report. While it is fine to use LLMs as part of the reporting process, you should use it with care. The final content should be accurate and should reflect your own voice, thoughts, determinations, and conclusions.
In particular, using LLMs can make the tone of reports feel stale and sanitised. While it’s possible to change the tone of an LLM’s output with the right prompt, we feel that it’s often better to rewrite the output in your own words where possible.
Safe usage of LLMs
LLMs are just a tool. You must be willing to stand by your work even if you are being assisted by an LLM. This means:
- You should check the output of the LLM for accuracy and relevance. LLMs are inherantly prone to making up information when they don’t know the answer. They are prone to giving answers that are old or for another similar system. They don’t know the client as well as you do.
- You should be aware of the habit for LLMs to agree with the user, even when the user is wrong. In general, LLMs won’t challenge the user unless they are specifically asked to.
LLMs also have technical risks for users:
- You should be aware of prompt injection and take steps to avoid risk to your system and manipulation of the output through prompt injection.
- You should ensure access control is appropriately applied to actions undertaken by an LLM.
- You should manually review any code developed by an LLM to ensure it is safe to run.
Respectful usage of LLMs
Your use of LLMs should consider the personal connections you have with your coworkers and clients. Putting an AI layer between you and your coworkers can damage and abstract that personal connection away. The last thing we would want is for communication to be filtered and manipulated through progressive layers of AI.
When using AI you should:
- Avoid generating “AI slop” and consider the general rule that “if you didn’t spend time creating it, why should they spend time reading it?” Sometimes the information that you enter into the LLM’s interface is more important than the content that it spits out.
- Clearly mark content that you send to coworkers when it is created by AI.
- Try to use your own words when communicating with your coworkers, not just the words LLMs tell you to use.
Using LLMs outside of the Volkis Claude subscription
You can use other LLMs outside of the Claude subscription, even for Volkis work, as long as you don’t include internal Volkis information or client data.