As Artificial Intelligence (AI) becomes increasingly integral to daily operations, ensuring its security is paramount. Testing the security of Large Learning Models (LLMs) has evolved into a complex task, necessitating new tools and playbooks that are still under development. A common question we encounter is: What are the known risks to AI today, and how do we evaluate them? While AI models can be exploited in various ways, we will call out three major areas: bad data, data leaks, and dangerous outputs.

Bad Data
AI models can produce unknown or undesired responses, often referred to as “hallucinations.” These hallucinations become particularly risky when accuracy is crucial. For instance, in scenarios where critical decisions depend on precise AI responses, hallucinations can have significant negative impacts. Users must verify data to avoid distributing inaccurate information or basing important decisions on potentially inaccurate data.

Data Leaks
AI models that combine distinct datasets and link metadata can inadvertently expose protected information. This is a significant concern for organizations with substantial proprietary data, as they rely on AI for quick reference and response. For instance, loading patient data into an LLM for analysis on a specific condition could potentially allow others unauthorized access to the patient data used in the model.

Dangerous Outputs
Another significant security concern involves dangerous outputs. These can include using AI to generating malicious scripts or payloads, or potentially posing unethical questions to receive equally unethical responses. Such outputs pose serious risks and must be evaluated carefully by LLM developers.

Preventing Security Risks
To mitigate these risks, models and implementations must be rigorously tested from the outset. This process is akin to social engineering, where red teamers attempt to bypass guardrails and controls to elicit sensitive data and responses from a user (or in this case an AI implementation). Similar to social engineering attacks, where pieces of information are gathered over time to form a complete picture, AI evaluation requires a step-by-step approach to fully test it. Each piece of knowledge gained should inform subsequent tests. For example, in the case of user social engineering, it may take several calls to a help desk to learn what the challenge response questions are before a password is reset. However, after combining the knowledge gained on each call, an attacker can create a full picture of what is needed to perform a successful attack. The same general approach should be taken when evaluating AI. Each step or knowledge gained should be used to build upon for the next test. Sometimes it will require knowledge sets outside of a typical red teamer’s knowledge. It may require knowledge in distinct subject areas which could require working alongside technical experts. Sadly, this may not be enough. There are regular disclosures of new techniques to subvert protections and guardrails to prove testing is still coming up short. Click here for an example of these disclosures.

The Role of AI in Testing AI
Given the complexity of AI capabilities, the most reliable way to test AI may be using AI itself. The security industry must develop robust and trusted AI audit engines capable of assessing the scope of the AI being tested and identifying ways to circumvent existing controls. Just as diamonds are used to cut diamonds, AI will be essential in testing AI. Until then, the best approach involves leveraging new perspectives and incorporating non-traditional resources into evaluations.
If you need assistance evaluating your AI model or understanding the risks before advancing with AI, contact us. We are here to help.