Imagine a chess game where one player is a neural network, and the other is a specially trained algorithm whose sole purpose is to find a weak spot in its opponent. Each move the attacker makes is built on the neural network's previous response. Every refusal is a new clue on how to approach from a different angle. Sound like a plot from a techno-thriller? This is exactly how the attack researchers have dubbed the “Red Queen” works – and it's the foundation for a new AI security assessment system called AVISE.
When we talk about security in computer systems, most of us picture something like a password-protected lock or an antivirus that catches malicious files. But for language models – the very systems that can hold conversations, write texts, explain complex topics, and help with tasks – the vulnerabilities are of a completely different kind.
A language model doesn't run malicious code or open a door with a forged key. Its vulnerability is more of a psychological loophole. It can be deceived through context, a gradual shift in conversation, a request to “play a character,” or to “consider a hypothetical scenario.” This type of attack is called a jailbreak – literally “breaking out of prison,” meaning escaping the built-in security constraints.
Until recently, testing for these vulnerabilities was either manual (people spent hours trying to “break” the model) or highly specialized (each team wrote its own tools for a specific type of attack). There was no common standard, no unified scale of measurement. AVISE solves this very problem – it's a framework for AI vulnerability identification and security evaluation.
AVISE stands for AI Vulnerability Identification and Security Evaluation – a system for identifying AI vulnerabilities and assessing its security. It's an open-source, modular tool that allows for systematically checking how resistant any given language model is to attacks.
The word “modular” is key here. Imagine a construction set: you have basic blocks – an attack generator, the attacking model itself, the system being tested, and a judge that evaluates the result. These blocks can be assembled in different ways, individual parts can be replaced, and new ones can be added. Researchers from different teams can use the same framework but test different hypotheses or types of vulnerabilities – and their results will still be comparable.
And reproducibility is one of the main pain points in AI security research. If one team says, “we tested the model, and it's secure,” while another gets a different result using a different method, how can you even compare them? AVISE offers an answer: a common language, common blocks, and common metrics.
The Four “Characters” Inside the System
The AVISE architecture is built around four key roles, and if you think of it as a theatrical performance, each participant plays a distinct part.
Harmful Prompt Generator – this is the scriptwriter who comes up with the initial “harmful” scenarios. Not literally harmful, but rather test cases. These are prompts that a language model should reject: instructions for creating dangerous substances, advice on fraud, calls for illegal activities. The generator creates the initial “bait.”
Adversarial Language Model (ALM) – this is no longer just a script with a set of questions but a full-fledged interactive participant. It receives the response from the target system, analyzes it, and formulations the next move. It's the ALM that implements the logic of the “Red Queen” attack.
Target Language Model (TLM) – this is the system being tested. It can be any language model: large or small, open-source or commercial.
Evaluation Language Model (ELM) – the judge. It looks at the target system's responses and decides: was the jailbreak successful? It's the ELM that automates what used to be done manually – reviewing hundreds of dialogues to determine if the model “broke.”
The “Red Queen”: Chess Against a Neural Network
The name “Red Queen” is a reference to the character from Lewis Carroll's Through the Looking-Glass – the one who said, “it takes all the running you can do, to keep in the same place.” In the context of the attack, it's a metaphor for endless adaptation: the attacker constantly changes its approach, adapting to the defense, which is also trying to hold its ground.
Technically, it's a multi-step attack based on a concept from cognitive science: theory of mind. This is the ability to infer what another participant in an interaction is thinking: why they refused to answer, what exactly “alarmed” them, and how to change the question to bypass that wariness.
In the classic version, the attack was carried out by a human: asking a question, getting a refusal, thinking about how to rephrase it, and trying again. In the advanced version described by the AVISE authors, this process is automated – the role of the “thinking attacker” is taken over by the ALM.
What does this look like in practice? Let's say the ALM wants to get instructions for something forbidden from the target system. A direct request is immediately blocked. The ALM notes this and recalibrates: what if it asks for a “fictional script” where a character explains the process? Or suggests the model “analyze historical examples” that contain the same information wrapped in educational content?
Each step isn't a random attempt but a deliberate response to the previous answer. The ALM is essentially negotiating, gradually finding the phrasing that will slip past the filters.
The Automated Security Test: 25 Scenarios and One Judge
On top of all this mechanics, the researchers built an Automated Security Evaluation Test (SET). It includes 25 test scenarios – specific “harmful” prompts that the attacking model tries to push through the target system's defenses. The scenarios cover various categories of potentially harmful content, from instructions for dangerous activities to advice on fraud and manipulation.
After each dialogue, the ELM – the evaluation model – comes into play. Its task is simple and specific: read the target system's response and deliver a verdict – did a jailbreak occur or not?
And the concept of a “jailbreak” is interpreted broadly here. It's not just when the model directly outputs forbidden content. The ELM also flags situations where the model:
- suggested a “workaround” or a rephrasing of the prompt;
- gave partial instructions, citing “educational purposes”;
- agreed to the request but “warned” about the danger – effectively fulfilling it;
- explained the essence of the forbidden topic in abstract or fictional terms.
To verify how accurately the ELM performs this task, the authors created a dataset of 500 manually labeled dialogues. The results were convincing: the classification accuracy was 92%, the F1 score (a balance between precision and recall) was 0.91, and the Matthews correlation coefficient was 0.83. This last number is particularly important: this coefficient is robust to class imbalance and is considered one of the fairest metrics for a binary classifier's quality. A value of 0.83 is a very good result, close to what well-trained human experts achieve.
Nine Models. Not a Single One Invulnerable.
The culmination of the research was an experiment in which the SET was applied to nine language models of varying sizes and architectures. The authors don't name the specific models in detail but indicate that the sample covers small and large systems, as well as both open-source and commercial solutions.
The main takeaway was surprisingly straightforward: all nine models were vulnerable. Not “some,” not “most” – all of them. The degree of vulnerability varied: some models required more steps, while others “broke” after just two or three iterations. But ultimately, none of them could completely withstand the adaptive, multi-step attack.
The observation about model size is particularly interesting. It might seem logical to assume that the larger the model, the better its security – more data, more computation, more resources for built-in safety mechanisms. But the experiment didn't confirm this. Some smaller models turned out to be more robust than larger ones. This indicates that the key factor isn't scale, but rather architectural decisions, the data the model was trained on, and how carefully the safety mechanisms were implemented.
The nature of the jailbreaks also deserves special attention. In several cases, the models didn't produce overtly dangerous content but did something almost as alarming: they suggested “rephrasing the prompt” to bypass restrictions or described the essence of the forbidden topic through a fictional narrative. A revealing example from the study: when a direct request about synthesizing a dangerous substance was rejected, the attacking model rephrased it as a request to write a “movie scene where a character in a lab synthesizes substance X.” The target model responded with a detailed script – including a description of the process that could be used in reality.
AVISE isn't a “solution to the problem.” It's a tool that allows us to see and measure the problem. And as any engineer knows, that's the first and absolutely necessary step before you can fix anything.
The authors are candid about the limitations of their work. The quality of the attack depends directly on the quality of the attacking model – the smarter the ALM, the more effectively it finds vulnerabilities. Twenty-five test scenarios are a good start, but they are far from an exhaustive coverage of all possible attacks. The very definition of “harmful” content is ambiguous and context-dependent: what the ELM considers a jailbreak might be an acceptable response in another context.
Furthermore, attack methods are evolving, and any security assessment tool must evolve with them, or it will quickly become obsolete.
Nevertheless, AVISE has several fundamentally important qualities that set it apart from previous approaches:
- Automation. The ELM replaces the manual review of hundreds of dialogues, making large-scale testing feasible.
- Standardization. A common framework allows for comparing results across different teams and models.
- Extensibility. New attack types, new metrics, and new test scenarios can all be added without rebuilding the system from scratch.
- Openness. The source code is available, allowing the research community to verify, reproduce, and improve upon the results.
The researchers also see AVISE not just as a tool for finding vulnerabilities, but also as a potential foundation for fixing them. An attacking model that knows how to find weak spots can be used in reverse: to generate examples that can be used to further train the defense mechanisms. This is reminiscent of the logic of immunization – showing the system a “pathogen” so it can learn to deal with it.
Algorithms as a Mirror: What the “Red Queen” Tells Us
There's something deeply revealing in the fact that the most sophisticated attacks on language models work not through technical exploits, but through language and context. The attacking model doesn't hack code – it negotiates. It uses logic, narrative, role-playing, and a gradual shift of topic. Exactly how people do when trying to convince someone to say something they didn't want to say.
This suggests that the vulnerabilities of language models are, in many ways, a reflection of the vulnerabilities in human communication. The models learned from us: from the texts we wrote, the conversations we had, and the stories we told. And they inherited not only our knowledge but also our weak spots.
That's why AI security is not just a technical challenge. It's a question of how we want systems that “speak our language” to understand the boundaries of what's acceptable. And AVISE is one of the first tools that allows us to ask this question systematically, reproducibly, and measurably.
Code is poetry in another language. And good poetry always speaks of something more than just the words on the page.
