When AI safety becomes a weapon against itself
The paradox of a world where a single word can be enough to trip up an artificial intelligence.
We have reached a rather surreal point in the evolution of artificial intelligence. A point where, to make a program hard to analyse, it may no longer be necessary to rely only on complex encryption algorithms or sophisticated obfuscation techniques. It might be enough to put into the code a series of words that the AI would rather not deal with.
This is not science fiction. It is what emerges from a recent investigation by the security researcher known as veritas, who found a JavaScript file containing references to biological weapons, toxic substances, cyberattacks and politically sensitive topics such as the Tiananmen Square massacre and Taiwanese independence.
The discovery came during a study of browser fingerprinting, the techniques websites use to identify their visitors' devices. While analysing the code of one of these systems, the researcher found functions with seemingly absurd names, referring to calculating lethal doses of toxic substances and to other dangerous activities.
Digging deeper, the researcher also found long texts hidden through special Unicode characters. Once decoded, these texts contained fake system instructions claiming to switch off the AI's safeguards, together with requests related to illegal or dangerous activities.
The obvious question is: what do biological weapons, tax evasion and Tiananmen Square have to do with a program that is supposed to simply collect information about a browser?
One possible answer is as simple as it is unsettling: those words may have been put there precisely to stop artificial intelligence from understanding the code.
The safety paradox
Modern AI models are trained to recognise and limit potentially harmful requests. It is an understandable precaution and, in many cases, a necessary one.
The problem arises when these mechanisms fail to properly distinguish between a request to carry out a dangerous action and the need to analyse content that talks about that same action.
Asking an AI for instructions to produce a biological weapon is one thing. Asking it to examine a suspicious program that contains those words and explain what it really does is another.
They are two profoundly different situations.
And yet, in the experiments described by the researcher, some OpenAI models ended up refusing the analysis, even though it had been explained that the dangerous expressions might have been inserted deliberately to provoke exactly that behaviour.
Another model, developed by the Chinese company Z.ai, reportedly completed the analysis.
This shows that not all AI systems react the same way and that the problem is not necessarily insurmountable. But it highlights an important weakness.
We have built tools capable of analysing millions of lines of code, finding vulnerabilities, recognising complex patterns and assisting security researchers.
And then we discover that, under certain circumstances, a sequence of words can be enough to bring everything to a halt.
It is hard not to see the absurdity of it.
When rules become a vulnerability
In information security there is a fundamental principle: any predictable behaviour of a system can become something an adversary exploits.
AI safety rules are no exception.
If a model tends to refuse certain categories of content, someone can deliberately use that same content to trigger a refusal.
It is not necessarily about convincing the AI to do something forbidden. Quite the opposite: in this case the presumed goal would be to stop it from doing something perfectly legitimate.
It is a reversal of the problem of so-called prompt injection, the attacks that try to manipulate a model's behaviour through instructions embedded in the data it has to process.
Here the sabotage could work without convincing the AI to obey malicious instructions at all. It would be enough to push it to stop.
In other words, you do not necessarily need to break the safety rules. It might be enough to turn them against the system itself.
And that changes the perspective radically.
Because if tomorrow software analysis tools rely more and more on artificial intelligence, anyone who wants to obstruct an investigation could try to put into their code elements designed specifically to trigger these restrictions.
Not to make the program incomprehensible to a human being, but to make it problematic in the eyes of a machine.
A new form of obfuscation
Traditionally, code obfuscation is meant to make it hard to understand how a program works.
You can rename variables, needlessly complicate instructions, hide strings or turn simple operations into sequences that look incomprehensible.
The goal is to hinder anyone trying to reconstruct the software's logic.
With the arrival of AI systems capable of analysing code, this picture is changing.
A language model can often identify structures and reconstruct meaning even when the program has been deliberately made hard to read.
But precisely because these AI systems do not just interpret machine instructions, and also assign meaning to the natural language found in the code, a new possibility for manipulation opens up.
Code becomes, at the same time, a set of instructions for the computer and a message addressed to the AI that is analysing it.
And this is where the phenomenon becomes particularly interesting.
We are entering an era in which software may be designed not only to be executed by machines and read by humans, but also to influence the behaviour of the AI systems tasked with studying it.
It is almost a form of social engineering applied to artificial intelligence.
Except that, instead of manipulating a person's emotions or beliefs, it tries to exploit a model's constraints and decision making.
The risk of confusing content with intent
There is, however, an even deeper question.
A word, on its own, should not automatically determine how dangerous a request is.
The word "poison" can appear in a toxicology handbook, a crime novel, a court document or a suspicious computer program.
Its meaning depends on the context.
In the same way, talking about a cyberattack does not necessarily mean wanting to carry it out. It can mean wanting to understand it, prevent it or counter it.
If an AI cannot properly distinguish these contexts, its safeguards can have unintended consequences.
The result risks being paradoxical: tools designed to increase security could end up hindering the very people who work to provide it.
A researcher analysing malware, an investigator studying compromising documents or a specialist hunting for a vulnerability should not be unable to work simply because the material under examination contains sensitive topics.
Security cannot be reduced to automatic censorship of words.
It should be based on understanding the purpose, the context and the consequences of a request.
Of course, drawing these distinctions is not always easy. There are ambiguous situations, and information that, even when requested for apparently harmless purposes, could be used to cause real harm.
But that very complexity shows how important it is to develop systems capable of more sophisticated judgement than simply recognising sensitive content.
A world learning to manipulate AI
For years we have wondered how well artificial intelligence would come to understand human beings.
Today the opposite question is emerging too: how good will human beings become at understanding and manipulating the behaviour of artificial intelligence?
The more these systems enter our daily activities, the greater the interest in exploiting their features, limits and vulnerabilities.
Not necessarily to obtain forbidden answers.
Sometimes, simply, to stop them from answering.
This is perhaps the most surprising aspect of the whole story.
We are used to imagining attacks on AI as attempts to bypass safeguards to obtain information that should not be provided.
But there is also the opposite problem: using those safeguards to stop AI from doing things it should be able to do.
A kind of sabotage through safety itself.
The potential consequences go well beyond code analysis.
If automated systems are used to examine documents, detect fraud, classify reports or assist investigations, it will become essential to prevent the material under analysis from manipulating the decisions of the tools tasked with interpreting it.
A document under examination should not be able to give orders to the system examining it.
A suspicious program should not be able to decide which parts of itself may be analysed.
And a sequence of words should not automatically gain the power to block a legitimate investigation.
Between hypothesis and reality
One point is important to make clear: in the case discovered by veritas, it has not been definitively proven that the authors of the code meant to obstruct artificial intelligence.
The program also performed ordinary anti-bot functions, and the identity of the developers, as well as their real intentions, has not been established.
The presence of sensitive words and hidden instructions is an interesting clue, not conclusive proof of their purpose.
At the same time, the experiments show that certain content can indeed interfere with the analysis carried out by some models.
The phenomenon therefore deserves attention regardless of the intentions of whoever wrote that specific program.
It does not mean that all AI systems are vulnerable in the same way, nor that a few forbidden words are enough to block any analysis.
It means, rather, that there is a new attack surface to study and understand.
And ignoring it would be a mistake.
The real challenge is not just making AI smarter
This story tells us something bigger than a technical curiosity.
It tells the contradictions of a technology that is becoming ever more powerful, but has yet to find a balance between operational capability, reliability and safety restrictions.
Safeguards are necessary. Nobody should want systems with no limits at all, especially when they can facilitate genuinely harmful activities.
But a safeguard that can be exploited to prevent a legitimate check risks becoming a vulnerability itself.
The solution is not to scrap the rules wholesale, but to design AI capable of recognising the difference between data to analyse and instructions to follow, between dangerous content and a dangerous request, between describing an illegal activity and attempting to carry it out.
It is a fundamental distinction, especially in a future where these systems may be asked to examine enormous amounts of information without constant human supervision.
The real challenge, then, will not only be building ever smarter AI.
It will be building AI that cannot easily be manipulated through the very rules designed to make it safe.
Because the paradox we are witnessing is hard to ignore: we have created machines capable of understanding language, and now we are starting to discover that language can also be used to stop them from understanding.
And perhaps this is the question that should worry us most.
Not only how much power we will be willing to hand over to artificial intelligence, but how vulnerable a world will become when it depends on systems whose behaviour can be influenced by content designed specifically to confuse them.
Because when a system's safety can be turned into a weapon against the system itself, the problem is no longer only about technology.
It is about the trust we decide to place in it.