Breaking barriers in artificial intelligence is always a thrilling challenge. As LLMs (large language models) continue to evolve, it’s becoming more difficult to bypass their programming. Anthropic’s researchers have discovered a new jailbreak method that treads into treacherous territory – conditioning an LLM to answer sensitive questions.
The team refers to this technique as “many-shot jailbreaking,” and their findings have been shared in a paper and with other AI professionals in hopes of finding a solution.
The vulnerability exploited is the extended “context window” in newer generations of LLMs. This memory capacity has grown from a few sentences to thousands of words, even whole books.
Anthropic’s study revealed that with a larger context window, LLMs can excel at tasks with ample examples within the prompt. Even trivia questions can be mastered after seeing them repeatedly. However, what surprised the researchers was the LLMs’ ability to answer inappropriate questions after being exposed to “in-context learning.” A bomb-building request would previously be denied but could be honored after being primed with 99 less-harmful questions.
Why does this occur? Despite our understanding of LLMs’ complex workings, it seems that they have a mechanism to identify and fulfill user expectations. If the user desires trivia, the model will gradually increase its wealth of trivial knowledge with each question. The same phenomenon is seen when answering distasteful inquiries.
The team promptly notified their peers, hoping that this incident would encourage open communication about potential exploits within the LLM community. On their own end, they found that restricting the context window does have some success, but it negatively impacts the model’s overall performance – a setback they cannot afford. Currently, they are developing measures to classify and contextualize queries before they are presented to the model. However, this only shifts the goalpost for hackers, a common trend in the intricate world of AI security.
Cybersecurity in artificial intelligence remains a constant battle where innovation and vigilance go hand in hand. As we evolve and improve, so do the threats we face. But with transparency and collaboration, we can stay one step ahead and continue to push the boundaries of what is possible with AI.
