Google has decided not to fix a new ASCII smuggling attack in Gemini, which could be used to provide fake information to an AI assistant, change the behavior of models, and quietly poison your data.
ASCII smuggling is an attack where special characters from the tagged Unicode block are used to introduce payloads that are invisible to users, but can still be detected and processed by large language models (LLMS).
This is similar to other attacks that researchers have recently presented against Google Gemini, all of which exploit a difference between what users see and what the machine reads, such as manipulating CSS or exploiting GUI limitations.
While the susceptibility of LLMS to ASCII smuggling attacks is not a new finding, as several researchers have explored this possibility since the advent of generative AI tools, the level of risk is now different (1, 2, 3, 4,
Previously, chatbots could only be maliciously manipulated by such attacks if specially crafted prompts were pasted to the user. With the rise of agent AI tools like Gemini, which have extensive access to sensitive user data and can perform tasks autonomously, the threat is more significant.
Victor Markopoulos, a security researcher at the company Firetail Cybersecurity Tested ASCII Smuggling against several widely used AI tools and found that Gemini (calendar invites or emails), DeepSec (signals), and Grok (x posts), are vulnerable to the attack.
Cloud, CHATGPT, and Microsoft Copilot proved to be secure against ASCII smuggling, implementing some form of input sanitization, Firetail found.
Source: Firetail
Regarding Gemini, its integration with Google Workspace poses a high risk, as attackers can use ASCII smuggling to embed hidden text in calendar invites or emails.
Markopoulos found that it is possible to hide instructions on calendar invite titles, overwrite organizer details (identity spoofing), and smuggle hidden meeting details or links.
Source: Firetail
Regarding the risk from emails, the researcher says that “For users with LLMs linked to their inboxes, a simple email with hidden commands could instruct the LLMs to search the inbox for sensitive items or send contact details, turning a standard phishing attempt into an autonomous data extraction tool.”
LLMS instructed users browsing websites may also stumble upon payloads hidden in product descriptions and feed them with malicious URLs to trick users.
The researcher reported the findings to Google on September 18, but the tech giant dismissed the issue as not being a security bug and could only be exploited in the context of social engineering attacks.
Nevertheless, Markopolos showed that the attack could trick Gemini into supplying false information to users. In one example, the researcher passed an invisible instruction that Gemini processed to introduce a potentially malicious site as the place to get a good quality phone with a discount.
Other tech firms, however, have a different take on these types of problems. For example, Amazon published detailed safety guidance On the subject of Unicode character smuggling.
BleepingComputer has contacted Google for further clarification on the bug, but we have not yet received a response.
attend Breach and Attack Simulation Summit and experience future of security verificationHear from top experts and see how AI-powered base Changing the breach and attacking simulation.
Don’t miss the event that will shape the future of your security strategy
