Over the years, the danger actors have used social engineering to trick employees to help stealing corporate data. Now a cyber security firm has found an AI agent or chatbot a way to bypass its safety security.
What is new that the agent goes through the cloud server reveals the stolen data, not through the agent.
The search was Redware created by researchers Given what they call shadolec vulgarity in the deep research module of Open AI chat.
The strategy involves sending an email to a victim on Gmail including hidden instructions for chat. This is called an indirect early injection attack. The hidden instructions include ways to move around the safety safety of the chatgpt.
The instructions can be hidden using small fonts, white-on-white texts, or formatted metadata, and it may include the signal to compile the list of name and credit card numbers in this user’s email inbox, encoded the results in base 64 and send them to this URL “. Encoding steps are important to disrupt copied data.
AI agents include some safety measures to exploit them in this way, but in hidden instructions “the failure of the final stage will result in the deficiencies of the report,” such as cheating the agent regardless of the instructions to the agent.
The novel that the redware says is that sensitive and private data can be leaked directly from Openai’s server, without funnel through the Chatgpt client. The underlying browsing tool of the agent performs autonomy, without the participation of any customer. Redware says, other prompt-injection attacks are client-side leaks, where the agent presents an attacker-controlled material (eg image) in the user’s interface, when exfILTION is triggered.
‘Almost impossible detection’
“Our attack makes the danger surface wide,” says the redware report. “Instead of relying on what the client displays, it exploits what is motivated to execute the backand agent.
Redware says, the data leak “It is almost impossible to find out by the affected organization.”
Redware told OpenAII about vulnerability, and it was decided before today’s announcement. Pascal GeinensThe director of the Redware of Cyber Threat Intelligence said that after the fix was implemented, his firm carried out several forms of his attack and found that he had been reduced. There is no evidence that this vulnerability was being exploited in the wild before being decided by Openai.
However, he told CSOONLINE, the strategy can work with other AI agents, not only through Gmail. It can work with any AI agent that links to the data source.
“I could imagine bad actors that they were casting a large mesh by sending a common email with an embedded command to exfiltrate sensitive information,” said. “Since it is an AI agent, once you can trick it to believe it, you can ask it to do a lot. For example, anyone (chatgate) can ask the agent if it is running as a deep research. If this is, ask the agent if it is accessible to GITHUB resources and if it does it, then all the APIs make it, then all the APIs make it to the secret key and if it does it, then all the API is a list of the secret key and if it does it. Post on the website.
“The difficulty of removing AI is to create sufficient urgency and reliable references (in hidden instructions) to convince AI that he is not harming anything. Originally, (this) Social Engineering Artificial Intelligence.”
Shadolec vulnerability tests used Gmail. However, Geenens said, the initial attack vector can be anything analyzed by the AI agent. The Chatgpt offers a connector for Gmail, Google Calendar, Outlook, Outlook Calendar, Google Drive, SharePoint, Microsoft Teams, Github and more.
This week, he said, Openai announced a new beta feature that allows any MCP (Model Reference Protocol) server to add to Chatgpt as a source or tool. He said, “It opens the agent to reach one of several tens of thousands of communities and the seller provided the MCP server as a source, which creates a new huge danger surface to the MCP server’s supply chain attacks,” he said.
Other researchers have also discovered zero-click quick injection weaknesses including ecolak and agentflair. The difference, Geenens said, the data with shadolec was leaked from the infrastructure of Openai and not from the client device running Chatgpt.
What should CSO do
To blunt such attack, he said that CSO needs:
- Conduct AI agents as privileged actors: Apply the same rule used for humans with internal resource access;
- ‘Read’ from ‘Act’ scope and service accounts, and where possible LLM (large language model) should clean input before ingestion. Strip/neutralize hidden html, level safe text when possible;
- Instrument and log AI agent actions. To capture each tool call/web request and enable forensic traceability and deity;
- AI assumes indications for agents are incredible inputs. Traditional Regax/state-machine detector will not firmly capture malicious signs, so use the cementic/LLM-based intentions;
- Apply the supply-chain regime. Sellers are required to testing quick-in-injection flexibility and hygiene; Include this requirement in questionnaires and contracts;
- There is a maturity model for autonomy. Start the AI agent only with the Authority, then graduate for supervised tasks after safety reviews, perhaps making a popup that asks, “Are you sure you want me to submit XXX on this server?”. Red-team with zero-click indirect quick injection playbook before scale-out.
‘A real issue’
Joseph SteinburgA US-based cyber security and AI expert said that this type of attack is “a real issue for those parties that allow AIS to automatically process their emails, documents etc.”
This malicious voice is like Prompt Embeding that can be done with Amazon’s Alexa, he said. “Of course,” he said, “If you keep your microphone closed on your Alexa devices, when you are using them, the problem is minimized. The same is right here. If you only allow emails that you know that you are safe to be processed by AI, you can send all emails. You can send AI, Adi that at the same time, we should not recognize at the same time, we should not have anything in the same time Which guarantees any and all harmful signals from reaching AI by nefarious parties.
Steinberg also said that while AI is to live here and its use will continue to expand, CSOs who understand cyber security issues and are concerned about weaknesses are already delaying the implementation of some types of functions. Therefore, he said, it is difficult to know if the specific new vulnerability discovered by Redware will inspire many CSOs to change their views.
“He said,” he said, “Redware has clearly shown that many of us in the cyber security profession are warning, real – and anyone who is rejecting our warning should pay attention as a fear of paranoid alarmists.”
“CSO should be very concerned about this type of vulnerability,” Johannes UlrichDean of Research at SANS Institute said about the redware report. “It is very difficult if it is not impossible to patch, and many similar weaknesses are still waiting for the discovery. AI is currently at the stage of blocking specific feats, but still far from finding ways to eliminate real vulnerability. The issue will be worse as the agent AI is applied more and more.”
There have been many similar or uniform weaknesses in the AI system recently, he told that he told that Struker from blog And Objective security,
The problem is always the same, he said: AI systems do not differentiate properly between user data and code (“Prompt”). This allows for the innumerable paths to modify the prompt used to process data. It is a mixture of original patterns, codes and data, he said, the root cause of most security weaknesses in the past, such as buffer overflow, SQL injection and cross-site scripting (XSS).
‘wake up call’
Redware generation said, “Shadolec” is a wakeup call that does not jump with security in AI. “” Organizations will have to use this technology. There is no doubt in my mind that AI will be an integral part of our life in the near future, but we need to tell organizations that it needs to be done safely and make them aware of the dangers. “
He said, “I wake up at night,” he said, “there is a conclusion from a Gartner report4 ways will influence generous AI Sisos and his teams ) Which was published in June 2023 and is based on a survey about Jeanai: ‘89% of business technologists will bypass cyber security guidance to meet a commercial purpose. ‘If the organizations first jump in this technique and believe after safety, it will not end well for the organization and technology. This is our work or mission, as a cyber security community, to make organizations aware of risks and come up with friction -free security solutions that enable them to deploy agent AI safely and productively. ,
