AI agent phishing: Proofpoint’s new defense

Email security has always been a game of cat and mouse. Viruses have been invented, and Antivirus software has been invented to catalog known viruses and detect their presence in email attachments and URLs. As viruses evolved into more sophisticated forms of malware, cybersecurity tools adapted to be able to scan and detect these new threats. Phishing became the next area, which gave rise to new tools as well as a whole new category of security, known as security awareness training. Now, The bad guys are attacking AI agents to bypass current security guardrails.

“AI assistants, co-pilots and agents significantly expand the enterprise attack surface in ways that traditional security architectures were not designed to handle,” said Todd Thieman, cybersecurity analyst at the research firm. Omdiya,

Enter a range of AI-based features for Proofpoint Prime Threat Protection Which were introduced at the company’s Proofpoint Protect 2025 event in September. They thwart hackers’ attempts to subvert the actions of AI agents by scanning for potential threats before email messages arrive in the inbox.

Traditional approach to email security

Most email security tools are designed to detect known bad signals like suspicious links, fake domains that look real, or attachments containing malware. This approach works well against traditional phishing, spam, and known exploits. But cybercriminals are now going after the many AI assistants and AI agents that have become embedded in the workplace.

They do this by taking advantage of signals (questions or commands in text or code form) that guide AI models and AI agents to either generate relevant responses or perform certain actions. Increasingly, emails contain hidden, malicious signals that use invisible text or special formatting designed to trick generic AI tools such as Microsoft Copilot And google gemini Performing insecure actions, such as exfiltrating data or bypassing security checks.

Thieman said, “Quick injection and other AI-targeted exploits represent a new class of attacks that use text-based payloads that manipulate machine logic rather than human behavior.”

Daniel Rapp, Chief AI and Data Officer proof pointAn example provided: The standard used for email messages is known as RFC-822 Explains the use of headers, plain text, and HTML. All this is not visible to the user. Attackers take advantage of this by embedding instructions in messages that are invisible to humans but completely readable by an AI agent. When AI processes text, embedded instructions get executed unintentionally. This may result in the deletion of data or may alter or corrupt the system’s behavior. There doesn’t seem to be anything wrong with legacy filters looking for malware or malformed links.

Daniel Rapp, Chief AI and Data Officer proof point,proof point

“In recent attacks we are seeing cases where the HTML and plain text versions are completely different,” Rapp said. “The email client renders the HTML version while there is an instant injection in invisible plain text that can be picked up and potentially acted upon by AI systems.”

There are two reasons why this strategy has proven effective: First, iIf the AI ​​assistant has access to the inbox, it can automatically take action on emails as they arrive. Second, Rapp said the verbose nature of AI agents makes them vulnerable to phishing and other social engineering tricks. A person may think twice about sending money to a Nigerian bank account. An AI agent can blindly follow orders to do this.

What differentiates the Proofpoint approach is that the company scans the emails before they reach the inbox. This has been practiced a lot. The company scans 3.5 billion emails every day, one third of the global total. Additionally, it scans around 50 billion URLs and 3 billion attachments daily. This is done inline, that is, while the email is traveling from the sender to the recipient.

“We have placed traceability capabilities directly into the delivery path, which means latency and efficiency are critical,” Rapp said.

This required level of speed is accomplished by training small AI models specifically on detection based on examples and foundational knowledge of larger language models (LLMs). For example, OpenAI GPT-5 is expected to be 635 billion parametersIt is not possible to use this amount of data for every email. Proofpoint has refined its models to approximately 300 million parameters. It distills and compresses its models to achieve low-latency, in-line performance without sacrificing detection fidelity. It updates those models every 2.5 days so that they are able to interpret message intent more effectively than just scanning indicators. This way, it detects hidden quick injections, malicious instructions, and other AI exploits before delivery.

“By stopping attacks before delivery, Proofpoint prevents user compromise and AI exploitation,” Rapp said. “Our secure email gateway can see emails and stop threats before they reach the inbox.”

Furthermore, Proofpoint uses an ensemble detection architecture. Instead of relying on a single detection mechanism, it combines hundreds of behavioral, reputational, and content-based signals to avoid attack vectors that could outweigh one method.

AI changes the security game

AI agents are being introduced into enterprise and consumer scenarios. Unfortunately, the rush to capitalize on AI’s potential often pushes security into an afterthought. Bad people know this. They are AI-enabling their cyber crime techniques and technologies to perfect the art of phishing for the AI ​​agent era.

“Security tooling must evolve from detecting known bad indicators to interpreting the intent of humans, machines, and AI agents,” Thieman said. “Approaches that identify malicious instructions or manipulated pre-delivery signals, ideally using distilled AI models for low-latency inline security, address a critical gap in security today.”

Proofpoint is ahead of the pack with these capabilities. Other cybersecurity vendors are expected to follow suit in the coming months. However, by that time, what other AI-generated threats will emerge?