AI & Emerging Tech

Vendor Warning on AI-Driven Attacks Misdates the $25 Million Arup Deepfake

Vendor Warning on AI-Driven Attacks Misdates the $25 Million Arup Deepfake

Trellix, the US security vendor, published a blog post on 22 September arguing that AI has turned deception into an industrial process. Its headline example, a $25 million deepfake fraud against an engineering firm is dated to early 2026. The fraud hit Arup’s Hong Kong office in January 2024.

The slip matters less for the history than for what it reveals about the evidence behind the new-era framing. Of the 3 cases doing the heavy lifting, one is more than two and a half years old, one rests on a single disclosure by the AI company whose product was abused and one is Android malware its own discoverer says has barely been seen in the wild.

The Arup Video Call Is Still the Template

Hong Kong police said in February 2024 that a finance employee had joined a video conference in which every other participant including a figure presented as the chief financial officer was a deepfake. He had doubted the original email which asked for a secret transaction. After the call he made 15 transfers totalling HK$200 million, about $25.6 million, public broadcaster RTHK reported, citing police.

Arup confirmed to CNN in May 2024 that it was the victim and that fake voices and images had been used. The company said none of its internal systems were compromised.

That is the detail boards tend to miss. No malware, no breach. The attackers beat a payment process, not a network.

GTG-1002 Is One Company’s Account

Anthropic disclosed on 14 November 2025 that it had detected, in mid-September that year, a campaign by a group it designates GTG-1002 and assesses with high confidence to be Chinese state-sponsored. The group manipulated Claude Code by posing as a security firm running defensive tests, then turned it on roughly 30 organisations in technology, finance, chemicals and government. Anthropic says the AI performed 80 to 90% of the tactical work. Humans stepped in at a handful of decision points such as choosing targets and approving data exfiltration.

Anthropic also reported that the model overstated its findings and at times fabricated credentials and that only a small number of intrusions succeeded. Neither detail appears in the Trellix post which describes the campaign as running into 2026. Anthropic’s report describes detecting and disrupting it in 2025.

The attribution comes from the vendor whose product was abused and no independent forensic analysis has been published. Treat it as a credible single-source account, not an established fact.

PromptSpy Barely Left the Lab

ESET disclosed PromptSpy on 19 February 2026 describing it as the first known Android malware to use generative AI in its execution flow. It sends Google’s Gemini an XML description of what is on screen and receives tap and swipe instructions that pin the malicious app in the recent-apps list. Its real job is to deploy a VNC module that gives operators remote control of the phone.

ESET researcher Lukáš Štefanko told Computer Weekly the malware had not appeared in ESET’s wider telemetry and may be a proof of concept. It was never on Google Play. ESET linked the campaign to financial fraud against users in Argentina and made no state attribution. The Trellix post links it to Chinese threat actors.

Removal requires rebooting into Safe Mode because the malware draws invisible overlays over the uninstall buttons.

A Face on a Video Call Is Not an Approval

None of these cases calls for a new product. In Arup’s case, the employee set aside his doubts because the people on the call looked and sounded like colleagues.

  • Call back on a number already on file, never one supplied in the request before any urgent or confidential transfer.
  • Require a second authorised approver above a fixed threshold with no exemption for requests that claim to come from the CEO or CFO.
  • Restrict Accessibility Service permissions through mobile device management and block sideloading on company phones.
  • Watch for the patterns Anthropic describes in GTG-1002: bursts of automated requests, rapid credential testing and reconnaissance across many hosts at once.

Then test the first rule. Have a senior colleague request an urgent transfer by video this week and see whether finance picks up the phone.

References

  1. The Industrialization of Deception: AI-Driven Threats
  2. AI Attacks Enter New Era as Hackers Use Deepfakes and Automation to Scale Deception
  3. Disrupting the first reported AI-orchestrated cyber espionage campaign
  4. ESET Research Discovers PromptSpy, the First Android Threat to Use Generative AI
  5. PromptSpy Ushers in the Era of Android Threats Using GenAI
  6. PromptSpy Android Malware May Exploit Gemini AI
  7. British Engineering Giant Arup Revealed as $25 Million Deepfake Scam Victim

This post is also available in: Svenska

Erik Berg

Erik Berg is CTO and Principal Security Architect at eBuilder Security, with more than a decade in blue team security operations across the private and public sectors, and a focus on emerging threats including the security risks that come with AI.