The Basics of AI-Powered Social Engineering
AI-powered social engineering is the use of artificial intelligence to manipulate people into handing over money, credentials or access. Attackers use generative AI to write convincing messages, clone voices and fake video calls so the old warning signs of a scam mostly disappear. It is social engineering made cheaper, faster and far more believable.
Social engineering itself is not new. What is new is the tooling. The UK National Cyber Security Centre (NCSC) judged in January 2024 that AI will almost certainly increase the volume and heighten the impact of cyber attacks and that the biggest gain for attackers is in social engineering. In plain terms, a trick that once needed a skilled fraudster and several days now takes minutes and reads perfectly.
How AI Changes Social Engineering
Older scams often gave themselves away. Clumsy grammar, an odd greeting or a translation error was usually enough to make someone pause. Generative AI removes those tells. The NCSC notes that AI can produce fluent and personalised lures without the spelling and grammatical mistakes that used to reveal phishing.
It also changes the economics of an attack. In a controlled study published in 2024, researchers led by Fred Heiding at Harvard found that fully AI-automated spear-phishing emails achieved a 54 percent click-through rate, matching human experts and far above the 12 percent rate for generic mass phishing. The AI did the background research and wrote each message at a fraction of the usual cost.
The same leap has reached audio and video. The FBI warns that criminals can clone a usable voice from a short audio clip and can run real-time video calls that impersonate an executive. Current research shows as little as a few seconds of recorded speech can be enough. An AI chatbot can also hold a convincing conversation for weeks which is what powers long-running romance and investment scams.
Types of AI-Powered Social Engineering Attacks
AI-powered social engineering takes several forms. These are the main ones to know.
- AI phishing and spear-phishing: Mass emails and targeted messages written by AI to look personal and legitimate at a scale no human team could match.
- Voice cloning and vishing: A cloned voice built from a short audio sample used in calls or voicemails to impersonate a manager, a colleague or a family member.
- Deepfake video: A faked live video call or recording that puts a trusted face, often a senior executive, behind a fraudulent request.
- AI chatbots for long cons: Automated chat that sounds human and sustains romance or investment scams over weeks, building trust before the ask.
- Synthetic identity and verification bypass: AI-generated faces, voices and documents used to pass identity checks and open accounts or reset access.
The Business Impact
The money at stake is large and well documented. In its 2024 Internet Crime Report the FBI found that business email compromise, the executive-impersonation scam that AI now makes easier caused 2.77 billion dollars in reported losses in the United States alone. Phishing and spoofing were the single most reported crime of the year.
The trend is upward. The FBI’s 2025 report logged over one million complaints, up from 859,532 the year before with AI-related fraud among the costliest categories. Because AI does the hard work, the barrier to entry has dropped. Smaller organisations are now targets alongside large enterprises.
The deeper cost is to trust. When a familiar face or voice can be faked, the usual human checks stop working. Staff can no longer rely on recognising a caller which means verification has to move from instinct to process. Done badly, that shift is where the losses happen.
Real-World Cases
The Arup Deepfake Video Call
In 2024 a finance worker at the Hong Kong office of Arup, the global engineering firm behind the Sydney Opera House, was invited to a video call with the company’s chief financial officer and several colleagues. Every person on that call was an AI deepfake, built from public footage.
The worker had first suspected a phishing email that asked for a secret transaction. Seeing the familiar faces on the call put his doubts to rest and he approved a series of transfers totalling about 25 million dollars. He only discovered the fraud after checking with head office. Hong Kong police disclosed the case in February 2024 and Arup later confirmed it was the firm involved, as CNN reported.
One habit would have stopped it. A call-back to the CFO on a number the worker already held before any money moved, would have exposed the fake.
The Ferrari Voice Clone That Failed
In July 2024 an executive at Ferrari received WhatsApp messages from an unknown number claiming to be the chief executive, Benedetto Vigna. A phone call followed using a clone of Vigna’s voice that captured his southern-Italian accent. The caller pushed an urgent and confidential acquisition and an NDA to sign at once.
The executive grew suspicious and asked the caller a question only the real Vigna could answer, about a book he had recently recommended. The call ended abruptly. Ferrari lost nothing. Bloomberg reported the attempt.
AI Voice Memos Impersonating Officials
Not every case targets a company payment. Through 2025 the FBI warned that attackers were sending AI-generated voice memos impersonating senior US officials to reach current and former officials and their contacts.
The aim was to build rapport and then move the target onto a separate encrypted app where access could be harvested. The FBI’s advice is direct. If a known contact reaches you on a new number or platform, confirm it through a channel you have used before.
AI-Powered Social Engineering and Compliance
For Swedish organisations this is now a legal matter, not only a security one. NIS2 is in force in Sweden as Cybersäkerhetslagen (SFS 2025:1506) since 15 January 2026. Article 21 lists security awareness training as a required measure and Article 20 makes the board responsible for approving and overseeing security, with members personally accountable.
If an attack succeeds and disrupts services, the incident-reporting clock starts. Cybersäkerhetslagen follows the NIS2 cascade to MCF (formerly MSB) and the sector authority, a 24-hour early warning, a 72-hour full notification and a final report within one month. Where personal data is exposed, GDPR Article 33 adds a separate 72-hour notification to IMY.
Financial entities have a further duty under DORA Article 17 which governs ICT incident management and is supervised by Finansinspektionen.
The EU AI Act adds a twist. From 2 August 2026 its Article 50 requires deepfakes to be labelled but that duty binds legitimate users of AI. A fraudster will never label a malicious fake so the AI Act supports the wider information ecosystem rather than shielding you from a targeted attack. Your real protection is the training and verification the other rules require.
How to Spot AI-Powered Social Engineering
You cannot reliably spot a modern fake by eye or ear. Real-time voice and video fakes now pass casual inspection so the tells to watch are behavioural, in the context around the request.
- Urgency and secrecy: A request that cannot wait and must be kept confidential.
- An unusual channel: A manager or supplier suddenly messaging on WhatsApp, a personal number or a surprise video call.
- Pressure to skip the process: A push to bypass the normal payment or sign-off steps just this once.
- A move to another app: An early nudge to continue on a different or encrypted platform.
- An unexpected money or access request: A change of bank details, a new payee or a request for credentials or codes.
If several of these appear together, treat the request as unverified until you have checked it however familiar the face or voice.
How to Defend Against AI-Powered Social Engineering
The single most effective control is simple. Verify any unusual or high-value request through a second known channel before you act, and never through the channel the request arrived on. This one habit would have stopped the Arup loss and it is what saved Ferrari.
Build that verification step into how your organisation actually works.
- Require two people to approve payments and supplier bank-detail changes above a set threshold.
- Agree a call-back protocol so unusual requests are confirmed on a number already on file.
- Set a code word or challenge question for sensitive requests, as Ferrari’s executive used.
- Run regular security awareness training and phishing simulations so staff practise the habit.
- Turn on multi-factor authentication and email authentication (SPF, DKIM and DMARC) to blunt account takeover and spoofing.
- Limit how much executive voice and video you publish since that footage is the raw material for a clone.
- Add AI detection and response that can flag anomalies quickly when a lure does land.
No single tool catches every fake. The organisations that avoid the loss are the ones that made verification a routine step rather than a judgement call under pressure.
