Deepfakes, a type of AI‑generated content, can be both highly realistic though not real and real content manipulated into something that appears realistic but is fake. Since it can be challenging to detect AI‑generated images, deepfakes are often unauthorized and used with nefarious intent by cyber criminals.

There are three broad categories of fakes:

• Shallowfakes
• Deepfakes (voice/audio)
• Deepfakes (video)

Shallowfakes

Shallowfake technology is now rather dated: It involves editing media via fairly basic editing software such as Photoshop, and strictly speaking it is not AI as it does not involve deep learning.

The term “shallow” might imply that shallowfakes are less threatening than their deepfake counterparts, however that is not the case. Shallowfakes can be made and deployed easier and faster than deepfakes.

One reported example was “that one piece of Canadian ID – a passport – was submitted around 2,500 times within a month, each time with changes to the name, address, or photograph.”¹ While this shallowfake-type fraud might suggest that it is less threatening than its deepfake counterparts, the fact that such fraud methodology does not require deep AI/machine learning methods to create it means that shallowfakes can be made and deployed more quickly and with more ease, a fraud risk to insurers.²

Deepfakes

Deepfakes pose a more challenging opportunity for insurers. At the point of taking out a policy, fraudsters can create documents and photographs of a property or business that doesn’t exist, for example, or amend images to insure a property that may exist, but in an entirely different form than represented in the altered images.

Voice/Audio deepfakes

Less obvious and less often considered than visual deepfakes are voice deepfakes, also known as audio deepfakes or voice cloning. Voice deepfakes are created by training AI models using recordings of a person’s voice. Once trained, the AI can generate new audio that mimics the original voice, including its tone, accent, and cadence.

Software can now create realistic, almost perfect, audio deepfakes with just a few hours of audio of the person whose voice is being replicated, and once a model of that person’s voice is replicated, that person can be made to say anything. Even reasonably good voice clones can now easily be produced from as little as a few minutes of a voice recording entered in software programs.

Video deepfakes

Video deepfakes are synthetic media in which a person's likeness is digitally altered or replaced with someone else's using machine learning techniques. The technology can create highly realistic videos that make it appear as though someone is saying or doing something they never actually did.

Deepfake Phishing

Phishing via email and text messages is a well-known practice, one about which we regularly receive training in our workplaces to help us identify and avoid falling victim to it. Deepfake phishing takes this risk up a notch to include voice cloning and video calls where an attacker can use a video deepfake over a teleconferencing platform to entice and convince victims to share confidential information (such as credentials) or manipulate them into carrying out unauthorized financial transactions.

Scammers can assume a person’s likeness and imitate their voice using only minutes or even seconds of publicly available video footage for reference. Then the phishing follows the same core principles that social engineering attacks follow; namely, to confuse or manipulate victims, exploit their trust, and bypass traditional security measures.

Deepfake Video Fraud Example

In January 2024, a senior executive from British engineering giant Arup was duped into a deepfake scam, which led to a loss of GBP 20 million.³

• An employee of Arup was invited to a video call and accepted. The meeting was attended by what were in fact deepfakes of the employee’s co‑workers, including the fake Chief Financial Officer.
• The scammers found publicly available video and audio of the impersonation targets via YouTube and used deepfake technology to emulate their voices
• During the video conference, since the fake participants looked like the real people, the employee trusted them. The scammers instructed the employee to make 15 transactions totalling HKD 200 million to five local bank accounts.
• After the initial video conference, the scammers followed up via instant messages, email, and additional one-on-one video calls.
• The employee acted on the scam instructions and distributed the funds. About a week later, the employee discovered it was a scam.
• It was only after participating in the video call and acting on the scam instructions by distributing the funds that the employee spoke to the company’s head office and realised it was a scam, reported RTHK, Hong Kong’s public broadcaster.⁴

What was extraordinary about the scam was that everyone on the video call – apart from the victim – was a computer-generated video impersonation of a real person employed by Arup.

Deepfake Audio Fraud Examples

Deepfake audio is commonly used for legitimate, positive purposes, but unfortunately, it is now also routinely used to commit financial fraud, data breaches, phishing scams, and automated misinformation attacks.

The first noted instance of an AI voice deepfake used in a targeted financial scam was in 2019, and it involved the CEO of a UK‑based energy company.⁵ Scammers mimicked the voice of the parent company’s CEO, including replicating his slight German accent and speech patterns. The scammers called the UK company’s CEO, pretending to be his boss, and instructed him to transfer EUR 220,000 to the supposed Hungarian bank account of a supplier. The CEO complied, and it was only after the money was transferred (and then moved out quickly to accounts in other locations) that the fraud was discovered.⁶

Another example of unauthorized use of an audio deepfake involved British actor/writer Stephen Fry. AI software was used to replicate his voice from his Harry Potter audiobook recordings without his consent. This cloned voice was then used to narrate a historical documentary about Nazi resistance. The deepfake was so accurate that it perfectly mimicked Fry’s accent and intonation, including German and Dutch names and words that he had never actually said. Concerned at how realistic it was, Fry highlighted the ethical and legal issues surrounding the technology, emphasizing the lack of protections for individuals whose voices can be digitally replicated. He also expressed fear about the potential misuse of AI, which could manipulate voices and likenesses without permission to create deepfake content.⁷

Deepfakes – Some Risk Transfer to Insurers

The risks to individuals and businesses from deepfakes have escalated. According to research commissioned by Mastercard,⁸ 20% of Australian businesses have received deepfake threats in the past 12 months, and of these, 12% fell for the manipulated content.⁹ The same research also found that more than one-third (36%) of Australian consumers have been targeted by deepfake scams between October 2023 to October 2024, with losses estimated to be tens of millions of dollars.¹⁰

Worse still, the MasterCard research revealed that 18% of businesses were targeted by deepfake scams in New Zealand January to October 2024 and of those nearly 50% of the businesses targeted actually fell victim to them.¹¹ Needless to say, this is an alarming statistic and one which we would hope might abate once there is greater general awareness around deepfake threats.

In New Zealand, scammers commonly posed as customer service representatives (38%), clients (29%), and suppliers or vendors (26%).¹²

In Australia, the mix was slightly different: Scams targeting businesses included attackers posing as customer service (44%), clients (38%), and suppliers or vendors (34%). Also impersonated in scams were CEOs, employees, board members, and law enforcement.¹³

In view of these serious and increasing risks, the insurance industry is progressively albeit cautiously providing Cyber Risk coverage as new and ever-increasing cyber threats appear on the landscape.

Broader policies pay for data breach crisis management, legal fees, forensics investigators of the IT repercussions, credit monitoring services, and PR advisors to manage the public relations issues.

Some policies, however, require specific conditions to trigger coverage, and loss flowing from the consequences of a deepfake impersonating a real person or that person’s voice may not be covered.

Deepfakes in Insurance Fraud

Insurance companies themselves can be the victims of fraud via AI technologies by their insureds who seek to make false or exaggerated insurance claims on their insurance policies or in claims where they are the third-party claimant.

One example is an incident shared by a UK insurance law firm: They reported that after investigating CCTV evidence submitted in support of an alleged accident, they discovered that the media had been manipulated to change the date, time, and registration of the vehicle purporting to have caused the incident.¹⁵

A survey of insurance industry professionals by U.S.‑based AI content detection and forensic company Attestiv found that while 80% of those surveyed felt concerned about the potential for fraudulent images being used in insurance transactions, only 20% had acted against any digitally altered images.¹⁶ The study identified the following trends:¹⁷

• The insurance industry has been increasingly embracing self-service and a high degree of automation for transactions such as claims.
• The reliance on media such as digital photos to settle insurance claims increased twofold since the COVID pandemic.

Also 80% of respondents “indicated concern for altered or tampered digital media used for insurance transactions, such as claims”; respondents were at least “moderately concerned” by the following media issues:¹⁸

• Altered photos that falsely inflate claims
• Misleading photos hiding information
• Old photos being used in new claims
• Photos of the wrong asset
• Images from the internet used in a claim
• Low-quality photos could obscure important details

While the insurance sector is rapidly increasing its use of automated services, the speed of adoption is not keeping up with the need for fraud detection. The Attestiv report results suggested that more than 50% of insurance professionals planned to automate services within a year, while a mere 22% were utilising a form of validation or fraud prevention for digital media. This is an alarming gap, one that could be leveraged by fraudsters.

Unfortunately, while fraud detection tools targeted against shallowfakes and deepfakes do exist, insurance companies lack personnel with AI expertise to assess, implement, and maintain such tools. Further, even if staff can be found, the costs can initially seem prohibitive to insurers operating in cyclical markets and often with slender margins, especially if the repercussions from shallowfake and deepfake fraud is not sufficiently appreciated.

Insurance Covers Potentially Exposed

To what extent are the risks from shallowfakes and deepfakes (as well as other exposures arising from AI) – whether created nefariously or intentionally in the normal course of business – insurable or, indeed, silently insured? Here’s a basic review of some insurance covers that can provide either silent or explicit AI cover.

Explicit AI Cover – We are starting to see some early development of new, explicit AI covers, especially in the U.S. Some providers have launched products for AI users that cover losses when an AI model does not deliver as promised. For example, if AI were to replace some valuers of property at a bank for loan assessments with an AI model, and the AI makes an error that may not have been made by a human, the policy may be triggered.

A Canadian start‑up insurer has launched a product providing a product warranty that AI models will work the way their sellers promise.¹⁹

In California, a cyber insurer has added an AI coverage endorsement to their cyber insurance policies.²⁰

Although this is a nascent category, the explicit AI cover market may well increase as demand increases.

Silent AI Cover – AI exposures may be “silently” covered where policies don’t have adequate exclusions. While AI exclusions are not yet widespread, the risks are increasing, and underwriters should not leave such cover open. Proactive decisions should be made as to whether to price in or exclude such risks.

How this is approached will likely vary between different lines of insurance. In Directors & Officers (D&O) insurance, for example, company boards might insist on the absence of policy exclusions given its importance in protecting the personal exposure and assets of directors. The extent of insurer appetite for such wide cover, however, might be another matter.

AI exposures may also lead to losses in conventional insurance lines, such as through crime, regulatory action, third-party compensation claims, or simple property damage claims. Exposures from the use or misuse of AI might be found in any of the following:

Conclusion

Underwriters and claims handlers must be cognizant of the ever-increasing new risks and exposures created by the exponential rise in the use of AI technology. With that, there are some defensive tools to spot deepfakes: the World Economic Forum lists four countermeasures that might be of interest to you.

Let’s embrace AI while remaining aware and being prepared.