TOP
Deepfakes, a type of AI‑generated content, can be both highly realistic though not real and real content manipulated into something that appears realistic but is fake. Since it can be challenging to detect AI‑generated images, deepfakes are often unauthorized and used with nefarious intent by cyber criminals.
There are three broad categories of fakes:
Shallowfake technology is now rather dated: It involves editing media via fairly basic editing software such as Photoshop, and strictly speaking it is not AI as it does not involve deep learning.
The term “shallow” might imply that shallowfakes are less threatening than their deepfake counterparts, however that is not the case. Shallowfakes can be made and deployed easier and faster than deepfakes.
One reported example was “that one piece of Canadian ID – a passport – was submitted around 2,500 times within a month, each time with changes to the name, address, or photograph.”1 While this shallowfake-type fraud might suggest that it is less threatening than its deepfake counterparts, the fact that such fraud methodology does not require deep AI/machine learning methods to create it means that shallowfakes can be made and deployed more quickly and with more ease, a fraud risk to insurers.2
Deepfakes pose a more challenging opportunity for insurers. At the point of taking out a policy, fraudsters can create documents and photographs of a property or business that doesn’t exist, for example, or amend images to insure a property that may exist, but in an entirely different form than represented in the altered images.
Less obvious and less often considered than visual deepfakes are voice deepfakes, also known as audio deepfakes or voice cloning. Voice deepfakes are created by training AI models using recordings of a person’s voice. Once trained, the AI can generate new audio that mimics the original voice, including its tone, accent, and cadence.
Software can now create realistic, almost perfect, audio deepfakes with just a few hours of audio of the person whose voice is being replicated, and once a model of that person’s voice is replicated, that person can be made to say anything. Even reasonably good voice clones can now easily be produced from as little as a few minutes of a voice recording entered in software programs.
Video deepfakes are synthetic media in which a person's likeness is digitally altered or replaced with someone else's using machine learning techniques. The technology can create highly realistic videos that make it appear as though someone is saying or doing something they never actually did.
Phishing via email and text messages is a well-known practice, one about which we regularly receive training in our workplaces to help us identify and avoid falling victim to it. Deepfake phishing takes this risk up a notch to include voice cloning and video calls where an attacker can use a video deepfake over a teleconferencing platform to entice and convince victims to share confidential information (such as credentials) or manipulate them into carrying out unauthorized financial transactions.
Scammers can assume a person’s likeness and imitate their voice using only minutes or even seconds of publicly available video footage for reference. Then the phishing follows the same core principles that social engineering attacks follow; namely, to confuse or manipulate victims, exploit their trust, and bypass traditional security measures.
Realism and Credibility – Deepfakes can create highly realistic audio, videos, or images that closely mimic reality. This realism makes it easier for attackers to impersonate trusted individuals, such as company executives or colleagues, thereby increasing the likelihood that victims will believe the fake content.
Enhanced Personalization – By combining deepfake technology with social engineering techniques, attackers can craft highly personalized messages. This personalization makes the phishing attempts more convincing and harder to detect.
Exploitation of Trust – Deepfake phishing exploits the inherent trust people have in visual and auditory information. When victims see or hear what appears to be a familiar and trusted person, they are more likely to comply with requests, such as sharing sensitive information or authorizing financial transactions.
Difficulty in Detection – Traditional security measures, such as email filters and voice recognition systems, may struggle to detect deepfakes due to their high level of sophistication. This shortcoming makes it easier for deepfake phishing attacks to bypass these defences.
Psychological Manipulation – Deepfake phishing often leverages psychological tactics to create a sense of urgency or fear, compelling victims to act quickly without thoroughly verifying the authenticity of the request.
In January 2024, a senior executive from British engineering giant Arup was duped into a deepfake scam, which led to a loss of GBP 20 million.3
What was extraordinary about the scam was that everyone on the video call – apart from the victim – was a computer-generated video impersonation of a real person employed by Arup.
Deepfake audio is commonly used for legitimate, positive purposes, but unfortunately, it is now also routinely used to commit financial fraud, data breaches, phishing scams, and automated misinformation attacks.
The first noted instance of an AI voice deepfake used in a targeted financial scam was in 2019, and it involved the CEO of a UK‑based energy company.5 Scammers mimicked the voice of the parent company’s CEO, including replicating his slight German accent and speech patterns. The scammers called the UK company’s CEO, pretending to be his boss, and instructed him to transfer EUR 220,000 to the supposed Hungarian bank account of a supplier. The CEO complied, and it was only after the money was transferred (and then moved out quickly to accounts in other locations) that the fraud was discovered.6
Another example of unauthorized use of an audio deepfake involved British actor/writer Stephen Fry. AI software was used to replicate his voice from his Harry Potter audiobook recordings without his consent. This cloned voice was then used to narrate a historical documentary about Nazi resistance. The deepfake was so accurate that it perfectly mimicked Fry’s accent and intonation, including German and Dutch names and words that he had never actually said. Concerned at how realistic it was, Fry highlighted the ethical and legal issues surrounding the technology, emphasizing the lack of protections for individuals whose voices can be digitally replicated. He also expressed fear about the potential misuse of AI, which could manipulate voices and likenesses without permission to create deepfake content.7
The risks to individuals and businesses from deepfakes have escalated. According to research commissioned by Mastercard,8 20% of Australian businesses have received deepfake threats in the past 12 months, and of these, 12% fell for the manipulated content.9 The same research also found that more than one-third (36%) of Australian consumers have been targeted by deepfake scams between October 2023 to October 2024, with losses estimated to be tens of millions of dollars.10
Worse still, the MasterCard research revealed that 18% of businesses were targeted by deepfake scams in New Zealand January to October 2024 and of those nearly 50% of the businesses targeted actually fell victim to them.11 Needless to say, this is an alarming statistic and one which we would hope might abate once there is greater general awareness around deepfake threats.
In New Zealand, scammers commonly posed as customer service representatives (38%), clients (29%), and suppliers or vendors (26%).12
In Australia, the mix was slightly different: Scams targeting businesses included attackers posing as customer service (44%), clients (38%), and suppliers or vendors (34%). Also impersonated in scams were CEOs, employees, board members, and law enforcement.13
In view of these serious and increasing risks, the insurance industry is progressively albeit cautiously providing Cyber Risk coverage as new and ever-increasing cyber threats appear on the landscape.
Broader policies pay for data breach crisis management, legal fees, forensics investigators of the IT repercussions, credit monitoring services, and PR advisors to manage the public relations issues.
Some policies, however, require specific conditions to trigger coverage, and loss flowing from the consequences of a deepfake impersonating a real person or that person’s voice may not be covered.
Insurance companies themselves can be the victims of fraud via AI technologies by their insureds who seek to make false or exaggerated insurance claims on their insurance policies or in claims where they are the third-party claimant.
One example is an incident shared by a UK insurance law firm: They reported that after investigating CCTV evidence submitted in support of an alleged accident, they discovered that the media had been manipulated to change the date, time, and registration of the vehicle purporting to have caused the incident.15
A survey of insurance industry professionals by U.S.‑based AI content detection and forensic company Attestiv found that while 80% of those surveyed felt concerned about the potential for fraudulent images being used in insurance transactions, only 20% had acted against any digitally altered images.16 The study identified the following trends:17
Also 80% of respondents “indicated concern for altered or tampered digital media used for insurance transactions, such as claims”; respondents were at least “moderately concerned” by the following media issues:18
While the insurance sector is rapidly increasing its use of automated services, the speed of adoption is not keeping up with the need for fraud detection. The Attestiv report results suggested that more than 50% of insurance professionals planned to automate services within a year, while a mere 22% were utilising a form of validation or fraud prevention for digital media. This is an alarming gap, one that could be leveraged by fraudsters.
Unfortunately, while fraud detection tools targeted against shallowfakes and deepfakes do exist, insurance companies lack personnel with AI expertise to assess, implement, and maintain such tools. Further, even if staff can be found, the costs can initially seem prohibitive to insurers operating in cyclical markets and often with slender margins, especially if the repercussions from shallowfake and deepfake fraud is not sufficiently appreciated.
To what extent are the risks from shallowfakes and deepfakes (as well as other exposures arising from AI) – whether created nefariously or intentionally in the normal course of business – insurable or, indeed, silently insured? Here’s a basic review of some insurance covers that can provide either silent or explicit AI cover.
Explicit AI Cover – We are starting to see some early development of new, explicit AI covers, especially in the U.S. Some providers have launched products for AI users that cover losses when an AI model does not deliver as promised. For example, if AI were to replace some valuers of property at a bank for loan assessments with an AI model, and the AI makes an error that may not have been made by a human, the policy may be triggered.
A Canadian start‑up insurer has launched a product providing a product warranty that AI models will work the way their sellers promise.19
In California, a cyber insurer has added an AI coverage endorsement to their cyber insurance policies.20
Although this is a nascent category, the explicit AI cover market may well increase as demand increases.
Silent AI Cover – AI exposures may be “silently” covered where policies don’t have adequate exclusions. While AI exclusions are not yet widespread, the risks are increasing, and underwriters should not leave such cover open. Proactive decisions should be made as to whether to price in or exclude such risks.
How this is approached will likely vary between different lines of insurance. In Directors & Officers (D&O) insurance, for example, company boards might insist on the absence of policy exclusions given its importance in protecting the personal exposure and assets of directors. The extent of insurer appetite for such wide cover, however, might be another matter.
AI exposures may also lead to losses in conventional insurance lines, such as through crime, regulatory action, third-party compensation claims, or simple property damage claims. Exposures from the use or misuse of AI might be found in any of the following:
Cyber policies – May respond to security and data breaches, system failures, or other privacy breaches relating to AI. For example, if a company’s AI system was hacked, costs associated with investigation, restoration of records, and notification requirements can be high. Cyber policies may also provide cover for regulatory investigation costs and fines to the extent they are legally insurable.
Commercial Liability policies provide cover for the liabilities of a business to third parties resulting from personal injury, property damage, and possibly personal / advertising injury (as it is sometimes called) caused by accidents such as claims for libel and slander, breach of copyright or trademarks, or similar exposures. Materials can inadvertently cause copyright and/or trademark infringements.21
Product Liability insurance may cover compensation and costs if a consumer suffers resulting damage from a product powered by AI, such as a simple smart home device.
Property insurance may include property damage and business interruption (BI) insurance cover to a business’ physical property and resulting BI when physical property is damaged or destroyed in the event that AI causes property damage, for example, if an AI‑powered thermal regulator malfunctions and causes a fire, or a hacked AI control system causes physical damage.
Professional Indemnity (PI) insurance covers may be relevant for exposures that arise from the reliance on or use of AI in the provision of services, especially by professionals such as solicitors, accountants, architects, financial advisors, or healthcare professionals (via Medical Malpractice PI covers). Exposure may also arise from regulatory actions. Liabilities, defence costs, regulatory investigation costs, and possible statutory fines might be covered under PI and Medical Malpractice policies as well as other policies that may also provide overlapping cover.
D&O insurance may protect directors or officers for securities or other liabilities. Some boards now have an AI board advisor.22 The use of AI in decision-making processes could lead to claims for breaches of duty or mismanagement arising from those decisions and where the company loses value it may lead to derivative (shareholder) claims. Exposures may also arise from the use of AI in the provision of services, in relation to their AI capabilities (i.e., AI washing), in relation to misuse of AI relating to data privacy, or even in relation to employment.
Employment Practices Liability Insurance (EPLI) may cover an employer for compensation claims for discrimination, unfair treatment, wrongful termination, or harassment resulting from AI systems. AI is increasingly being used to assist in selecting and ranking candidates; if any bias or discrimination infiltrates the process because of the use of AI, there may be EPLI exposures.
Underwriters and claims handlers must be cognizant of the ever-increasing new risks and exposures created by the exponential rise in the use of AI technology. With that, there are some defensive tools to spot deepfakes: the World Economic Forum lists four countermeasures that might be of interest to you.
Let’s embrace AI while remaining aware and being prepared.
