There have been reports of scammers are using the latest AI deepfake voice clones to sound like family members in distress to swindle relatives out of money.
AI Voice Technology
The kinds of artificially generated voice technology tools that bad actors could be using for ‘imposter scams’ may include ElevenLabs, Murf, Play.ht, Respeecher, Resemble, Overdub, ReadSpeaker, Voice.ai, and many more.
How Are Voices Cloned?
Typically, AI voice cloning software can take a short voice sample of the human voice you want to replicate, analyse it, learn it, and replicate it (instantly). This cloned, ‘deepfake’ voice can then be replicated reading any script provided to it.
How Do These Imposter Scams Work?
As highlighted in a recent US Washington Post report, a common form of the scam involves the scammers obtaining a short (just seconds) audio sample of a person’s voice (e.g. from YouTube, podcasts, TikTok, Instagram or a Facebook video). The scammer then runs this through voice cloning software and makes a short clip of the person sounding distressed and asking for money to be transferred to help them (e.g. for legal fees). This audio clip is then played down the phone to loved ones (e.g. parents or grandparents) who are duped and react by transferring funds into the bitcoin account (because it’s not traceable) named in the scammer’s audio clip, or named in a follow-up phone call by the scammers.
How Big Is The Problem?
US Federal Trade Commission statistics show that in 2022, the impostor scam was the most frequent type of fraud reported and generated the second-highest losses for those targeted, e.g. out of 36,000 reports, more than 5,000 victims lost $11 million in scams over the phone.
Examples
Some examples of AI deepfake voice ‘imposter’ scams include:
– A Canadian couple being scammed out of $21,000 following a phone call from someone claiming to be a lawyer who told them their son was in jail for killing a diplomat in a car accident and needed legal fees.
– Ruth Card, a Saskatchewan (Canada) grandmother who (would have) paid $9,400 to help pay legal fees for her grandson (if the bank hadn’t intervened) following a bogus accident explained in an AI fake phone call from her grandson and a call from bogus lawyer on his behalf.
Complicated
This type of scam is difficult to stop for several reasons, such as:
– There are now so many voice cloning options, many of which are free and don’t require proof of whose voice is being cloned or the customer making the clip.
– The scam phone calls can be made from anywhere in the world.
– It’s difficult to decide which agencies have jurisdiction to investigate which cases if the scammers operate out of different countries
– The bitcoin accounts that money is paid into can’t be traced.
– There’s no insurance to re-imburse victims for the money.
– Courts have not yet decided when/if companies are liable for harm caused by AI deepfake voice technology and AI companies are showing no signs of slowing down the release of new AI services and features.
What Does This Mean For Your Business?
Deepfake technology that enables people to generate highly realistic images, videos and audio of people is prevalent and although it presents opportunities (e.g. creating videos or realistic avatars for businesses and special effects in films), it is essentially unregulated and clearly has the potential for misuse, such as the spread of misinformation and fraud.
There are currently fears that the technology is advancing and being made publicly available too quickly and is too far ahead of an assessment of the risks and any regulation (hence the recent open letter including Elon Musk’s signature). Deepfake scams present another real, significant and growing risk to businesses and the advice on receiving such a call should be to try and remain calm and sceptical, have a system in place before funds can be transferred where checks must be made, and to first try to contact the person directly for whom the funds are required. That said, voice cloning technology is incredibly good, scammers can be very convincing, and it is difficult for people to fight a strong emotional response to help their loved ones and friends. Some people believe that more needs to be done in the first instance to make voice cloning software companies have more checks in place and to perhaps become liable in some way if fraud is carried out using their services.
