The Next Battle Against Fraud: AI Deepfakes

For a long time, there has been talk of scams where innocent people are tricked into making incorrect payments or giving away their bank IDs. However, the possibility of being deceived by these methods is diminishing thanks to technological development and information campaigns from Swedish banks, following initiatives from the government and the Swedish Bankers’ Association. The next major battle against fraudsters will take place in the AI arena.

A trained eye or ear can detect it fairly quickly. Something simply isn’t right when the fund profile Adam Gerge, in what appears to be some kind of collaboration with Nordnet, touts “rocket stocks” in a film on Facebook and wants us to join his exclusive WhatsApp group. It is also a slight understatement to state that something is not entirely right when a Måns Zelmerlöw with strange mimicry speaks warmly about crypto, his “secret” investment opportunity. What appears to be Adam Gerge or Måns Zelmerlöw is intended to get viewers with a craving for a shortcut to wealth to give away their contact information in order to then, one assumes, be contacted by the scammers with some type of “investment offer.”

It’s about AI-generated films, so-called deepfakes. The technology is not yet perfect. Revealing misspellings, poor language, and unnatural mimicry mean that most people see through the clips that are spread today as fake. But the tools are constantly improving, and when clips are spread extremely widely, as they do on social media, the risk increases significantly that people will be fooled. It is only a matter of time before a clip with SVT’s Anders Holmberg talking about his fantastic crypto investment cannot be distinguished from the real, actual Anders Holmberg – even for a trained eye.

The problem when new technology is quickly adopted by fraudsters is that authorities and society constantly seem to be one step behind. The Swedish Financial Supervisory Authority commendably warns on its website about deepfakes in connection with investment fraud, but does it really reach the group most vulnerable to this type of fraud – those with the lowest financial knowledge? One risks simply not reaching those who would need to take part in the message.

I naturally do not have all the answers on how we should combat investment fraud that is done with the help of deepfakes. But social media platforms, authorities, industry organizations, banks – yes, society at large – must join forces. The major platforms have a significant responsibility because this is where the clips are spread. The platforms must act faster and more efficiently, including by labeling AI-generated content.

The financial industry must also act. Investment fraud with deepfakes must be higher on the agenda. It is not enough to just discuss old fraud methods such as poorly spelled emails or text messages. Deepfakes can become much more dangerous, as they become harder and harder to see through and spread so widely to so many people.

One can also cautiously ask the question: what does it do to a society and its information security when we can no longer trust what we see?

Frida Bratt, Savings Economist at Nordnet

---

Enjoyed this post by Thibault Helle? Subscribe for more insights and updates straight from the source.