AI has made online scams more convincing – be prepared and recognise the warning signs
Imagine you are on a busy work trip when your phone alerts you to a message saying that a parcel you have been waiting for is ready for collection. You assume it must be the parcel you recently ordered and click the link in the message without giving it much thought.
The message is entirely plausible. The page that opens also looks identical to the genuine Posti website. Yet it is a scam. There are no spelling mistakes or awkward sentences.
Markus Miettinen, Professor of Cybersecurity at the University ofJyväskylä, says he recently almost fell for just this kind of scam.
The scam’s credibility depends on its timing. Miettinen was expecting a parcel when he received the message. In such a situation, it is easy to go on autopilot and act without thinking because the message is exactly what you were expecting.
Scammers exploit a sense of urgency
According to Miettinen, the types of scams used by cybercriminals are largely the same as before AI tools became available.
Now, however, AI can write convincing scam messages in impeccable Finnish very quickly. Criminals have more tools at their disposal.
AI has made scams more convincing than before, says Miettinen. Most often, however, criminals still seek to reach a large group of people and hope that the timing is right for the recipient.”
Scammers also exploit a sense of urgency.
“A scam works best when it relates to an ongoing situation,” says Miettinen. “If a message is expected and familiar in format, people can easily go on autopilot. Messages often also include details that heighten the sense of urgency, because people are less able to think things through when they are in a hurry.”
Deadlines can heighten this sense of urgency: a message might claim that a parcel will be returned to the sender, a bank account or card will be blocked, or a late payment will result in additional charges or debt collection.
Tailored scams target large sums
The scams encountered by ordinary citizens are still mostly aimed at large groups of people. Scams that require more extensive preparation and use information gathered about the victim to tailor messages are often aimed at organisations, where the potential financial gains are also greater.
Perhaps the best-known AI-assisted scam took place in 2024 and targeted the Hong Kong-based company Arup. An employee in the company’s finance department joined a video conference in which the employee believed they were speaking with the company’s financial director and several other colleagues. In reality, the other participants were AI-generated fakes.
The employee was deceived into transferring a total of about $25 million to various bank accounts controlled by the criminals.
Miettinen says that carrying out major scams requires extensive background work, and such operations resemble professional corporate espionage more than indiscriminately distributed scam messages.
In the Arup case, this required knowledge of the company’s procedures, identification of key people and access to video and audio recordings of them. Finally, the criminals created a plausible narrative and ensured that the target would not check the information with another source, for example by phone.
AI has lowered the technical barriers to carrying out this type of attack. Previously, producing a convincing deepfake video would have required specialist expertise, but creating convincing fake audio and video is now much easier.
Finding security vulnerabilities has become easier
While scams targeting people remain the most significant form of cybercrime, AI also makes it possible to exploit vulnerabilities in software and online environments more efficiently, Miettinen says.
The software applications we use contain various flaws, which security researchers have traditionally investigated and analysed. What has changed is that AI can now quickly review vast amounts of source code, identify vulnerabilities and suggest not only fixes but also possible ways to exploit those vulnerabilities in cyberattacks.”
In practice, software security flaws can now be identified more quickly and comprehensively. Potential attackers can also use AI to write code that exploits a vulnerability as soon as it is discovered.
The next cyberthreat involves manipulating AI models
The next battle in cybersecurity, however, concerns not software but AI itself. Panu Moilanen, Senior University Lecturer in Security and Strategic Analysis at the University of Jyväskylä, shares this view.
In the future, attackers will not necessarily try to break into systems but may instead attempt to influence what AI considers to be true.
Unlike cybercriminals, those conducting online influence operations are often state-controlled actors.
“Attempts are already being made to manipulate AI models by feeding them false data,” Moilanen explains. “The aim is to influence how AI answers questions.”
Online influence operations seek to create chaos
This is known as LLM grooming: producing vast amounts of content intended to become part of AI training data. The aim is to get AI to repeat the desired narrative.
Whereas cybercriminals usually seek financial gain, those conducting online influence operations aim to manipulate opinions, weaken public trust in society, increase uncertainty and undermine society’s ability to function, for example, by fuelling disputes and conflicts between different groups.
When a public comments section suddenly fills up with posts from hundreds of people expressing apparently similar opinions on an issue, it creates the impression that everyone thinks alike.
AI makes it easy to write social media posts and comments, as well as longer articles or entire web pages. This form of influence is not new, but AI has made it possible on a much larger scale.
Practical tips for tackling cybercrime and online influence operations
Markus Miettinen and Panu Moilanen offer three tips to help people protect themselves against cybercrime and online influence operations.
- Pause if a message triggers a strong emotional response. A sense of urgency, fear, hatred, excitement or shock may indicate an attempt to influence or deceive you.
- Check the information against several sources. Don’t rely on a single message, image or video. In Finland, reliable sources generally include public authorities, news media and experts. Remember, however, that these sources, as well as people you know, can be impersonated.
- Don’t automatically trust what you see or hear. AI can be used to create convincing fake audio and visual material.