Imagine seeing a video where a famous person – a singer, an entrepreneur, or a politician – promotes an investment project or asks for help. The voice sounds familiar, and so does the face. But that person isn't actually in the video. What you're seeing is a deepfake, and with each passing month, these fakes are becoming increasingly difficult to distinguish from reality.
These are no longer isolated incidents or something exotic, but rather a well-oiled industry.
A Familiar Face as a Tool of Deception
Scammers realized long ago that the shortest path to gaining trust is to use someone people already trust. This is precisely why celebrity deepfakes have become one of the most common tools in online fraud.
The scheme is simple: they take a public figure with a recognizable face, voice, and reputation. Using AI, they create a video or audio clip in which this person says something they never actually said – promoting cryptocurrency, urging people to transfer money, or talking about a «unique opportunity.» The viewer sees a familiar face and, for a split second, lowers their critical defenses. That's all it takes.
Moreover, the technologies that once required significant computing power and specialized knowledge are now available to just about anyone. Ready-made services exist where, for a small fee, you can generate a convincing video with any face – no programming skills or understanding of how it works «under the hood» required.
Avatar Farms: When One Scammer Becomes a Thousand
But celebrity deepfakes are just the tip of the iceberg. Another, less conspicuous but no less dangerous practice is developing in parallel: so-called avatar farms.
Simply put, this is the mass production of fake identities. Not one fake «star», but hundreds and thousands of fictional people – complete with faces, names, backstories, social media profiles, and even behavioral patterns that mimic real individuals.
What's the purpose? There are several goals:
- Manipulating public opinion. An army of fake accounts can create the illusion of mass support for an idea, product, or person.
- Fraud schemes. Fake «people» build relationships with real users – through chats, comments, and private messages – and then exploit that trust for personal gain.
- Advertising and reputation manipulation. Fictitious accounts inflate views, reviews, and likes, creating a false impression of popularity.
It used to take hours to create a convincing fake profile. Now, it takes minutes. Generative AI can create realistic portraits of non-existent people, write coherent biographies, mimic communication styles, and even adapt a «personality» to a specific platform.
How It Works in Practice
A typical scam chain looks something like this. First, a «personality» is created – a face generated by a neural network, and an invented name, backstory, and place of work. Then, this avatar «lives» online for a while, making posts, commenting on others' content, and gaining followers. This is called «warming up» – the account starts to look authentic.
After that, the main operation begins: the avatar makes contact with real people. Sometimes, it's a romance scam (a «connection» with an attractive stranger that eventually ends with a request for money). Sometimes, it's an investment proposal. And sometimes, it's simply spreading disinformation.
Those who aren't expecting a trick are especially vulnerable: elderly users, people in an emotionally unstable state, and those looking for companionship or financial opportunities.
Why It's Hard to Stop
There are several reasons why combating avatar farms and deepfakes is difficult.
First, scale. Whereas a scammer once had to personally handle every conversation, today, part of the communication can be automated. A single person can manage hundreds of fake profiles at once.
Second, the quality of fakes is improving faster than the methods to detect them. Deepfake detectors are constantly falling behind: generation models evolve quickly, while verification tools lag.
Third, the psychological factor. We are evolutionarily wired to trust faces and voices. When the brain «recognizes» a familiar person, critical thinking momentarily shuts down. That's precisely when the trap is sprung.
Finally, the infrastructure for fraud has become accessible. Tools that were once in the hands of a small circle of specialists are now sold as a service – complete with an interface, support, and even «quality guarantees.»
What to Do About It – At Least on a Personal Level
It's impossible to fully protect yourself from deepfakes and fake accounts on your own. But reducing the risks is entirely possible.
A few practical guidelines:
- An unexpected offer from an unknown contact is a reason to be wary. This is especially true if the offer involves money, investments, or urgent action.
- A video by itself is not proof. Even if it features a familiar face and voice, it doesn't mean that person actually said it. Before you trust it, check official sources.
- A reverse image search can help. If a profile picture is a generated face, special services can sometimes detect it.
- A biography that's too perfect is a suspicious sign. Real people are inconsistent: they have contradictions, gaps in their posting history, and mismatched details. Avatars often look too «polished.»
The main thing is to remember that trust has become a resource that can be forged just like documents or signatures. This isn't a reason for paranoia, but it is a reason to be a little more mindful of who and why we trust online.
Technology will continue to evolve, and the line between «real» and «generated» will only become blurrier. For now, awareness is one of the few tools that truly works in the user's favor.