AI-powered scams are becoming more convincing
I recently read the article “AI Scammers Are Better at Building Trust Than Humans”, and I wanted to go one level deeper and find the original research behind it. The study is “Love, Lies, and Language Models: Investigating AI’s Role in Romance-Baiting Scams”.
The research focuses on so-called romance-baiting scams, often called “pig butchering”, where criminals first build a relationship and emotional trust, and only later move the victim toward a fake investment or fraudulent platform.
Photo: macrovector na Magnific
The researchers describe the process as:
➡️ Hook → Line → Sinker
➡️ The “hook” starts the conversation.
➡️ The “line” is the long trust-building phase.
➡️ The “sinker” is the final financial exploitation.
In the experiment, participants unknowingly communicated for a week with both a human and a Claude-based AI agent. At the end of the interaction, the AI and the human asked participants to perform a small action used as a proxy for trust.
The result:
➡️ 46% complied with the AI request, compared with only 18% for the human.
➡️ Participants also rated the AI as more trustworthy, 3.78 vs. 3.31 on a five-point scale, and around 80% of all messages were sent to the AI rather than the human counterpart.
➡️ Only one participant independently concluded during the conversation that they were interacting with AI.
The research also included interviews with former scam workers and trafficking survivors from scam compounds. Those interviews helped researchers reconstruct how these operations work in practice and showed that AI is already being used for translation, improving language, creating more convincing personas and generating deepfakes.
AI can automate the slowest and most expensive part of social engineering: building trust over time. That changes the economics of fraud.
One operator can potentially run many more conversations simultaneously, in multiple languages, 24/7, with consistent tone, memory and personalization.
And as models improve, these attacks will become even cheaper, more scalable and more convincing.
Unfortunately, the basic human defence remains surprisingly old-fashioned:
➡️ critical thinking,
➡️ avoiding emotional decisions,
➡️ slowing down when someone creates urgency,
➡️ independent verification,
➡️ never treating familiarity as proof of identity,
➡️ and, frankly, maintaining a healthy level of “light paranoia.”
Author: Sebastian Burgemejster
Comments