Images of the finance minister, Katy Gallagher, and the foreign minister, Penny Wong, have been used in deepfake investment scam videos in the latest examples of a concerning trend of politicians being used in online fraud.
The Nationals senator Bridget McKenzie and the former prime minister Scott Morrison have also had their faces used in scams through Facebook videos that appear to be generated by AI, with more than a dozen examples this week of paid ads reaching thousands of Australians before being removed.
“I want to be clear that I, or any other politician, would never spruik products or investment opportunities to Australians online,” Gallagher said.
Stephen Jones, the federal minister responsible for anti-scam work, has warned that artificial intelligence could “turbocharge” fraud and has flagged looming reforms to compel social media companies to do more.
Guardian Australia has uncovered dozens of ads that have recently run on Facebook that appear to be part of a coordinated network of scam activity. The image of McKenzie has been used in a large number of them, with a recent appearance on Sky News with host Chris Kenny being manipulated to make it seem as though the journalist and politician were spruiking an investment where “every Australian will receive $7,700 in the first week”.
The videos appear to have used AI to manipulate the vision and voices of McKenzie and Kenny, as well as other politicians such as Wong, Morrison and Gallagher.
Kenny warned viewers during his Sky News program on Wednesday that the images were a scam.
At least 11 different pages have posted exactly the same video, reaching thousands of Australians through Facebook’s promoted ad network. Some pages posting this scam and similar others have admins based in countries including Ukraine and Vietnam, or paid in currencies including the Singaporean dollar, Peruvian sol, Brazilian real, Mexican peso, Danish krone and Egyptian pound.
Most of the pages only promoted the ad with small amounts, equivalent to a few hundred Australian dollars. Many of the pages have either been deleted or had their ads removed for not complying with Facebook’s advertising rules. All the ads are now inactive, with many only active for a short time.
But at least one version of the McKenzie ad was boosted with up to A$15,000 in paid ad spending and reached up to 250,000 Australians.
“This highlights the issue with AI. People’s own identity, reputation, their face and voice can be used in ways they never imagined,” McKenzie said.
She said she had tried reporting the scam videos to authorities, but claimed “no one is very clear with how to deal with this”.
“It delegitimises and further impacts trust in institutions and in parliamentarians, when our identity and images are used for such purposes,” McKenzie said.
A similarly manipulated video, using the same font and visual style, appears to show Gallagher conducting a television interview with the ABC. The video promises “$36,000 monthly to every citizen who invests $400” under what the scam claims is an “anti-inflation plan” devised by the government. That ad reached up to 2,000 people.
At least four pages have been posting similar videos of Morrison, who quit parliament earlier this year. In one doctored video, the original of which was shot inside his Parliament House office in May 2023, Morrison’s AI-manipulated voice describes himself as “billionaire, businessman, politician and just a happy man”, promoting another get-rich-quick scheme.
Paid ads promoting those posts reached up to 70,000 people.
after newsletter promotion
Wong’s image is also being used in what appears to be a similar but unrelated scam, manipulated to appear to promote a financial product. Manipulated video of the Atlassian founders, Mike Cannon-Brookes and Scott Farquhar, also appears in the post, which reached fewer than 1,000 people.
Gallagher noted the government’s work to crack down on scams, but said the fact she had featured in such frauds “serves as an important reminder to not believe everything you see online”.
“Deepfake videos are a terrible tool used by crooks to prey on people’s hard-earned savings, and we know they’re on the rise. Any video showing this is fake and should be reported to Scamwatch,” she said, noting the federal initiative run by the Australian Competition and Consumer Commission.
Jones, the minister for financial services, also warned artificial intelligence was increasing instances of online scams.
“These guys are crumbs but not stupid. They mimic genuine advertisements with hi-tech celebrity fakes. The use of AI will turbocharge their scams,” he said.
“The onus is on social media operators to hunt this stuff down and remove it from their platforms.
“Our next wave of reforms will require banks, telecommunications companies and social media platforms to protect their customers from scams.”
The government is undertaking processes to stamp out AI-related harms, considering reforms such as mandatory watermarking of AI-generated images or other “guardrails” to protect against misuse.
But McKenzie said she was concerned about harms already occurring.
“This is a technology that is being so rapidly adopted, so rapidly misused, and we are talking about a very sophisticated, well-resourced, I would suggest criminal element working on this,” she said of the scams.
“It’s only a hop, skip and jump to have those who would seek to do our democracy harm to be using the same tactics.”
Source link
lol
