"Critical Alert: Australian Schools Warned About Apps That Can Undress Images"

Share this @internewscast.com

This article includes mentions of technology-facilitated sexual violence.

Australian schools are being encouraged to promptly report deepfake issues to the relevant authorities due to the “rapid increase” of ‘nudify’ apps on the internet.

eSafety commissioner Julie Inman Grant has contacted education ministers, advising them to ensure schools are following state and territory regulations and mandatory reporting requirements, the online safety agency announced on Friday.

This statement follows eSafety’s observation that the number of reports to its image-based abuse scheme concerning digitally altered intimate photos, including deepfakes, from individuals under 18 has more than doubled over the past year and a half, compared to the total reports over the previous seven years.

The majority of these reports, four out of five, involved young girls being targeted.

eSafety did not reveal the specific number of complaints received. Nevertheless, despite the “rapid increase” being a significant issue, Inman Grant suggested the situation might be even more severe.

“We suspect what is being reported to us is not the whole picture,” Inman Grant said.

“We have heard from school leaders and education officials that deepfake occurrences are becoming more common, especially as children can easily access and misuse nudify apps in school environments,” she said.

‘A crisis affecting school communities’

Deepfake technology allows users to manipulate images and video using artificial intelligence, which eSafety says is a “crisis affecting school communities across Australia”.

It says tools, especially ‘nudify’ apps, are increasingly being used among young people, and can be exploited to generate non-consensual synthetic explicit images, including of children.

The technology can be free, fast and easy to use — and can cause deep personal harm.

“With just one photo, these apps can nudify the image with the power of AI in seconds,” Inman Grant said.

“Worryingly, these apps have been used to shame, intimidate, and sexually blackmail children both in and outside of school. There are also reports of some of these images being exchanged among students for money.”

A ‘normalisation’ of creating deepfakes

Asher Flynn — a criminology professor at Monash University, who specialises in AI-facilitated abuse — said the rise in reports to eSafety is “confronting, but not unexpected”.

“We have seen a proliferation of ‘user-friendly’ deepfake creation tools emerging online, and we have also seen a normalisation of creating deepfake content,” she told SBS News.

Flynn’s research also shows an increase in sexualised deepfake abuse, with similar impacts to other forms of technology-facilitated sexual violence. This can include causing physical, psychological, social, reputational and financial harms to victims.

“We are also seeing a range of motivations for the incidents, from sexual gratification, to intentionally causing harm, controlling or degrading the target of the image, right through to thinking it’s funny, building social status among peers, and curiosity in how the process of creating a deepfake works,” she said.

Shahriar Kaisar, a senior lecturer of information systems from RMIT University — who specialises in the use of generative AI — said deepfakes have become a “severe” issue over the last couple of years with the rise of generative AI.

“It has become quite readily available to everyone,” he told SBS News.

“Creating deepfakes has become so much easier — and the way the images propagate online is with very rapid speed.”

When it comes to young people, he warned this can happen online or in school communities, such as messaging apps, “where we have even more limited visibility of what they are sharing among themselves”.

‘We will not hesitate to take regulatory action’

Inman Grant said the agency has been engaging with police, app makers and their host platforms to “put them on notice”.

Laws designed to require global tech giants to tackle harmful online content, including deepfakes and ‘nudify’ apps, were recently introduced to parliament.

She said mandatory standards, which carry penalties of up to $49.5 million for companies that breach them, come into full effect this week.

“We will not hesitate to take regulatory action,” she said.

Flynn said a multifaceted response is required. In addition to holding platforms and creators to account, she said this includes education and awareness raising, and prevention resources to help shift “problematic gendered norms and expectations to encourage a culture that does not condone this type of behaviour”.

Legal responses must also be in place to ensure there are consequences for perpetration — along with responses to help young people understand the harmful behaviour, Flynn said.

Kaisar also supports a “holistic approach”.

“We are working on the regulation and it was great the bill was passed last year,” he said.

“But the more important thing would be raising awareness and an ethical understanding of technology among school kids.”

eSafety has released an updated toolkit for schools on how to prepare for and manage deepfake incidents.

It is strongly encouraged that schools report any potential criminal offence to the local police.

“I’m calling on schools to report allegations of a criminal nature, including deepfake abuse of underage students, to police and to make sure their communities are aware that eSafety is on standby to remove this material quickly,” Inman Grant said.

“It is clear from what is already in the public domain, and from what we are hearing directly from the education sector, that this is not always happening.”

If you or someone you know is impacted by family and domestic violence, call 1800RESPECT on 1800 737 732, text 0458 737 732, or visit 1800RESPECT.org.au. In an emergency, call 000.