eSafety did not reveal the specific number of complaints received. Nevertheless, despite the “rapid increase” being a significant issue, Inman Grant suggested the situation might be even more severe.

“We have heard from school leaders and education officials that deepfake occurrences are becoming more common, especially as children can easily access and misuse nudify apps in school environments,” she said.

‘A crisis affecting school communities’

“Worryingly, these apps have been used to shame, intimidate, and sexually blackmail children both in and outside of school. There are also reports of some of these images being exchanged among students for money.”

A ‘normalisation’ of creating deepfakes

“We are also seeing a range of motivations for the incidents, from sexual gratification, to intentionally causing harm, controlling or degrading the target of the image, right through to thinking it’s funny, building social status among peers, and curiosity in how the process of creating a deepfake works,” she said.

When it comes to young people, he warned this can happen online or in school communities, such as messaging apps, “where we have even more limited visibility of what they are sharing among themselves”.

‘We will not hesitate to take regulatory action’

“We will not hesitate to take regulatory action,” she said.

“But the more important thing would be raising awareness and an ethical understanding of technology among school kids.”