UNICEF Australia: Algorithmic Kids
UNICEF is the United Nations Children’s Fund, working in 190 countries for the survival, protection and development of every child, with a focus on the lives of children who are the most disadvantaged and excluded.
UNICEF Australia has been leading the way in respect of digital technology and its impacts on young Australians and has a goal of Australia being the international champion of child- centred AI – a global hub for innovation in safe and responsible AI for kids, mitigating its risks and maximising its opportunities.
AI is rapidly changing childhood in Australia. From interacting with AI chatbots that are almost indistinguishable from a person, to generating new ideas, images and videos seemingly out of thin air, children in Australia are already living AI-augmented lives.
UNICEF Australia asked AWO to undertake a rapid review of some of the most pressing impacts of AI on Australian children.
-
Deepfakes and nudification tools
A nudification tool is a type of GenAI application – often in the form of an app or website – that uses AI to create sexually explicit deepfake images of real people, typically by digitally removing clothing from a source image. These tools are designed to distort reality and simulate nudity, often without the subject’s knowledge or consent. While marketed as entertainment or novelty, they are increasingly being used to abuse, exploit, and humiliate children and young people. The eSafety Commissioner has issued an urgent call for schools to report deepfake incidents to appropriate authorities as the rapid proliferation of ‘nudify’ apps online takes a growing toll on communities around Australia.
-
AI Companions
The emergence of AI companions – i.e. chatbots and virtual agents designed to simulate human-like relationships – has introduced a new dimension to children’s digital lives. These systems, powered by large language models, are increasingly accessible through mobile apps, web platforms, and social media-integrated services. While marketed as tools for emotional support, entertainment, or self-expression, AI companions raise significant concerns regarding their psychological developmental, and social impacts on children and young people in Australia. Often romantic and sexualised in nature, a lack of safeguards is putting developmentally vulnerable children at risk.
-
Child exploitation material and AI
GenAI tools are being used to alter or create illegal child exploitation and abuse material. Ease of use is creating an explosion of illegal content, making an already complex problem worse. Without appropriate safeguards, there is a significant concern around the use of GenAI tools for the production of child exploitation and abuse material. These models also present unique challenges regarding the dissemination and detection of such material on the internet.
The report also looks at opportunities arising from AI in education, as well as the use of AI-powered online safety tools. These findings point to a complex picture. Whilst there are both risks and opportunities from AI for children in Australia, it is important that the harms are mitigated and addressed appropriately.
UNICEF Australia used the paper in their ongoing advocacy to pursue child-centred AI in Australia, including sharing the findings directly with government and tech industry stakeholders, and speaking to the research in the media, on panels, and at academic conferences. The paper has made a crucial contribution to several live policy debates and reforms in Australia, including the creation of the Australian AI Safety Institute, and the release of Australia’s National AI Plan.
Read the full ‘Algorithmic Kids’ report.
