Australian eSafety Commissioner Issues Urgent Advisory Over Rising AI Manipulation of School Imagery and Data Scraping Risks

In a significant move to address the escalating threats posed by generative artificial intelligence, Australia’s eSafety Commissioner issued a comprehensive advisory on July 28, 2026, urging educational institutions nationwide to overhaul their digital image-sharing protocols. The warning comes in response to a disturbing surge in the misuse of school-related photographs, where images of students and staff are being harvested from official websites and social media accounts to create non-consensual, AI-generated content. The commissioner’s office revealed that between January and March 2026, it received over 100 formal reports involving anonymous accounts that targeted specific school communities. These accounts utilized sophisticated AI tools to transform benign school photos into harmful materials, including sexualized deepfakes, face swaps, and other forms of manipulated imagery, which were subsequently distributed across popular social media platforms such as TikTok and Instagram.

The advisory highlights a critical vulnerability in how schools manage their digital footprints. For years, the practice of sharing student achievements, sporting events, and faculty milestones online has been a staple of school community engagement. However, the rapid advancement of AI technology has turned these public archives into a "raw material" source for malicious actors. The eSafety Commissioner’s intervention underscores a pivot from general online safety advice to a specific, urgent call for technical and policy-based defenses against AI-driven exploitation.

The Evolution of Digital Risk: A Chronology of AI Misuse

The current crisis is the culmination of several years of mounting evidence regarding the unregulated intersection of AI and personal data. To understand the gravity of the July 28 advisory, it is necessary to trace the timeline of events that led to this regulatory flashpoint.

In early 2024, Human Rights Watch (HRW) published a landmark report documenting that the personal photographs of Australian children were being systematically scraped from the internet. This data harvesting was not merely for academic research but was being used to train large-scale AI models. Many of these images were sourced from school websites, where privacy settings were often inadequate to prevent automated web crawlers from indexing and downloading high-resolution imagery.

By late 2024 and throughout 2025, the proliferation of "easy-to-use" deepfake software allowed individuals with minimal technical expertise to generate realistic, manipulated content. The barrier to entry for digital harassment dropped significantly, leading to a rise in localized incidents within Australian schools. By February 2026, international human rights organizations began documenting the "human cost" of these unregulated tools, noting that children were increasingly becoming the primary targets of AI-generated sexualized content.

The surge in reports recorded by the eSafety Commissioner in the first quarter of 2026 served as the final catalyst for the current advisory. The data revealed a pattern: anonymous accounts would scrape an entire faculty directory or a "student of the month" gallery, apply AI filters or face-swapping algorithms, and then re-upload the content to social media to humiliate or extort the victims.

Data Scraping and the Mechanics of AI Training

The advisory brings to light the technical process of "scraping," a method where automated scripts scan the web to collect data. In the context of AI, these images are fed into neural networks to teach the software how to replicate human features, expressions, and environments. When schools post high-quality images of children, they are inadvertently providing high-quality training data.

Beyond the creation of deepfakes, there is the secondary risk of metadata and contextual information. Photos stored in datasets used to train AI models often retain "tags" or descriptions. These can include names, school locations, specific event dates, and even the schedules of students. Human Rights Watch has warned that once this information is ingested into an AI model’s training set, it becomes nearly impossible to "unlearn" or delete. This creates a permanent digital shadow that can follow a child into adulthood, exposing them to potential identity theft, stalking, or long-term reputational damage.

The eSafety Commissioner noted that the "indefinite accessibility" of these images is one of the most harrowing aspects of the problem. Unlike traditional cyberbullying, where a post might be deleted by a platform moderator, AI-generated content can be replicated and modified infinitely, resurfacing across different jurisdictions and platforms beyond the reach of local law enforcement.

Regulatory Responses and the Children’s Online Privacy Code

The Australian government is currently in the final stages of developing a legislative framework to combat these issues. The proposed Children’s Online Privacy Code, which saw its official exposure draft published in March, is expected to be finalized by the end of 2026. This code represents a critical opportunity to align Australian law with international human rights standards, which dictate that the "best interests of the child" must be the primary consideration in all digital data processing.

The draft code aims to impose strict obligations on technology companies, requiring them to implement "privacy by design" for any service likely to be accessed by children. However, advocacy groups and the eSafety Commissioner are now calling for even more explicit prohibitions. There is a growing consensus that the code should specifically:

1. Prohibit Data Scraping: Explicitly outlaw the automated harvesting of children’s images and personal data for the purpose of training AI models without verified, high-level parental consent.
2. Ban Digital Likeness Replication: Create legal barriers against the unauthorized digital manipulation or replication of a minor’s likeness, regardless of whether the intent is deemed "parody" or "malicious."
3. Mandate Transparency: Require AI developers to provide clear documentation on the sources of their training data and provide a mechanism for the immediate removal of children’s data upon request.

The eSafety Commissioner’s advisory serves as a bridge between current school policies and this forthcoming legislation. By urging schools to act now, the commissioner is attempting to mitigate harm while the legal machinery catches up to the technology.

Stakeholder Reactions and Institutional Impact

The educational sector has reacted to the advisory with a mixture of concern and a call for more resources. School administrators have pointed out that they are caught between the desire to celebrate student achievements and the need to protect student safety.

"The digital landscape has changed so rapidly that policies written even two years ago are now obsolete," said a representative from a leading Australian secondary school association. "We are now advising schools to move away from public-facing galleries and toward secure, password-protected portals for parents. But even then, the risk of a ‘trusted’ user downloading and sharing those images remains a concern."

Human rights advocates have been more pointed in their assessment. A spokesperson for Human Rights Watch emphasized that the burden of protection should not fall solely on schools and parents. "Children should not have to worry that a photograph from their school play will be turned into a weapon against them," the spokesperson stated. "The responsibility lies with the tech companies that build these tools and the governments that allow them to operate without safeguards. Australia must make it clear as a matter of law that children’s images are not raw material for AI models."

Social media platforms, including TikTok and Instagram, have also come under fire in the advisory. While these platforms have policies against non-consensual sexual content, the eSafety Commissioner’s report suggests that the "anonymous accounts" mentioned are often able to bypass automated moderation filters by using subtle AI manipulations that do not immediately trigger standard "not-safe-for-work" (NSFW) detectors.

Analysis: The Future of Digital Identity and Privacy

The implications of the eSafety Commissioner’s advisory extend far beyond the schoolyard. This situation represents a fundamental shift in the concept of digital identity. In the pre-AI era, a photograph was a static record of a moment in time. In the AI era, a photograph is a "dynamic asset" that can be reconfigured into an infinite number of scenarios.

For children, who are still developing their identities, the impact of seeing a manipulated version of themselves online can be psychologically devastating. The rise of "face swapping" and sexualized deepfakes contributes to a culture of digital insecurity, where the victim has no control over their own likeness. This undermines the right to privacy and the right to dignity, both of which are enshrined in international law.

Furthermore, the advisory highlights a growing "regulatory gap." While Australia has robust laws regarding child pornography, the legal definitions of "AI-generated" versus "real" imagery are often blurred in current statutes. Prosecutors may find it difficult to bring charges under existing laws if the imagery is entirely synthetic, even if it is clearly based on the likeness of a real child. The Children’s Online Privacy Code is seen as the necessary tool to close this gap by focusing on the data and the process of creation rather than just the final output.

Recommended Actions for Schools and Parents

In light of the July 28 advisory, the eSafety Commissioner has recommended several immediate steps for educational institutions:

• Review Image Permissions: Move from "opt-out" to "opt-in" systems for all social media and website photography.
• Lower Image Resolution: Post only low-resolution images online, which are less useful for AI training models.
• Utilize Watermarking: Apply visible and invisible watermarks to school photos to discourage scraping and aid in tracking if images are misused.
• Limit Identifiable Information: Avoid pairing student names with photos and remove metadata (EXIF data) that includes location and time stamps before uploading images.
• Education and Awareness: Implement curriculum-based learning that teaches students about the risks of AI manipulation and how to report harmful content.

As the government moves toward finalizing the Children’s Online Privacy Code later this year, the eSafety Commissioner’s advisory stands as a stark reminder of the high stakes involved. The goal is to ensure that the digital environment remains a space for learning and growth, rather than a hunting ground for AI-driven exploitation. The message from the Australian authorities is clear: the protection of children in the age of artificial intelligence requires not just better habits, but a fundamental change in the law.