Child safety online Archives | Page 12 of 86

FTC targets nudify tools over TAKE IT DOWN Act compliance

The US Federal Trade Commission has sent warning letters to 12 websites offering so-called ‘nudify’ tools, expanding its enforcement of the TAKE IT DOWN Act.

The law requires covered platforms to provide a process for people to request the removal of intimate photos or videos shared online without consent, and to remove that content within 48 hours of a valid request.

The letters were sent to companies offering tools that can take a clothed image of a person and generate non-consensual sexualised images by removing clothing. According to the FTC, the companies appear to be violating the law by failing to provide a process for victims to request the removal of non-consensual intimate images from their platforms.

FTC Chairman Andrew N. Ferguson said platforms ‘no longer have any excuses’ and must comply with their obligations under the TAKE IT DOWN Act or face consequences.

The letters urge the companies to comply with the law immediately. Platforms that fail to do so could face FTC legal action and civil penalties of up to $53,088 per violation.

The action follows the FTC’s start of TAKE IT DOWN Act enforcement on 19 May. Ahead of the deadline, Ferguson sent letters to major platforms, including Alphabet, Amazon, Apple, Meta, Microsoft, Reddit, Snapchat, TikTok and X, reminding them of their compliance obligations.

The FTC has also issued business guidance on compliance with the law, which was signed in May 2025 and gave companies one year to meet its requirements.

Why does it matter?

The warning letters show how the FTC is applying the TAKE IT DOWN Act directly to AI-powered nudify tools, not only mainstream social media platforms. The action reflects growing concern that generative AI can make image-based sexual abuse easier to create and distribute at scale, while placing stronger legal pressure on platforms to give victims a fast removal process.

Would you like to learn more about AI, tech, and digital diplomacy? If so, ask our Diplo chatbot!

Australia’s regulator targets AI-nudify platform over child safety and deepfake risks

Australia’s eSafety Commissioner has begun enforcement action against another AI-powered ‘nudify’ service accused of failing to protect children from exposure to sexually explicit deepfake images.

The regulator issued a formal Direction to Comply to one of the most visited nudify services in Australia, giving the provider 14 days to implement stronger protections preventing children from accessing the platform. eSafety said the service allows users to upload images of real people and generate sexually explicit deepfake content on demand.

The regulator warned that such technologies can facilitate non-consensual exploitation, cyberbullying, sexual extortion, image-based sexual abuse, misogynistic harassment and exploitation of minors. The service had attracted nearly 40,000 Australian visits per month as of March 2026, following a sharp increase in traffic over the previous six months.

The enforcement action was taken under Australia’s Age-Restricted Material Codes, which came into force in March 2026. The codes are designed to prevent children from accessing or being exposed to age-restricted material, including pornography, high-impact violence, self-harm, suicide or disordered eating content.

eSafety said the Argentina-based provider failed to respond to earlier engagement after the codes took effect and had not committed to improving protections for children. The regulator chose not to name the service to avoid inadvertently promoting it.

If the service does not meet the requirements within the 14-day timeframe, eSafety may pursue further action, including civil penalties of up to AU$49.5 million and delisting notices to search engine providers that help facilitate access to the site.

The action follows earlier enforcement in late 2025 that led three widely used nudify services, which had reportedly been used to generate child sexual exploitation material in schools, to withdraw from Australia. Those services have since relaunched under new ownership with additional safety measures, including mandatory age assurance.

Why does it matter?

The case shows how online safety regulators are beginning to apply age-assurance and child protection rules directly to generative AI services. Nudify platforms are treated as high-risk because they can enable non-consensual sexualised deepfakes, image-based abuse and exploitation involving minors at scale. Australia’s enforcement approach also signals that regulators may target foreign-based AI services when they are accessible to local users and fail to implement safeguards.

Would you like to learn more about AI, tech and digital diplomacy? If so, ask our Diplo chatbot!

Snap, YouTube, TikTok and Meta settle Kentucky school district lawsuit

At mid-May, Snap, YouTube and TikTok have reached a settlement in a lawsuit brought by the Breathitt County School District in Kentucky. The school district alleged that social media platforms contributed to learning disruption, mental health concerns, and additional financial pressures on schools.

Terms of the settlement have not been disclosed. Meta remained a defendant in the same litigation and was scheduled to proceed to trial on June 15th. However, on May 21st, the company also reached a settlement. The case is one of a broader series of lawsuits involving social media platforms and alleged harms affecting minors and schools.

This follows earlier related cases settled by Snap and TikTok. The companies have faced multiple lawsuits related to alleged harms associated with social media use. In a separate case, a jury awarded damages to a plaintiff in litigation involving Google and Meta. Meta has also recently been ordered to pay $375 million in a separate case brought by New Mexico’s attorney general.

Beyond seeking monetary awards of $60 millions, plaintiffs and state authorities have also called for changes to platform design and online safety measures affecting minors. Additional lawsuits involving social media platforms and youth safety issues remain ongoing in US courts.

Would you like to learn more about AI, tech, and digital diplomacy? If so, ask our Diplo chatbot!

Researchers analyse framing of youth social media use in Australian policy discussions

A new paper, published in the journal SAGE Journals, examines how concerns around children’s social media use are framed in debates surrounding regulation and online safety. The study was authored by researchers Justine Humphry, Catherine Page Jeffery, and Jonathon Hutchinson.

According to the paper, the research focuses on debates related to Australia’s proposed social media age restriction laws. The authors argue that media coverage and political discussion increasingly frame teenagers’ social media use as an issue requiring stronger regulation.

The paper explores how discussions about risk, safety, and childhood vulnerability influenced calls for age-based restrictions on social media. The study also examines how age-based regulation emerged as a proposed policy response to wider concerns about digital platforms and young people.

The research contributes to broader academic discussions on digital governance, youth agency, and media regulation. The research was published as an open-access article in Convergence and centres on policy debates in Australia.

Would you like to learn more about AI, tech and digital diplomacy? If so, ask our Diplo chatbot!

eSafety Commissioner and Sport Integrity Australia focus on online harms in sport

Australia’s eSafety Commissioner and Sport Integrity Australia have launched a joint initiative focused on online safety in sport.

The Online Safety in Sport Summit brought together representatives from sporting organisations, government agencies, researchers, law enforcement, and technology companies. The discussions focused on cyberbullying, online harassment, and harmful digital behaviour affecting athletes and sporting communities.

During the summit, Australia’s eSafety Commissioner Julie Inman Grant said harmful behaviour linked to sport increasingly occurs across social media, messaging applications, and online communities.

Research presented during the summit, titled ‘The Digital Sideline’, found that nearly one in five children participating in organised sport reported experiencing cyberbullying related to sporting activities.

Officials in Australia said that many reported online harms involved peers, including teammates and competitors, and occurred through private messages and group chats.

Participants highlighted the importance of prevention measures, early intervention, and cooperation between sporting organisations, regulators, and technology companies.

Why does it matter?

Online abuse within sport is becoming an increasingly significant policy and governance issue as digital platforms reshape athlete visibility, fan interaction, and youth participation. Cyberbullying, online harassment, and hate speech can affect mental health, athlete safety, participation rates, and broader social cohesion.

Would you like to learn more about AI, tech and digital diplomacy? If so, ask our Diplo chatbot!

FTC begins TAKE IT DOWN Act enforcement

The US Federal Trade Commission has begun enforcing the TAKE IT DOWN Act, requiring covered platforms to remove non-consensual intimate images and known identical copies within 48 hours of a valid request.

The FTC enforces Section 3 of the law, which sets a 19 May 2026 deadline for covered platforms to create a process allowing victims and survivors to request the removal of intimate photos or videos shared online without their consent.

As part of its enforcement role, the agency has launched TakeItDown.ftc.gov, a website where victims and survivors can submit complaints about platforms that fail to act on valid removal requests or have not created a removal process.

FTC Chairman Andrew N. Ferguson said the law gives families recourse against digital exploitation and extortion, particularly where children are affected. He added that ‘in the age of AI, anyone can be targeted’, making protections against abuse especially urgent.

The FTC has also published guidance for consumers whose non-consensual intimate images are posted online and separate guidance for businesses on complying with the law.

Ahead of the enforcement deadline, Ferguson sent letters to major platforms, including Alphabet, Amazon, Apple, Automattic, Bumble, Discord, Match Group, Meta, Microsoft, Pinterest, Reddit, SmugMug, Snapchat, TikTok and X, reminding them of their obligations under the TAKE IT DOWN Act.

Why does it matter?

The law gives victims of non-consensual intimate image abuse a faster route to removal and gives the FTC a direct enforcement role when platforms fail to comply. Its importance has grown with the spread of AI-generated sexual abuse material and deepfake tools, which can make intimate image abuse easier to create, copy and distribute at scale.

Would you like to learn more about AI, tech, and digital diplomacy? If so, ask our Diplo chatbot!

UK regulator updates online safety guidance on AI-generated intimate imagery

Ofcom has announced proposed measures intended to strengthen protections against illegal intimate image abuse online, including AI-generated explicit deepfakes and non-consensual image sharing.

The UK regulator said it is updating its Illegal Content Codes to recommend that certain online platforms use automated detection technologies to identify illegal intimate images.

According to Ofcom, hash matching systems convert images into digital identifiers that can help platforms detect repeated uploads of harmful content. Ofcom specifically referenced the StopNCII database as a recommended tool for platforms implementing the technology.

Ofcom said the measures are intended to improve protections against AI-generated intimate imagery and digitally manipulated sexual content.

The recommendations complement recent UK legislation addressing non-consensual intimate imagery and AI-enabled nudification tools.

Ofcom said the updated Illegal Content Codes are expected to enter into force in autumn 2026, subject to parliamentary approval. The regulator also said additional online safety measures under consultation may be announced later in the year.

The measures form part of the UK’s implementation of the Online Safety Act and related online safety obligations for digital platforms.

Why does it matter?

AI-generated deepfakes and synthetic sexual imagery are rapidly becoming major online safety and digital rights concerns globally. Regulators increasingly fear that existing moderation systems cannot keep pace with the scale and speed of AI-generated abuse. Ofcom’s decision illustrates how governments are beginning to shift towards mandatory or strongly encouraged proactive detection systems, particularly for highly harmful content involving intimate imagery, harassment, and exploitation.

Would you like to learn more about AI, tech and digital diplomacy? If so, ask our Diplo chatbot!

UK’s Ofcom accepts X commitments on illegal hate and terror content moderation

Ofcom has accepted a series of public commitments from X aimed at strengthening protections for UK users against illegal hate speech and terrorist content under the Online Safety Act framework.

Under the commitments, X will review suspected illegal terrorist and hate content reported through its dedicated UK illegal content reporting tool within an average of 24 hours. As a backstop, the platform will review at least 85% of such reports within 48 hours. Ofcom said the targets, if met, would give UK users some of the strongest protections on X globally.

X is also committed to engaging external experts on reporting systems for illegal hate and terror content, following concerns from organisations that reports submitted to the platform were not always clearly acknowledged or acted on. The company also said it would withhold access to accounts reported for posting illegal terrorist content in the UK if it determines they are operated by or on behalf of a terrorist organisation proscribed in the UK.

Ofcom said X will submit quarterly performance data over the next 12 months so the regulator can monitor whether the platform is meeting its commitments. The regulator added that its broader compliance programme examining how major social media services handle illegal hate and terrorist material remains ongoing.

The announcement comes amid wider scrutiny of illegal hate content on major social media platforms. Ofcom said evidence gathered from civil society and expert organisations, including the Antisemitism Policy Trust, Tech Against Terrorism and Tell MAMA, indicates that such content persists on some of the largest social media sites.

Ofcom also noted that its investigation into X’s Grok remains ongoing, focusing on the company’s compliance with duties to deal with illegal content and the systems it has in place to do so.

Why does it matter?

The commitments show how the UK’s Online Safety Act is beginning to translate into concrete performance expectations for major platforms. Review-time targets, expert engagement and regular reporting to Ofcom could make illegal hate and terrorist content moderation more measurable. Still, the wider test will be whether X delivers these protections in practice and whether similar pressure is applied across other large platforms.

Would you like to learn more about AI, tech and digital diplomacy? If so, ask our Diplo chatbot!

UK committee urges stronger online safety protections

The UK Parliament’s Science, Innovation and Technology Committee has urged the government to strengthen online safety protections for young people, following evidence on proposals to restrict social media access for under-16s.

Committee Chair Dame Chi Onwurah wrote to Science, Innovation and Technology Secretary Liz Kendall and AI and Online Safety Minister Kanishka Narayan after an evidence session on age-based restrictions.

The committee said there is strong and consistent evidence of significant individual harms linked to social media use, alongside a growing body of evidence showing wider negative impacts. It said there is a clear need to protect people, especially young users, from those harms.

The letter argues that responsibility for preventing harm should not rest solely on young people or parents. It says government inaction on online safety is not an option and calls for stronger enforcement of existing age restrictions

The committee also urged the government to revisit its July 2025 report on social media misinformation. Although the government accepted almost all of the report’s conclusions, the committee said it rejected almost all recommendations for change. It is now calling for action on misinformation, harmful algorithms, and online harms in the new parliamentary session.

Dame Chi Onwurah said: ‘The status quo, where social media companies are neither accountable nor responsible for preventing harms, isn’t acceptable. It’s clear social media can cause real harm and more must be done to protect people, especially young users. If any other consumer product caused these harms, it would’ve been recalled or changed. Shouldn’t the same be true for social media services and design features?’

She added: ‘The government must urgently address gaps in the regulation, legislation and enforcement of online safety. It should revisit and adopt my committee’s previous recommendations on tackling misinformation and harmful algorithms and bring forward legislation to effectively tackle online harms in the new parliamentary session.’

Would you like to learn more about AI, tech, and digital diplomacy? If so, ask our Diplo chatbot!

AI governance priorities outlined by EU at UN dialogue

The European Union has called for the UN Global Dialogue on AI Governance to focus on responsible innovation, human rights, capacity-building and stronger interoperability between AI governance frameworks.

In a statement delivered on behalf of the EU and its member states, the bloc said the dialogue should examine AI’s social, economic, ethical, cultural, linguistic, technical and environmental implications. It also argued that responsible AI innovation should be framed not only as a risk-management challenge, but also as an opportunity for public benefit in areas such as education and government.

The EU urged participants to address who controls the data, compute and value chains behind AI systems. It also highlighted linguistic and cultural diversity, warning that AI systems trained mainly on a limited number of languages can produce less accurate and more costly outputs for speakers of underrepresented languages.

Capacity-building was presented as a core condition for effective AI governance, particularly for developing countries. The EU said countries and institutions need the skills, systems and human capacity to evaluate, question and deploy AI responsibly, while treating AI infrastructure as a matter of public interest rather than only market access or proprietary control.

The statement also identified agentic AI as an emerging governance frontier, arguing that such systems raise new questions around accountability, oversight and control that existing frameworks do not yet adequately address.

On safe and trustworthy AI, the EU called for greater compatibility between governance approaches to prevent regulatory arbitrage and support responsible cross-border deployment. It said trust should not rely only on self-assessment or voluntary disclosure, but also on auditability, traceability, validation mechanisms, certification approaches and evaluation frameworks for high-risk systems.

The EU also urged a human-centric, human rights-based approach grounded in international law. It identified AI-facilitated gender-based violence, harmful AI-generated content affecting children and older persons, manipulative algorithmic systems, data exploitation and AI-enabled surveillance as areas requiring dedicated attention.

The statement called for the UN dialogue to build on existing initiatives, including those led by UNESCO, ITU, UNDP, OHCHR, GPAI, the Council of Europe, the Hiroshima Process and AI summit processes. The EU also supported more interactive thematic sessions, continuity between dialogue editions and a co-chairs’ summary reflecting both converging and diverging views.

Why does it matter?

The EU statement shows how global AI governance debates are moving beyond broad principles towards questions of implementation, institutional capacity and interoperability between frameworks. By linking AI infrastructure, human rights, auditability and agentic AI, the EU is signalling that future international coordination will need to address both today’s deployment risks and the governance challenges posed by more autonomous systems.

Would you like to learn more about AI, tech, and digital diplomacy? If so, ask our Diplo chatbot!