Child safety online Archives | Page 6 of 80

eSafety Commissioner and Sport Integrity Australia focus on online harms in sport

Australia’s eSafety Commissioner and Sport Integrity Australia have launched a joint initiative focused on online safety in sport.

The Online Safety in Sport Summit brought together representatives from sporting organisations, government agencies, researchers, law enforcement, and technology companies. The discussions focused on cyberbullying, online harassment, and harmful digital behaviour affecting athletes and sporting communities.

During the summit, Australia’s eSafety Commissioner Julie Inman Grant said harmful behaviour linked to sport increasingly occurs across social media, messaging applications, and online communities.

Research presented during the summit, titled ‘The Digital Sideline’, found that nearly one in five children participating in organised sport reported experiencing cyberbullying related to sporting activities.

Officials in Australia said that many reported online harms involved peers, including teammates and competitors, and occurred through private messages and group chats.

Participants highlighted the importance of prevention measures, early intervention, and cooperation between sporting organisations, regulators, and technology companies.

Why does it matter?

Online abuse within sport is becoming an increasingly significant policy and governance issue as digital platforms reshape athlete visibility, fan interaction, and youth participation. Cyberbullying, online harassment, and hate speech can affect mental health, athlete safety, participation rates, and broader social cohesion.

Would you like to learn more about AI, tech and digital diplomacy? If so, ask our Diplo chatbot!

FTC begins TAKE IT DOWN Act enforcement

The US Federal Trade Commission has begun enforcing the TAKE IT DOWN Act, requiring covered platforms to remove non-consensual intimate images and known identical copies within 48 hours of a valid request.

The FTC enforces Section 3 of the law, which sets a 19 May 2026 deadline for covered platforms to create a process allowing victims and survivors to request the removal of intimate photos or videos shared online without their consent.

As part of its enforcement role, the agency has launched TakeItDown.ftc.gov, a website where victims and survivors can submit complaints about platforms that fail to act on valid removal requests or have not created a removal process.

FTC Chairman Andrew N. Ferguson said the law gives families recourse against digital exploitation and extortion, particularly where children are affected. He added that ‘in the age of AI, anyone can be targeted’, making protections against abuse especially urgent.

The FTC has also published guidance for consumers whose non-consensual intimate images are posted online and separate guidance for businesses on complying with the law.

Ahead of the enforcement deadline, Ferguson sent letters to major platforms, including Alphabet, Amazon, Apple, Automattic, Bumble, Discord, Match Group, Meta, Microsoft, Pinterest, Reddit, SmugMug, Snapchat, TikTok and X, reminding them of their obligations under the TAKE IT DOWN Act.

Why does it matter?

The law gives victims of non-consensual intimate image abuse a faster route to removal and gives the FTC a direct enforcement role when platforms fail to comply. Its importance has grown with the spread of AI-generated sexual abuse material and deepfake tools, which can make intimate image abuse easier to create, copy and distribute at scale.

Would you like to learn more about AI, tech, and digital diplomacy? If so, ask our Diplo chatbot!

UK regulator updates online safety guidance on AI-generated intimate imagery

Ofcom has announced proposed measures intended to strengthen protections against illegal intimate image abuse online, including AI-generated explicit deepfakes and non-consensual image sharing.

The UK regulator said it is updating its Illegal Content Codes to recommend that certain online platforms use automated detection technologies to identify illegal intimate images.

According to Ofcom, hash matching systems convert images into digital identifiers that can help platforms detect repeated uploads of harmful content. Ofcom specifically referenced the StopNCII database as a recommended tool for platforms implementing the technology.

Ofcom said the measures are intended to improve protections against AI-generated intimate imagery and digitally manipulated sexual content.

The recommendations complement recent UK legislation addressing non-consensual intimate imagery and AI-enabled nudification tools.

Ofcom said the updated Illegal Content Codes are expected to enter into force in autumn 2026, subject to parliamentary approval. The regulator also said additional online safety measures under consultation may be announced later in the year.

The measures form part of the UK’s implementation of the Online Safety Act and related online safety obligations for digital platforms.

Why does it matter?

AI-generated deepfakes and synthetic sexual imagery are rapidly becoming major online safety and digital rights concerns globally. Regulators increasingly fear that existing moderation systems cannot keep pace with the scale and speed of AI-generated abuse. Ofcom’s decision illustrates how governments are beginning to shift towards mandatory or strongly encouraged proactive detection systems, particularly for highly harmful content involving intimate imagery, harassment, and exploitation.

Would you like to learn more about AI, tech and digital diplomacy? If so, ask our Diplo chatbot!

UK’s Ofcom accepts X commitments on illegal hate and terror content moderation

Ofcom has accepted a series of public commitments from X aimed at strengthening protections for UK users against illegal hate speech and terrorist content under the Online Safety Act framework.

Under the commitments, X will review suspected illegal terrorist and hate content reported through its dedicated UK illegal content reporting tool within an average of 24 hours. As a backstop, the platform will review at least 85% of such reports within 48 hours. Ofcom said the targets, if met, would give UK users some of the strongest protections on X globally.

X is also committed to engaging external experts on reporting systems for illegal hate and terror content, following concerns from organisations that reports submitted to the platform were not always clearly acknowledged or acted on. The company also said it would withhold access to accounts reported for posting illegal terrorist content in the UK if it determines they are operated by or on behalf of a terrorist organisation proscribed in the UK.

Ofcom said X will submit quarterly performance data over the next 12 months so the regulator can monitor whether the platform is meeting its commitments. The regulator added that its broader compliance programme examining how major social media services handle illegal hate and terrorist material remains ongoing.

The announcement comes amid wider scrutiny of illegal hate content on major social media platforms. Ofcom said evidence gathered from civil society and expert organisations, including the Antisemitism Policy Trust, Tech Against Terrorism and Tell MAMA, indicates that such content persists on some of the largest social media sites.

Ofcom also noted that its investigation into X’s Grok remains ongoing, focusing on the company’s compliance with duties to deal with illegal content and the systems it has in place to do so.

Why does it matter?

The commitments show how the UK’s Online Safety Act is beginning to translate into concrete performance expectations for major platforms. Review-time targets, expert engagement and regular reporting to Ofcom could make illegal hate and terrorist content moderation more measurable. Still, the wider test will be whether X delivers these protections in practice and whether similar pressure is applied across other large platforms.

Would you like to learn more about AI, tech and digital diplomacy? If so, ask our Diplo chatbot!

UK committee urges stronger online safety protections

The UK Parliament’s Science, Innovation and Technology Committee has urged the government to strengthen online safety protections for young people, following evidence on proposals to restrict social media access for under-16s.

Committee Chair Dame Chi Onwurah wrote to Science, Innovation and Technology Secretary Liz Kendall and AI and Online Safety Minister Kanishka Narayan after an evidence session on age-based restrictions.

The committee said there is strong and consistent evidence of significant individual harms linked to social media use, alongside a growing body of evidence showing wider negative impacts. It said there is a clear need to protect people, especially young users, from those harms.

The letter argues that responsibility for preventing harm should not rest solely on young people or parents. It says government inaction on online safety is not an option and calls for stronger enforcement of existing age restrictions

The committee also urged the government to revisit its July 2025 report on social media misinformation. Although the government accepted almost all of the report’s conclusions, the committee said it rejected almost all recommendations for change. It is now calling for action on misinformation, harmful algorithms, and online harms in the new parliamentary session.

Dame Chi Onwurah said: ‘The status quo, where social media companies are neither accountable nor responsible for preventing harms, isn’t acceptable. It’s clear social media can cause real harm and more must be done to protect people, especially young users. If any other consumer product caused these harms, it would’ve been recalled or changed. Shouldn’t the same be true for social media services and design features?’

She added: ‘The government must urgently address gaps in the regulation, legislation and enforcement of online safety. It should revisit and adopt my committee’s previous recommendations on tackling misinformation and harmful algorithms and bring forward legislation to effectively tackle online harms in the new parliamentary session.’

Would you like to learn more about AI, tech, and digital diplomacy? If so, ask our Diplo chatbot!

AI governance priorities outlined by EU at UN dialogue

The European Union has called for the UN Global Dialogue on AI Governance to focus on responsible innovation, human rights, capacity-building and stronger interoperability between AI governance frameworks.

In a statement delivered on behalf of the EU and its member states, the bloc said the dialogue should examine AI’s social, economic, ethical, cultural, linguistic, technical and environmental implications. It also argued that responsible AI innovation should be framed not only as a risk-management challenge, but also as an opportunity for public benefit in areas such as education and government.

The EU urged participants to address who controls the data, compute and value chains behind AI systems. It also highlighted linguistic and cultural diversity, warning that AI systems trained mainly on a limited number of languages can produce less accurate and more costly outputs for speakers of underrepresented languages.

Capacity-building was presented as a core condition for effective AI governance, particularly for developing countries. The EU said countries and institutions need the skills, systems and human capacity to evaluate, question and deploy AI responsibly, while treating AI infrastructure as a matter of public interest rather than only market access or proprietary control.

The statement also identified agentic AI as an emerging governance frontier, arguing that such systems raise new questions around accountability, oversight and control that existing frameworks do not yet adequately address.

On safe and trustworthy AI, the EU called for greater compatibility between governance approaches to prevent regulatory arbitrage and support responsible cross-border deployment. It said trust should not rely only on self-assessment or voluntary disclosure, but also on auditability, traceability, validation mechanisms, certification approaches and evaluation frameworks for high-risk systems.

The EU also urged a human-centric, human rights-based approach grounded in international law. It identified AI-facilitated gender-based violence, harmful AI-generated content affecting children and older persons, manipulative algorithmic systems, data exploitation and AI-enabled surveillance as areas requiring dedicated attention.

The statement called for the UN dialogue to build on existing initiatives, including those led by UNESCO, ITU, UNDP, OHCHR, GPAI, the Council of Europe, the Hiroshima Process and AI summit processes. The EU also supported more interactive thematic sessions, continuity between dialogue editions and a co-chairs’ summary reflecting both converging and diverging views.

Why does it matter?

The EU statement shows how global AI governance debates are moving beyond broad principles towards questions of implementation, institutional capacity and interoperability between frameworks. By linking AI infrastructure, human rights, auditability and agentic AI, the EU is signalling that future international coordination will need to address both today’s deployment risks and the governance challenges posed by more autonomous systems.

Would you like to learn more about AI, tech, and digital diplomacy? If so, ask our Diplo chatbot!

Italy lawsuit against Meta and TikTok tests child safety rules

A first hearing has taken place at the Milan Business Court in a case brought by MOIGE, the Italian Parents’ Movement, and a group of families against Meta and TikTok over the protection of minors on social media platforms.

According to MOIGE, the class-wide injunction seeks to protect around 3.5 million Italian children aged between 7 and 14 who are allegedly active on social platforms despite age restrictions. The organisation described the case as the first such action in Europe focused on protecting minors in the digital sector.

The hearing focused on preliminary objections, including challenges by lawyers for Meta and TikTok to the jurisdiction and competence of Italian courts to rule on the companies’ conduct. MOIGE said the platforms also contested documents submitted by its legal team concerning the alleged effects of recommendation algorithms on minors.

According to MOIGE, the documents refer to concerns around variable reinforcement mechanisms, infinite scrolling and behavioural profiling allegedly designed to maximise engagement among younger users. The organisation and the families’ lawyers argue that such design features raise concerns over addictive behaviour and wider risks to children’s well-being.

MOIGE’s lawyers urged the court to proceed quickly, arguing that delays could prolong potential harm affecting minors in Italy. The case will continue with further hearings, with the court expected to set the next steps in the proceedings.

Why does it matter?

The case could become an important test of how courts assess platform responsibility for children’s safety, age restrictions and recommendation systems. If the action advances, it may contribute to wider European debates on algorithmic design, age verification, addictive platform features and whether child online safety should be treated not only as a content moderation issue, but also as a consumer protection and public health concern.

Would you like to learn more about AI, tech and digital diplomacy? If so, ask our Diplo chatbot!

OpenAI sued over alleged ChatGPT role in Florida State University shooting

The family of a victim killed in the April 2025 Florida State University shooting has filed a federal lawsuit in Florida against OpenAI, alleging that ChatGPT enabled the attack. The lawsuit was filed on Sunday by Vandana Joshi, the widow of Tiru Chabba, who was killed alongside university dining director Robert Morales.

The complaint states that the accused shooter, Phoenix Ikner, engaged in extensive conversations with ChatGPT months before leading up to the incident. According to the suit, those exchanges included images and discussions about firearms he had acquired, ideological material, ideological far-right beliefs, and possible outcomes of violent attacks.

The chatbot is further accused of providing contextual information about campus activity and commenting on factors that could increase public attention in violent incidents. This is indicated by the fact that at one point, ChatGPT said, ‘if children are involved, even 2-3 victims can draw more attention’. The filing also claims Ikner asked about legal consequences and planning considerations shortly before the attack.

The lawsuit contends that OpenAI failed to identify escalating risk indicators within the conversations and did not adequately prevent harmful guidance. It argues the system ‘failed to connect the dots’ despite Ikner’s repeated questions about suicide, terrorism and mass shootings.

OpenAI has rejected responsibility for the attack, claiming its platform is not to blame. Company spokesperson Drew Pusateri said ChatGPT generated factual responses that could be found broadly across publicly available information and did not encourage or promote illegal activity. He also stated that OpenAI continues to strengthen safeguards to identify harmful intent, reduce misuse and respond appropriately when safety risks arise.

Joshi’s complaint argues that the system reinforced the shooter’s beliefs and failed to interrupt conversations involving violent ideation. The filing alleges the ChatGPT inflamed, validated and endorsed delusional thinking and contributed to planning discussions while ‘convincing him that violent acts can be required to bring about change’.

The lawsuit forms part of a broader wave of litigation involving AI systems and alleged harm. OpenAI is already facing separate lawsuits linked to incidents involving violence and suicide, raising wider questions about safeguards and user protection

Florida’s Attorney General James Uthmeier announced a criminal investigation into OpenAI and ChatGPT following a review of chat logs connected to the case. Uthmeier said in a statement that ‘If ChatGPT is a person it would be facing charges for murder’.

Would you like to learn more about AI, tech and digital diplomacy? If so, ask our Diplo chatbot!

Ofcom investigates adult platforms under Online Safety Act age-check rules

Ofcom has opened investigations into the providers of pimpbunny.com and kemono.cr to assess their compliance with age-check rules under the UK’s Online Safety Act.

The regulator said pornography services must use ‘highly effective’ age checks to determine whether users are over 18 before allowing access to pornographic material. The investigations will examine whether there are reasonable grounds to believe the providers have failed, or are failing, to comply with those duties.

Ofcom said it prioritised action against the providers based on the risk of harm posed by their services. The regulator took account of user numbers, including significant increases in traffic since age-check laws came into force last summer.

Separately, Ofcom has issued a provisional decision concerning fapello.com, saying it has reasonable grounds to believe the provider is in breach of its duties under the Online Safety Act. Fapello can make representations before Ofcom reaches a final decision.

Ofcom also expanded its ongoing investigation into XGroovy to examine whether it failed to respond adequately to formal information requests from the regulator. The developments form part of wider UK enforcement efforts around online child safety, age assurance and platform accountability under the Online Safety Act.

Why does it matter?

The investigations show that Ofcom is moving from guidance to enforcement under the UK’s Online Safety Act, particularly for services hosting pornographic material. Age assurance has become a central test of the UK’s child online safety regime, with regulators assessing not only whether platforms have age checks in place, but whether those checks are effective enough to prevent children from readily accessing explicit content.

Would you like to learn more about AI, tech and digital diplomacy? If so, ask our Diplo chatbot!

Data Protection Act regulations bring AI code requirement into force

The UK has brought into force regulations requiring the Information Commissioner to prepare a code of practice on the processing of personal data in relation to AI and automated decision-making.

The Data Protection Act 2018 (Code of Practice on Artificial Intelligence and Automated Decision-Making) Regulations 2026 were made on 16 April, laid before Parliament on 21 April, and came into force on 12 May. The regulations apply across England and Wales, Scotland and Northern Ireland.

Under the regulations, the Information Commissioner must prepare a code giving guidance on good practice in the processing of personal data under the UK GDPR and the Data Protection Act 2018 when developing and using AI and automated decision-making systems.

The code must also include guidance on good practice in the processing of children’s personal data. Automated decision-making is defined by reference to provisions in the UK GDPR and the Data Protection Act 2018 inserted through the Data (Use and Access) Act 2025.

The instrument also modifies the panel requirements for preparing or amending the code. Any panel established to consider the code must not consider or report on aspects relating to national security.

The explanatory note states that no full impact assessment was prepared for the instrument because the regulations themselves are not expected to have a significant impact on the private, voluntary or public sectors. The Information Commissioner must produce an impact assessment when preparing the code.

Why does it matter?

The regulations move UK guidance on AI, automated decision-making and personal data onto a statutory track. The eventual code could become an important reference point for organisations using AI systems that process personal data, particularly where automated decisions or children’s data are involved. For now, the main development is procedural: the Information Commissioner is required to prepare the code, while the practical compliance details will follow through that process.

Would you like to learn more about AI, tech, and digital diplomacy? If so, ask our Diplo chatbot!