Content Policy Archives | Digital Watch Observatory

Study raises concerns over AI-generated content on TikTok

New research from Kapwing suggests that AI-generated content now accounts for a significant share of videos shown on TikTok, raising concerns about content quality, authenticity and misinformation. The study suggests that nearly 59% of videos served to new users can be classified as AI-generated ‘slop’, with similarly high levels reported in feeds aimed at children.

Analysis across thousands of videos found that AI-generated material is particularly concentrated in the Kids category, where it accounts for around 57% of content. Science, education and health-related content also showed comparatively high levels of AI-generated production, while categories such as fitness, music and fashion remained largely dominated by human creators.

Researchers warn that the growing volume of AI-generated content could undermine information quality and increase exposure to misleading, repetitive or low-value material, particularly among younger audiences. Concerns focus on how algorithmic recommendation systems amplify such content, shaping early viewing experiences for new users.

In response, TikTok has introduced tools allowing users to adjust the amount of AI content in their feeds and launched initiatives aimed at improving AI literacy. Despite these measures, the findings suggest that AI-generated videos are becoming an increasingly prominent feature of the short-form social media ecosystem.

Why does it matter?

The findings highlight how generative AI is reshaping online content ecosystems by dramatically lowering the cost and effort required to produce large volumes of media. As recommendation algorithms prioritise engagement and scale, AI-generated content can spread rapidly, influencing what users see and how information is consumed.

The trend also raises broader questions about platform governance, content moderation and digital literacy. If synthetic content becomes increasingly difficult to distinguish from human-created material, platforms may face growing pressure to improve labelling, verification and recommendation systems. The issue is particularly significant for younger users and for content categories such as education, science and health, where misinformation can have wider societal consequences.

Would you like to learn more about AI, tech, and digital diplomacy? If so, ask our chatbot!

IWF backs Pope Leo XIV call for responsible AI development

The Internet Watch Foundation has welcomed Pope Leo XIV’s reflections on AI, arguing that AI systems must be developed with stronger safeguards to protect children from abuse.

In a blog post, the IWF said the Pope’s message that technology should serve the common good and remain subject to human judgement and accountability reflects the risks its analysts are already seeing online.

The organisation warned that AI is being used to generate highly realistic child sexual abuse images and videos at scale. It said the number of AI-generated child sexual abuse videos identified by the IWF in 2025 increased by more than 260%, with nearly two-thirds falling into the most severe category of abuse.

The IWF also raised concerns about AI-nudification tools, which can generate realistic sexualised images of children and other individuals. Following the Child Dignity in the Artificial Intelligence Era conference in Rome, the organisation joined more than 100 organisations and individuals in supporting calls for a global ban on such tools.

The IWF said AI safety should be built into products from the earliest stages of development. Through its Safety by Design work, the organisation is calling for companies to assess, test and mitigate risks before AI systems reach the public.

It also called for stronger regulation, global alignment and enforceable safety-by-design standards to prevent the creation and spread of AI-generated child sexual abuse material.

Why does it matter?

The IWF’s warning shows how generative AI is creating urgent child protection risks, especially through realistic synthetic abuse material and nudification tools. The issue is no longer only content moderation after harm occurs; it increasingly concerns model design, testing, deployment and accountability before AI systems reach users. That makes safety by design, developer responsibility and international coordination central to AI governance.

Would you like to learn more about AI, tech and digital diplomacy? If so, ask our Diplo chatbot!

European Parliament backs AI Act simplification and nudifier app ban

The European Parliament has approved amendments to parts of the EU AI Act as part of the digital omnibus package, postponing some compliance deadlines while adding a ban on AI systems used to create non-consensual sexually explicit content.

MEPs backed the changes with 423 votes in favour, 57 against and 174 abstentions. The measures are intended to simplify compliance for companies while preserving the AI Act’s risk-based structure and core safeguards.

Under the approved text, obligations for stand-alone high-risk AI systems would apply from 2 December 2027. Obligations for AI systems embedded as safety components in products covered by the EU sectoral safety and market surveillance legislation would apply from 2 August 2028.

The text also delays the obligation to watermark AI-generated content until 2 December 2026. By then, AI-generated content will need to be labelled in a machine-readable way to support transparency.

Parliament also approved a ban on AI systems that generate child sexual abuse material or create images, videos or audio depicting an identifiable person’s intimate parts or sexually explicit activities without consent. Providers would not be allowed to place such systems on the EU market unless they include adequate technical safeguards to prevent the creation of such material. The ban would also apply to deployers using systems for that purpose.

Other changes include removing overlapping requirements for AI used in machinery products, clarifying the definition of ‘safety component’, extending some SME exemptions to small mid-cap enterprises, and streamlining enforcement of certain general-purpose AI systems through the EU AI Office.

The legislation still needs formal adoption by the Council before it can enter into force.

Why does it matter?

The vote shows the EU trying to adjust the implementation AI the AI Act without reopening the law’s overall risk-based architecture. Delaying some deadlines could reduce legal uncertainty for businesses and give standards, guidance, and support measures more time to mature. At the same time, the proposed ban on nudification tools and AI-assisted child sexual abuse material addresses a fast-growing harm linked to generative AI, especially image and video manipulation targeting women and minors.

Would you like to learn more about AI, tech and digital diplomacy? If so, ask our Diplo chatbot!

Humanists UK urges government to adopt human-centred AI principles

Humanists UK has urged the UK government to place human dignity, democratic oversight and human flourishing at the centre of AI governance.

The call followed a House of Lords debate on the impact of AI on human relationships and society, during which peers discussed the ethical, social and regulatory challenges raised by rapidly advancing AI systems.

Humanists UK pointed out to the government the Luxembourg Declaration on Artificial Intelligence and Human Values, adopted by Humanists International in 2025. The declaration argues that AI should support human judgement, the common good, democratic governance, transparency, autonomy and protection from harm.

Lord Michael Cashman, a patron of Humanists UK and member of the All-Party Parliamentary Humanist Group, urged the government not to ‘reinvent the wheel’ and said the declaration already sets out principles relevant to AI governance.

Liberal Democrat peer Lord Clement-Jones said the debate showed a convergence of values across different traditions, including the need for democratic oversight, transparency and safeguards to ensure AI serves human beings rather than replacing them.

Responding for the government, Digital Economy Minister Baroness Lloyd of Effra said AI is already changing the economy, public services and human relationships. She said the government’s responsibility is to ensure that the transformation strengthens rather than diminishes the fabric of society.

Humanists UK said it has written to Baroness Lloyd and shared a copy of the Luxembourg Declaration.

Why does it matter?

The story reflects the growing role of civil society, religious groups and ethical movements in AI governance debates. While it does not signal a new UK policy, it shows how discussions on AI safety are broadening beyond technical risk to include human dignity, democratic accountability, transparency, autonomy and the public interest. Such value-based frameworks may influence how governments frame future AI regulation, assurance and safeguards.

Would you like to learn more about AI, tech and digital diplomacy? If so, ask our Diplo chatbot!

Children’s online safety and screen time under growing UK scrutiny

The UK Government Office for Science has highlighted the need for evidence-led policy on children’s online lives, warning that digital technologies bring both benefits and risks while long-term evidence remains limited.

In an article published on the GOV.UK, the Government Chief Scientific Adviser noted that 97% of UK teenagers aged 13 to 15 now own a mobile phone, while almost one-fifth of children aged three to five also own one. Children aged eight to nine spend an average of two hours per day online, rising to four hours for those aged 13 to 14, excluding gaming time.

The article said children use digital platforms to maintain friendships, access communities, and find support, and that some are also using AI companions for well-being and emotional regulation. AI tools are increasingly being used for learning and schoolwork, with around half of children reporting AI use.

However, the government adviser warned that children face risks including harmful content, cyberbullying, privacy breaches, false or misleading information, unlimited scrolling, personalised algorithms and other features designed to maximise engagement.

The article said there is not enough robust long-term data to determine with confidence how digital technologies are affecting children. It also warned that the use of AI should not prevent children from developing skills such as written expression and critical thinking.

The Government Office for Science said stronger evidence and continued evaluation are needed to inform policy, including the Department for Science, Innovation and Technology’s consultation on the impacts of growing up online, which covers social media, AI chatbots, gaming sites and other online services.

Why does it matter?

The article is relevant because it frames children’s online safety as an evidence and governance challenge, not only a moral panic over screen time. UK policymakers are weighing restrictions on social media, gaming platforms, AI chatbots and other online services, but the Government Office for Science stresses that long-term evidence remains incomplete. That makes transparency, evaluation and proportional safeguards central to future online safety policy.

Would you like to learn more about AI, tech, and digital diplomacy? If so, ask our chatbot!

Canada introduces Safe Social Media Act targeting online harms and AI chatbots

Canada has introduced the Safe Social Media Act, legislation that would establish new online safety requirements for social media platforms and certain AI chatbot services. Bill C-34 aims to make regulated services more accountable for addressing online harms before they occur.

The Safe Social Media Act would create a new legislative and regulatory framework through the proposed Digital Safety Act. Regulated services would be required to identify, assess and mitigate risks on their platforms, implement safety-by-design features, make user guidelines easily accessible, provide tools such as blocking and reporting mechanisms, and publish Digital Safety Plans.

The bill would prohibit children under the age of 16 from holding social media accounts. Social media services could seek an exemption if they demonstrate that sufficient safeguards for children are in place.

The Safe Social Media Act is organised around three core duties: a Duty to Protect Children, a Duty to Act Responsibly and a Duty to Make Certain Content Inaccessible. Social media services would be required to assess and mitigate risks associated with seven categories of harmful content, including child sexual victimisation, content inducing a child to self-harm, cyberbullying, hatred, violence, terrorism or violent extremism, and intimate content shared without consent.

Regulated social media services would also be required to make certain content inaccessible to users in Canada, including content that sexually victimises a child or revictimises a survivor, and intimate content communicated without consent, including sexualised deepfakes. The government said these categories can cause substantial and lasting harm even when a single item is shared.

Under the proposed legislation, AI chatbot services would be subject to a tailored Duty to Act Responsibly. The proposed requirements include mitigating the risk that chatbots communicate harmful content, being transparent about reporting thresholds in crisis situations, and reducing the risk of harmful chatbot behaviour.

The legislation would establish an independent Digital Safety Commission of Canada responsible for enforcing the framework, assessing compliance, conducting audits and inspections, issuing compliance orders and imposing administrative monetary penalties. The Commission would also handle certain complaints, develop guidance and support research on online safety best practices.

Why does it matter?

The Safe Social Media Act reflects a growing international shift towards preventative online safety regulation. Rather than focusing solely on the removal of illegal content after it appears, the proposed framework would require platforms and AI services to assess risks proactively and implement measures designed to reduce harm before it occurs.

The inclusion of AI chatbot services is particularly notable, as governments worldwide are increasingly examining the safety implications of generative AI systems. If adopted, the legislation could position Canada among the first countries to apply a comprehensive online safety framework that combines platform accountability, child protection measures and AI-specific obligations under a single regulatory regime.

Would you like to learn more about AI, tech, and digital diplomacy? If so, ask our Diplo chatbot!

EU publishes the final Code for labelling AI-generated content

The European Commission has published the final Code of Practice on marking and labelling AI-generated content, offering practical guidance for providers and deployers preparing to comply with transparency obligations under the EU AI Act.

The code is voluntary, but the underlying transparency obligations in Article 50 of the AI Act will apply from 2 August 2026. The Commission said the code is intended to help organisations implement those obligations in a consistent, practical and proportionate way.

The framework covers two main areas. Providers of generative AI systems are guided on marking and detecting AI-generated or manipulated audio, image, video and text content, including through machine-readable solutions where technically feasible. Deployers are guided on labelling deepfakes and AI-generated or manipulated text published to inform the public on matters of public interest.

Under the AI Act, users must also be informed when they are interacting with interactive AI systems, such as chatbots. The transparency requirements are intended to help people recognise when content has been generated or altered by AI and to reduce the risk of deception and manipulation.

The Commission has also published a set of the EU icons that deployers may use to label certain AI-generated content. The code does not replace the AI Act or future Commission guidelines on Article 50, which are expected before the transparency obligations begin to apply.

The Commission and the AI Board will now assess the code’s adequacy. If assessed positively, providers and deployers who sign the code may use its measures to help demonstrate compliance with the AI Act’s transparency rules.

Why does it matter?

The code is an important step in turning the AI Act’s transparency provisions into operational practice. Labelling and machine-readable marking rules could shape how platforms, AI providers, media organisations and other deployers handle synthetic text, images, audio and video. The measures are especially relevant for public-interest information, where undisclosed AI-generated or manipulated content can affect trust, elections, journalism and public debate.

Would you like to learn more about AI, tech and digital diplomacy? If so, ask our Diplo chatbot!

Snapchat introduces friends-only content sharing for users under 16

Snapchat has begun rolling out new content-sharing protections for users aged 13 to 15, limiting the visibility of their Stories and Spotlight videos to mutually accepted friends.

Under the new experience, younger teens will have a dedicated profile where they can create, save and showcase content. Still, it will not be visible to one-sided followers or the wider Snapchat community. Snap said users in this age group will no longer be able to post Spotlight content that is visible to non-friend audiences.

The company said the change is intended to create a more private sharing environment for younger teenagers. Snapchat users under 16 will also no longer have engagement metrics such as favourite counts.

Snap said users aged 16 to 17 will have an optional introduction to public sharing, with additional safeguards, limited distribution and parental visibility. Users aged 18 and over will continue to have full access to public profiles and broader distribution tools.

The update forms part of Snapchat’s wider teen safety approach, which includes stricter default privacy settings, limits on unwanted contact, moderated public content and parental tools through Family Center.

Why does it matter?

The update reflects a broader shift towards age-appropriate design and privacy-by-default settings for younger users. By limiting public distribution for users aged 13 to 15, Snapchat is reducing minors’ exposure to unknown audiences and public engagement metrics. The change is relevant to ongoing regulatory debates on children’s online safety, platform design, algorithmic distribution and the mental health effects of public social media engagement.

Would you like to learn more about AI, tech and digital diplomacy? If so, ask our Diplo chatbot!

MIT study warns of AI reliance in news verification

A new MIT Media Lab study suggests that using AI to verify news can improve short-term accuracy but may not help users build lasting skills to detect misinformation.

The month-long study followed 67 participants as they assessed news headlines and image pairs. Participants were 21% more accurate at detecting false information when assisted by an AI chatbot during a session. However, their unassisted performance on new news items declined by 15 percentage points by the fourth week compared with before the study began.

Researchers linked the finding to the ‘AI dependency paradox’, in which tools that improve immediate performance can also encourage users to rely on automated guidance rather than develop their own judgement. The study found that some participants shifted from active analysis to passive acceptance of AI suggestions, even as some believed their own abilities were improving.

The researchers said the way AI systems interact with users matters. Tools that ask guided questions and encourage reasoning appear more likely to support long-term learning than systems that simply provide direct answers.

The findings point to the need for stronger AI literacy as chatbots become more common in news consumption, education and information verification. Researchers also noted limitations, including the small set of validated news items and a participant pool focused on the United States and the United Kingdom.

Why does it matter?

AI is increasingly becoming part of how people search for, verify and consume news. The study suggests that using AI as a shortcut for fact-checking may reduce users’ ability to evaluate information independently, while better-designed systems could support learning and critical reasoning. That distinction matters for educators, platforms and policymakers working on misinformation, media literacy and responsible AI use.

Would you like to learn more about AI, tech, and digital diplomacy? If so, ask our chatbot!

New York moves to curb undisclosed news scraping by AI bots

New York lawmakers have passed legislation aimed at restricting ‘stealth crawlers’, automated bots that access and scrape content from news websites without identifying themselves. If signed by Governor Kathy Hochul, New York would become the first US state to impose such transparency requirements.

The bill would require companies operating such bots to identify themselves when accessing the websites of news organisations. It would also prohibit activity that damages, impairs or places undue burdens on news websites, or otherwise causes economic harm to publishers.

Supporters, including the New York State Broadcasters Association and the New York News Publishers Association, argue that undisclosed scraping allows technology companies to use journalistic content for AI and other automated services while reducing traffic and revenue opportunities for publishers.

The legislation would authorise the New York Attorney General’s office to take enforcement action against non-compliant companies, with civil penalties of up to $15,000 per day for violations. The measure was passed by lawmakers in New York and now awaits the governor’s decision.

Why does it matter?

The legislation reflects growing tensions between news publishers and technology companies over the use of online content for AI training, search services and other automated applications. Publishers increasingly argue that large-scale content scraping can generate commercial value for technology firms while undermining the business models that support journalism.

If enacted, the measure could establish one of the first state-level transparency frameworks governing automated content collection in the United States. It may also influence broader debates about AI training data, web scraping practices, publisher rights and the relationship between technology platforms and news organisations.

Would you like to learn more about AI, tech and digital diplomacy? If so, ask our Diplo chatbot!