A recent report by the Center for Countering Digital Hate (CCDH) has revealed that Instagram failed to remove abusive comments directed at female politicians who may run in the 2024 US elections. The study examined over half a million comments on posts by prominent female figures from the Democratic and Republican parties, including Vice President Kamala Harris and Senator Marsha Blackburn.
Over 20,000 comments were flagged as ‘toxic,’ with a significant number containing sexist, racist abuse and even death and rape threats. Despite violating Instagram’s community standards, 93% of the harmful comments remained on the platform.
Meta, the parent company of Instagram, highlighted the tools available to users to filter out offensive content but acknowledged the need to review the CCDH report and promised to act on any content that breaches their policies. The report further emphasised that women of colour were particularly vulnerable to online abuse during the 2020 election and criticised social media algorithms for amplifying harmful content. Advocacy groups are increasingly calling on social media platforms to better enforce their safety guidelines to protect users from targeted abuse.
OnlyFans, a platform known for offering subscribers ‘authentic relationships’ with content creators, faces scrutiny over the use of AI chatbots impersonating performers. Some management agencies employ AI software to sext with subscribers, bypassing the need for human interaction. NEO Agency, for example, uses a chatbot called FlirtFlow to create what it claims are ‘genuine and meaningful’ connections, although OnlyFans’ terms of service prohibit such use of AI.
Despite these rules, chatbots are prevalent. NEO Agency manages about 70 creators, with half using FlirtFlow. The AI engages subscribers in small talk to gather personal information, aiming to extract more money. While effective for high-traffic accounts, human chatters are still preferred for more personalised interactions, especially in niche erotic categories.
Similarly, Australian company Botly offers software that generates responses for OnlyFans messages, which a human can then send. Botly claims its technology is used in over 100,000 chats per month. Such practices raise concerns about transparency and authenticity on platforms that promise direct interactions with creators.
The issue coincides with broader discussions on online safety. The UK recently amended its Online Safety Bill to combat deepfakes and revenge porn, highlighting the rising threat of deceptive digital practices. Meanwhile, other platforms like X (formerly Twitter) have officially allowed adult content, increasing the complexity of managing online safety and authenticity.
In a groundbreaking event, Kenza Layli, an AI-generated Moroccan influencer, has been crowned the first Miss AI. Layli, created by Myriam Bessa of the Phoenix AI agency, aims to bring diversity and inclusivity to the AI creator landscape. With nearly 200,000 Instagram followers and 45,000 on TikTok, Layli is entirely AI-generated, from her images to her captions and acceptance speech.
The inaugural Miss AI contest, organized by the influencer platform Fanvue, attracted entries from 1,500 AI programmers worldwide. Layli’s creator, Myriam Bessa, will receive $5,000, support on Fanvue, and a publicist to elevate Layli’s profile. Runners-up included AI contestants Lalina Valina from France and Olivia C from Portugal.
La modelo e influencer virtual marroquí Kenza Layli es la inteligencia artificial que obtuvo el primer lugar en el concurso Miss Universo AI 2024. pic.twitter.com/JdhyNSKy3R
Unlike earlier virtual influencers, these contestants were created solely using AI programs such as DALL·E 3, Midjourney, and Stable Diffusion, with their speeches and posts generated by ChatGPT. Layli’s Instagram page features her fondness for the color red, motivational advice, and support for her national sports team.
Judges, including AI influencer Aitana Lopez and human pageantry historian Sally-Ann Fawcett, assessed contestants on looks, AI tool usage, and social media influence. Despite Layli’s unique representation, experts warn that AI beauty pageants may further homogenize beauty standards, reflecting existing biases in society.
As deepfake pornography becomes an increasing threat to women online, both international and domestic lawmakers face difficulties in creating effective protections for victims. The issue has gained prominence through cases like that of Amy Smith, a student in Paris who was targeted with manipulated nude images and harassed by an anonymous perpetrator. Despite reporting the crime to multiple authorities, Smith found little support due to the complexities of tracking faceless offenders across borders.
Recent data shows that deepfake pornography is predominantly used for malicious purposes, with 98% of such videos being explicit. The FBI has identified a rise in “sextortion schemes,” where altered images are used for blackmail. Public awareness of these crimes is often heightened by high-profile cases, but many victims are not celebrities and face immense challenges in seeking justice.
Efforts are underway to address these issues through new legislation. In the US, proposed bills aim to hold perpetrators accountable and require prompt removal of deepfake content from the internet. Additionally, President Biden’s recent executive order seeks to develop technology for detecting and tracking deepfake images. In Europe, the AI Act introduces regulations for AI systems but faces criticism for its limited scope. While these measures represent progress, experts caution that they may not fully prevent future misuse of deepfake technology.
A new UN report, released by the UNESCO latest Global Education Monitor (GEM), explores how technology affects girls’ education from a gender perspective.
The report celebrates two decades of reduced discrimination against girls but also notes technology’s negative effects on their educational outcomes. It addresses challenges such as online harassment, access disparities in ICT, and the harmful influences of social media on mental health and body image, which can impede academic performance. Additionally, the report sheds light on the gender gap in STEM fields, underscoring the underrepresentation of women in STEM education and careers.
While highlighting that appropriately used social media can enhance girls’ awareness and knowledge of social issues, the GEM team also calls for increased educational investment and stricter digital regulations to promote safer, more inclusive environments for girls worldwide.
Why does it matter?
The report coincided with the International Girls in ICT Day, supported by the ITU, during which the UN Secretary-Generalemphasised the need for greater support and resources for girls in Information and Communication Technology (ICT), noting that globally, women (65%) have less access to the internet compared to men (70%). The persistent access gap in ICT and its disproportionately adverse effects on girls, despite years of acknowledgement, suggests a need for a more aggressive approach in policy and resource allocation to truly level the playing field.
A recent report by GLAAD sheds light on the pervasive presence of anti-trans content across Meta’s platforms, including Facebook, Instagram, and Threads. According to GLAAD, Meta has failed to adequately moderate extreme anti-trans hate, as evidenced by the continued presence of posts promoting harmful practices like conversion therapy, mass killing of trans people, and derogatory slurs. This content often leads to mass harassment, with victims subjected to online abuse and real-life violence, as explained by GLAAD President and CEO Sarah Kate Ellis in a press release.
Moreover, GLAAD’s report highlights that much of the anti-trans content violates multiple community standards, particularly Hate Speech, on Meta’s platforms and often originates from high-follower accounts profiting from spreading hateful anti-LGBTQ narratives. Fear-mongering posts containing lies, conspiracy theories, and violent rhetoric drive engagement and revenue for both account owners and Meta, raising ethical concerns about the platform’s role in amplifying hate speech.
Furthermore, GLAAD’s report raises questions about the effectiveness of current content moderation practices and the accountability of social media platforms in combating hate speech. Despite calls from LGBTQ celebrities, public figures, and allies for greater protection against anti-trans hate in an open letter in June 2023, Meta’s platforms continue to serve as breeding grounds for harmful content, highlighting systemic failures in addressing online extremism.
Why does it matter?
GLAAD’s findings underscore the urgent need for social media companies like Meta to take concrete action against online hate speech targeting marginalised communities – particularly when they are scaling down their misinformation and hate speech moderation teams. The prevalence of anti-trans content not only violates Meta’s community standards but also perpetuates harmful stereotypes and contributes to real-world harm, including violence against transgender individuals.
As part of the process towards developing a Global Digital Compact (GDC), the UN Secretary-General has issued a policy brief outlining areas in which ‘the need for multistakeholder digital cooperation is urgent’: closing the digital divide and advancing sustainable development goals (SDGs), making the online space open and safe for everyone, and governing artificial intelligence (AI) for humanity.
The policy brief also suggests objectives and actions to advance such cooperation and ‘safeguard and advance our digital future’. These are structured around the following topics:
Digital connectivity and capacity building. The overarching objectives here are to close the digital divide and empower people to participate fully in the digital economy. Proposed actions range from common targets for universal and meaningful connectivity to putting in place or strengthening public education for digital literacy.
Digital cooperation to accelerate progress on the SDGs. Objectives include making targeted investments in digital public infrastructure and services, making data representative, interoperable, and accessible, and developing globally harmonised digital sustainability standards. Among the proposed actions are the development of definitions of safe, inclusive, and sustainable digital public infrastructures, fostering open and accessible data ecosystems, and developing a common blueprint on digital transformation (something the UN would do).
Upholding human rights. Putting human rights at the centre of the digital future, ending the gender digital divide, and protecting workers are the outlined objectives in this area. One key proposed action is the establishment of a digital human rights advisory mechanism, facilitated by the Office of the UN High Commissioner for Human Rights, to provide guidance on human rights and technology issues.
An inclusive, open, secure, and shared internet. There are two objectives: safeguarding the free and shared nature of the internet, and reinforcing accountable multistakeholder governance. Some of the proposed actions include commitments from governments to avoid blanket internet shutdowns and refrain from actions disrupting critical infrastructures.
Digital trust and security. Objectives range from strengthening multistakeholder cooperation to elaborate norms, guidelines, and principles on the responsible use of digital technologies, to building capacity and expanding the global cybersecurity workforce. The proposed overarching action is for stakeholders to commit to developing common standards and industry codes of conduct to address harmful content on digital platforms.
Data protection and empowerment. Ensuring that data are governed for the benefit of all, empowering people to control their personal data, and developing interoperable standards for data quality as envisioned as key objectives. Among the proposed actions are an invitation for countries to consider adopting a declaration on data rights and seeking convergence on principles for data governance through a potential Global Data Compact.
Agile governance of AI and other emerging technologies. The proposed objectives relate to ensuring transparency, reliability, safety, and human control in the design and use of AI; putting transparency, fairness, and accountability at the core of AI governance; and combining existing norms, regulations, and standards into a framework for agile governance of AI. Actions envisioned range from establishing a high-level advisory body for AI to building regulatory capacity in the public sector.
Global digital commons. Objectives include ensuring inclusive digital cooperation, enabling regular and sustained exchanges across states, regions, and industry sectors, and developing and governing technologies in ways that enable sustainable development, empower people, and address harms.
The document further notes that ‘the success of a GDC will rest on its implementation’. This implementation would be done by different stakeholders at the national, regional, and sectoral level, and be supported by spaces such as the Internet Governance Forum and the World Summit on the Information Society Forum. One suggested way to support multistakeholder participation is through a trust fund that could sponsor a Digital Cooperation Fellowship Programme.
As a mechanism to follow up on the implementation of the GDC, the policy brief suggests that the Secretary-General could be tasked to convene an annual Digital Cooperation Forum (DCF). The mandate of the forum would also include, among other things, facilitating collaboration across digital multistakeholder frameworks and reducing duplication; promoting cross-border learning in digital governance; and identifying and promoting policy solutions to emerging digital challenges and governance gaps.
On 24–25 March 2023, the UN Women Regional Office for East and Southern Africa and IamtheCODE (a global movement dedicated to advancing science, technology, engineering, arts, mathematics and design education) held a two-day hackathon in Johannesburg, South Africa. The hackathon focused on equipping 102 high school girls from Gauteng province’s townships with the digital literacy skills they need to succeed in the future. The overall goal was to promote young girls and women as innovators who would progress gender equality and women’s and girls’ empowerment through digital literacy. The girls had to create innovative technological solutions to advance any of the 17 sustainable development goals.
In her appeal, UNESCO Director-General Audrey Azoulay urged people to observe the International Day of Women in Multilateralism by emphasising women’s contribution to fostering peace and ensuring sustainable development. The focus is on the battle against hate speech, with a special emphasis on the harassment of and violence against women in the digital environment, which is a commitment at the core of UNESCO’s raison d’être, she said. On International Day, UNESCO hosts a global discussion at its Paris headquarters to encourage effective responses to online gendered misinformation.
The agency’s ongoing efforts to develop guidelines for the oversight of digital platforms, ensuring that information is a ‘public good’ while upholding freedom of expression, will be guided by recommendations.
Visual threats intelligence company Sensity discovered an AI bot embedded in Telegram. The bot is taking users’ photos and sharing them as photo-realistic simulated nude images, without their knowledge or consent.
However, this bot seems to target women exclusively. Over 680,000 women are unaware that their photos have been uploaded to this bot on Telegram. 70% of these women reside in Russia or Eastern Europe. Also, 70% of these photos come from social media or private sources.
Unlike other algorithms for creating deepfake videos or photos, this bot requires only one photo to generate multiple fake nude images. Nina Jankowicz, author of How to Lose the Information War, said ‘This is all part and parcel of the broader abuse and harassment that women have to deal with in the online environment, whether that’s just trolling or whether it’s the gendered and sexualized abuse coming from all sides of the political spectrum.
It’s used as a weapon to try to push women out of the public sphere. However, this is just an extension of that.’ The media reports that all sensitive data discovered during the investigation has been disclosed to Telegram, [Russian social media site] VK, and relevant law enforcement authorities. There is no response from Telegram or VK yet.
Would you like to learn more about AI, tech and digital diplomacy? If so, ask our Diplo chatbot!
