In a televised cabinet meeting, President Yoon condemned technology exploitation for such criminal acts, emphasising the dangers posed by anonymity in these cases. Although he did not specifically mention Telegram, the platform has been under scrutiny in South Korea for years due to its association with previous online sexual blackmail rings.
The Korea Communications Standards Commission (KCSC), South Korea’s media regulator, plans to meet to discuss measures to combat the growing issue of sexually explicit deepfakes. The commission has announced plans to increase efforts to block and delete such content and to double the number of personnel monitoring digital sex crimes. Additionally, a 24-hour hotline will be established for victims seeking assistance.
South Korean police have reported a sharp increase in deepfake sex crimes, with 297 cases recorded in the first seven months of this year, compared to 156 cases in 2021 when data collection began. The victims often include school students and female military personnel, highlighting the widespread impact of these crimes.
The KCSC is also seeking more cooperation from social media platforms like Telegram, X, Meta’s Facebook and Instagram, and Google’s YouTube to tackle the issue. The commission’s chairman, Ryu Hee-lim, stressed the severity of these crimes, which undermine individual dignity and personal rights.
An Australian transgender woman has won a significant legal battle against a female-only social networking app, Giggle for Girls, after being removed from the platform. The Federal Court ruled that the app’s decision to revoke Roxanne Tickle’s account amounted to indirect gender identity discrimination, awarding her A$10,000 in damages plus legal costs.
The court’s decision marks the first ruling on gender identity discrimination since the country amended the Sex Discrimination Act in 2013. The judge, Robert Bromwich, highlighted that Giggle for Girls only recognised sex assigned at birth as a valid basis for identifying as a man or woman. Tickle had undergone gender-affirming surgery and had her birth certificate updated.
Experts view the ruling as a victory for transgender rights in Australia, with Professor Paula Gerber from Monash University stating that the case sends a clear message against treating transgender women differently from cisgender women. The app, which was marketed as a safe space for women, had previously suspended operations but is expected to relaunch soon.
Tickle expressed relief at the verdict, calling it ‘healing’ after facing online abuse. Giggle for Girls’ founder, Sally Grover, acknowledged the judgement and affirmed that the fight for women’s rights would continue.
A recent report by the Center for Countering Digital Hate (CCDH) has revealed that Instagram failed to remove abusive comments directed at female politicians who may run in the 2024 US elections. The study examined over half a million comments on posts by prominent female figures from the Democratic and Republican parties, including Vice President Kamala Harris and Senator Marsha Blackburn.
Over 20,000 comments were flagged as ‘toxic,’ with a significant number containing sexist, racist abuse and even death and rape threats. Despite violating Instagram’s community standards, 93% of the harmful comments remained on the platform.
Meta, the parent company of Instagram, highlighted the tools available to users to filter out offensive content but acknowledged the need to review the CCDH report and promised to act on any content that breaches their policies. The report further emphasised that women of colour were particularly vulnerable to online abuse during the 2020 election and criticised social media algorithms for amplifying harmful content. Advocacy groups are increasingly calling on social media platforms to better enforce their safety guidelines to protect users from targeted abuse.
OnlyFans, a platform known for offering subscribers ‘authentic relationships’ with content creators, faces scrutiny over the use of AI chatbots impersonating performers. Some management agencies employ AI software to sext with subscribers, bypassing the need for human interaction. NEO Agency, for example, uses a chatbot called FlirtFlow to create what it claims are ‘genuine and meaningful’ connections, although OnlyFans’ terms of service prohibit such use of AI.
Despite these rules, chatbots are prevalent. NEO Agency manages about 70 creators, with half using FlirtFlow. The AI engages subscribers in small talk to gather personal information, aiming to extract more money. While effective for high-traffic accounts, human chatters are still preferred for more personalised interactions, especially in niche erotic categories.
Similarly, Australian company Botly offers software that generates responses for OnlyFans messages, which a human can then send. Botly claims its technology is used in over 100,000 chats per month. Such practices raise concerns about transparency and authenticity on platforms that promise direct interactions with creators.
The issue coincides with broader discussions on online safety. The UK recently amended its Online Safety Bill to combat deepfakes and revenge porn, highlighting the rising threat of deceptive digital practices. Meanwhile, other platforms like X (formerly Twitter) have officially allowed adult content, increasing the complexity of managing online safety and authenticity.
In a groundbreaking event, Kenza Layli, an AI-generated Moroccan influencer, has been crowned the first Miss AI. Layli, created by Myriam Bessa of the Phoenix AI agency, aims to bring diversity and inclusivity to the AI creator landscape. With nearly 200,000 Instagram followers and 45,000 on TikTok, Layli is entirely AI-generated, from her images to her captions and acceptance speech.
The inaugural Miss AI contest, organized by the influencer platform Fanvue, attracted entries from 1,500 AI programmers worldwide. Layli’s creator, Myriam Bessa, will receive $5,000, support on Fanvue, and a publicist to elevate Layli’s profile. Runners-up included AI contestants Lalina Valina from France and Olivia C from Portugal.
La modelo e influencer virtual marroquí Kenza Layli es la inteligencia artificial que obtuvo el primer lugar en el concurso Miss Universo AI 2024. pic.twitter.com/JdhyNSKy3R
Unlike earlier virtual influencers, these contestants were created solely using AI programs such as DALL·E 3, Midjourney, and Stable Diffusion, with their speeches and posts generated by ChatGPT. Layli’s Instagram page features her fondness for the color red, motivational advice, and support for her national sports team.
Judges, including AI influencer Aitana Lopez and human pageantry historian Sally-Ann Fawcett, assessed contestants on looks, AI tool usage, and social media influence. Despite Layli’s unique representation, experts warn that AI beauty pageants may further homogenize beauty standards, reflecting existing biases in society.
As deepfake pornography becomes an increasing threat to women online, both international and domestic lawmakers face difficulties in creating effective protections for victims. The issue has gained prominence through cases like that of Amy Smith, a student in Paris who was targeted with manipulated nude images and harassed by an anonymous perpetrator. Despite reporting the crime to multiple authorities, Smith found little support due to the complexities of tracking faceless offenders across borders.
Recent data shows that deepfake pornography is predominantly used for malicious purposes, with 98% of such videos being explicit. The FBI has identified a rise in “sextortion schemes,” where altered images are used for blackmail. Public awareness of these crimes is often heightened by high-profile cases, but many victims are not celebrities and face immense challenges in seeking justice.
Efforts are underway to address these issues through new legislation. In the US, proposed bills aim to hold perpetrators accountable and require prompt removal of deepfake content from the internet. Additionally, President Biden’s recent executive order seeks to develop technology for detecting and tracking deepfake images. In Europe, the AI Act introduces regulations for AI systems but faces criticism for its limited scope. While these measures represent progress, experts caution that they may not fully prevent future misuse of deepfake technology.
A new UN report, released by the UNESCO latest Global Education Monitor (GEM), explores how technology affects girls’ education from a gender perspective.
The report celebrates two decades of reduced discrimination against girls but also notes technology’s negative effects on their educational outcomes. It addresses challenges such as online harassment, access disparities in ICT, and the harmful influences of social media on mental health and body image, which can impede academic performance. Additionally, the report sheds light on the gender gap in STEM fields, underscoring the underrepresentation of women in STEM education and careers.
While highlighting that appropriately used social media can enhance girls’ awareness and knowledge of social issues, the GEM team also calls for increased educational investment and stricter digital regulations to promote safer, more inclusive environments for girls worldwide.
Why does it matter?
The report coincided with the International Girls in ICT Day, supported by the ITU, during which the UN Secretary-Generalemphasised the need for greater support and resources for girls in Information and Communication Technology (ICT), noting that globally, women (65%) have less access to the internet compared to men (70%). The persistent access gap in ICT and its disproportionately adverse effects on girls, despite years of acknowledgement, suggests a need for a more aggressive approach in policy and resource allocation to truly level the playing field.
A recent report by GLAAD sheds light on the pervasive presence of anti-trans content across Meta’s platforms, including Facebook, Instagram, and Threads. According to GLAAD, Meta has failed to adequately moderate extreme anti-trans hate, as evidenced by the continued presence of posts promoting harmful practices like conversion therapy, mass killing of trans people, and derogatory slurs. This content often leads to mass harassment, with victims subjected to online abuse and real-life violence, as explained by GLAAD President and CEO Sarah Kate Ellis in a press release.
Moreover, GLAAD’s report highlights that much of the anti-trans content violates multiple community standards, particularly Hate Speech, on Meta’s platforms and often originates from high-follower accounts profiting from spreading hateful anti-LGBTQ narratives. Fear-mongering posts containing lies, conspiracy theories, and violent rhetoric drive engagement and revenue for both account owners and Meta, raising ethical concerns about the platform’s role in amplifying hate speech.
Furthermore, GLAAD’s report raises questions about the effectiveness of current content moderation practices and the accountability of social media platforms in combating hate speech. Despite calls from LGBTQ celebrities, public figures, and allies for greater protection against anti-trans hate in an open letter in June 2023, Meta’s platforms continue to serve as breeding grounds for harmful content, highlighting systemic failures in addressing online extremism.
Why does it matter?
GLAAD’s findings underscore the urgent need for social media companies like Meta to take concrete action against online hate speech targeting marginalised communities – particularly when they are scaling down their misinformation and hate speech moderation teams. The prevalence of anti-trans content not only violates Meta’s community standards but also perpetuates harmful stereotypes and contributes to real-world harm, including violence against transgender individuals.
As part of the process towards developing a Global Digital Compact (GDC), the UN Secretary-General has issued a policy brief outlining areas in which ‘the need for multistakeholder digital cooperation is urgent’: closing the digital divide and advancing sustainable development goals (SDGs), making the online space open and safe for everyone, and governing artificial intelligence (AI) for humanity.
The policy brief also suggests objectives and actions to advance such cooperation and ‘safeguard and advance our digital future’. These are structured around the following topics:
Digital connectivity and capacity building. The overarching objectives here are to close the digital divide and empower people to participate fully in the digital economy. Proposed actions range from common targets for universal and meaningful connectivity to putting in place or strengthening public education for digital literacy.
Digital cooperation to accelerate progress on the SDGs. Objectives include making targeted investments in digital public infrastructure and services, making data representative, interoperable, and accessible, and developing globally harmonised digital sustainability standards. Among the proposed actions are the development of definitions of safe, inclusive, and sustainable digital public infrastructures, fostering open and accessible data ecosystems, and developing a common blueprint on digital transformation (something the UN would do).
Upholding human rights. Putting human rights at the centre of the digital future, ending the gender digital divide, and protecting workers are the outlined objectives in this area. One key proposed action is the establishment of a digital human rights advisory mechanism, facilitated by the Office of the UN High Commissioner for Human Rights, to provide guidance on human rights and technology issues.
An inclusive, open, secure, and shared internet. There are two objectives: safeguarding the free and shared nature of the internet, and reinforcing accountable multistakeholder governance. Some of the proposed actions include commitments from governments to avoid blanket internet shutdowns and refrain from actions disrupting critical infrastructures.
Digital trust and security. Objectives range from strengthening multistakeholder cooperation to elaborate norms, guidelines, and principles on the responsible use of digital technologies, to building capacity and expanding the global cybersecurity workforce. The proposed overarching action is for stakeholders to commit to developing common standards and industry codes of conduct to address harmful content on digital platforms.
Data protection and empowerment. Ensuring that data are governed for the benefit of all, empowering people to control their personal data, and developing interoperable standards for data quality as envisioned as key objectives. Among the proposed actions are an invitation for countries to consider adopting a declaration on data rights and seeking convergence on principles for data governance through a potential Global Data Compact.
Agile governance of AI and other emerging technologies. The proposed objectives relate to ensuring transparency, reliability, safety, and human control in the design and use of AI; putting transparency, fairness, and accountability at the core of AI governance; and combining existing norms, regulations, and standards into a framework for agile governance of AI. Actions envisioned range from establishing a high-level advisory body for AI to building regulatory capacity in the public sector.
Global digital commons. Objectives include ensuring inclusive digital cooperation, enabling regular and sustained exchanges across states, regions, and industry sectors, and developing and governing technologies in ways that enable sustainable development, empower people, and address harms.
The document further notes that ‘the success of a GDC will rest on its implementation’. This implementation would be done by different stakeholders at the national, regional, and sectoral level, and be supported by spaces such as the Internet Governance Forum and the World Summit on the Information Society Forum. One suggested way to support multistakeholder participation is through a trust fund that could sponsor a Digital Cooperation Fellowship Programme.
As a mechanism to follow up on the implementation of the GDC, the policy brief suggests that the Secretary-General could be tasked to convene an annual Digital Cooperation Forum (DCF). The mandate of the forum would also include, among other things, facilitating collaboration across digital multistakeholder frameworks and reducing duplication; promoting cross-border learning in digital governance; and identifying and promoting policy solutions to emerging digital challenges and governance gaps.
On 24–25 March 2023, the UN Women Regional Office for East and Southern Africa and IamtheCODE (a global movement dedicated to advancing science, technology, engineering, arts, mathematics and design education) held a two-day hackathon in Johannesburg, South Africa. The hackathon focused on equipping 102 high school girls from Gauteng province’s townships with the digital literacy skills they need to succeed in the future. The overall goal was to promote young girls and women as innovators who would progress gender equality and women’s and girls’ empowerment through digital literacy. The girls had to create innovative technological solutions to advance any of the 17 sustainable development goals.
