Content governance

2024 Predictions

In 2024, elections in over 70 countries, including India, the United States, the United Kingdom, the European Union, and Mexico, will rely heavily on online content. As election campaigns will be carried via online platforms, it will increase risks for the spread of mis/disinformation via deepfake videos, texts, and sounds.

Simultaneously, AI offers some hope for the detection of fraudulent content. Nonetheless, in 2024, the volume of deepfakes and identity manipulation generated by AI is likely to outnumber its detection capabilities.

Elections

In 2024, according to the Economist, 4.2 billion people in 76 countries will hold some sort of national election. Digital platforms and tools will play an important role in campaigns and elections. Their relevance increases given the high importance of these elections for the future of democracy itself.

AI empowers the generation of content, including fake and manipulative ones such as the following deepfake of Trump and Fauci posted by Ron DeSantis.

Link: https://www.youtube.com/watch?v=hLuUmNkS21A

Some platforms like TikTok, Discord, and Twitch are developing new tools to handle election disinformation, while others like X and Meta have rolled back their policies. However, experts have already expressed concerns about platforms not having sufficient resources for monitoring complexity of online content during elections.

In Michigan, USA, there is impending legislation to regulate AI in political advertising highlighting the need for transparency and accountability in how AI is leveraged during electoral processes.

Content Moderation

Major tech companies, such as Alphabet, X, Meta, and TikTok, will play an increasingly prominent role in content policy and moderation. They are becoming de facto content regulators, determining what content is allowed and what is removed from their platforms. In 2024, these companies are expected to continue adapting their content moderation policies to address the growing concerns around misinformation, fake news, and violent content.

Governance and regulation

The implementation of the EU’s Digital Service Act (DSA) will gain momentum in 2024. Analogous to the  ‘Brussels effect’ from data governance, DSA is likely to be emulated in other jurisdictions worldwide.

The internet has become an unregulated space where violent ideologies flourish unchecked. The US and EU’s joint statement on cyber resilience  reflects a concerted effort to address cyberterrorism and online radicalisation.

Because governments and tech companies lack adequate policies and technical tools for content governance, the arbitrary prohibition of certain content could cause significant social and political unrest. Tensions will spill over into the streets and squares from online spaces. Following the crisis of 2024, more stringent content governance policies will emerge.

Fragmentation of content spaces

The digital space is fragmenting with the development of smaller and segregated online communities of like-minded people. For example, when Trump was banned on Twitter he moved to Truth Social platform which gathers users with similar views.

This trend of fragmentation means a further disintegration of social spaces and ‘online squares’ with far-reaching consequences for social cohesion and political systems.

2023 Recap

In 2023, there is a growing consensus that something has to be done with content moderation. Apart from this shared concern, most other questions are open: by who, how, and where should content be regulated?

The underlying question is whether social media companies can develop satisfactory self-regulation policies in-house or if they will need to be forced to regulate content, as is happening in Europe with the Digital Service Act.

The US Congress supports amending Section 230 of the US Communications Decency Act (1996), which is the founding document of the social media industry and shields it from responsibility for hosted content.

The year 2023 was a critical one for online content moderation. Major tech companies, including Alphabet, Meta, and X, faced significant challenges due to layoffs, content moderation policy reversals, and new legislation coming into force.

Experts warned that the reversal of content policies at Alphabet, Meta, and X threatened democracy. Media watchdogs highlighted that the layoffs at these top social media firms created a ‘toxic environment’ as the 2024 elections await next year. The layoffs, numbering more than 40,000, were seen as a threat to the health and safety of these platforms. X (formerly Twitter) got onto the EU Commission’s radar for having significantly fewer content moderators than its rivals.

The European Union (EU) led the way in drafting internet and social media laws, including regulations around content moderation. The Digital Services Act (DSA), approved by the Council of the EU in July 2022, enabled Ireland to hold the online platforms accountable for their content. These regulations included the immediate takedown of illegal content, mandatory risk assessments of algorithms, and increased transparency in content moderation practices.

In August, the DSA began implementing strict online content measures on 19 very large online platforms and search engines. These measures ranged from the obligation to label all adverts and inform users who’s behind the ads, to allowing users to turn off personalised content recommendations. The DSA’s impact extended beyond the boundaries of the EU, affecting any company servicing European users, regardless of where it is based.

Major social media platforms were additionally warned about non-compliance in digital diplomacy when Thierry Breton flew to Silicon Valley in August to remind Big Tech CEOs about Brussels’ expectations. Previously, X, for instance, pulled out of the code to tackle disinformation, but Breton insisted on its compliance with the DSA when operating in the EU.

Besides Twitter turning into X, TikTok developments marked this year. In the US, several states introduced legislation to ban the app. Critics argued that banning TikTok may violate First Amendment rights and would set a dangerous precedent of curtailing the right to free expression online.

TikTok, seen as an extended arm of the Chinese government, faced broader backlash as Belgium, Norway, the Netherlands, the UK, France, New Zealand, Austria, and Australia issued guidelines against installing and using TikTok on government devices. In response, China ‘made solemn démarches’ to Australia over the Australian ban on TikTok on government devices.

Content moderation also hit a rather massive bump in late 2023 with the onset of global conflicts when the platforms drew a lot of criticism for failing to combat harmful content. As people grappled with the violence unfolding in Israel and Gaza, social media platforms became inundated with graphic images and videos of the conflict. This made it hard for anyone looking for information about the conflict to parse falsehood from truth.

In response to these challenges, tech companies took various measures. Meta established a special operations centre staffed with experts, including fluent Hebrew and Arabic speakers. TikTok established a command centre for its safety team, added moderators proficient in Arabic and Hebrew, and enhanced automated detection systems. X removed hundreds of Hamas-linked accounts and removed or flagged thousands of pieces of content.

Content moderation saw its day, or should we say days, in court as well. Meta, the parent company of Facebook and Instagram, confronted a legal battle initiated by over 30 US states. The lawsuit claimed that Meta intentionally and knowingly used addictive features while concealing the potential risks of social media use. And in the UK, the Online Safety Act came into effect, imposing new responsibilities on social media companies.

In an interesting development in global digital policy, UNESCO has been working on a set of Guidelines for regulating digital platforms – an international and soft law approach to content policy.

Finally, the ease of content generation with AI has marked a significant risk to the digital information space and especially the upcoming elections, contributing to the spread of fake news, deepfakes, and overall challenging the democratic electoral process. We also saw Meta outline a new policy on AI political advertising.

AI and content policy

Overall, by automating and simplifying online content moderation procedures, AI has the potential to improve the enforcement of content policies. However, difficulties and ethical concerns must be addressed, such as algorithmic biases that unintentionally result in the unjust targeting or exclusion of particular groups and issues like algorithmic transparency and accountability. In addition, the proliferation of AI-generated content adds considerably to the debate.

AI in content moderation

AI is widely employed to help identify and remove prohibited or harmful content more efficiently. AI-powered systems can process large volumes of content, reducing the burden on human moderators and improving response times. AI algorithms can increase the accuracy of content moderation by identifying potentially dangerous information by examining trends, context, and other criteria. Another area where AI is employed is content filtering, i.e. categorising content based on user preferences and community guidelines. For instance, Facebook relies, to a large extent, on AI in its content review process. AI machine learning models are used to detect and remove or reduce the visibility of content violating community standards even before anyone reports it. In other cases, AI may send content to human reviewers to double-check and decide on the content while the technology learns and improves from each decision. However, AI-powered content filtering raises many questions and ethical considerations. The limitations are often related to transparency, accuracy, and bias. AI-based content moderation often comes with a lack of transparency and the inability to explain how decisions are made. AI tools might not grasp the nuances and contextual variations present in human speech or be less accurate when analysing non-English or translated texts. Finally, AI may also reinforce existing biases, further marginalising and censoring at-risk groups. 

AI in content creation and dissemination

AI has been widely utilised to create different types of content, from emails to news articles and research papers. AI is also used in generating images and even composing music. For instance, natural language processing (NLP) models such as ChatGPT can suggest ideas, generate drafts, and converse with the user. AI-powered image and video editing tools can automate specific tasks, such as image enhancement or video editing, speeding up content creation. The ease with which one can generate any type of content raises concerns about the spread of easily generated misinformation. There are already several examples of the use of ChatGPT by journalists, leading to a major backlash from both people affected by the content and its broader readership. As AI becomes better at simulating reality, the problem of deepfakes and other AI-generated visual and audio content becomes all the more serious. Learn more on AI Governance.

  One of the main sociocultural issues is content policy, often addressed from the standpoints of human rights (freedom of expression and the right to communicate), government (content control), and technology (tools for content control). Discussions usually focus on three groups of content:

How is content policy conducted?

Governmental filtering of content

Governments that filter access to content usually create an Internet Index of websites blocked for citizen access. Technically speaking, this is done with the help of router-based IP blocking, proxy servers, and DNS redirection. Content filtering occurs in a growing number of countries (see opennet.net).

Private rating and filtering systems

Faced with the potential risk of the disintegration of the Internet through the development of various national barriers (filtering systems), W3C and other like-minded institutions made proactive moves proposing the implementation of user-controlled rating and filtering systems. In these systems, filtering mechanisms can be implemented by software on personal computers or at server level controlling Internet access.

This method allows users to implement their own filtering systems without national intervention. It remains to be seen, however, whether governments will sufficiently trust their citizens to create their own filters.

Content filtering based on geographical location

Another technical solution related to content is geo-location software, which filters access to particular web content according to the geographic or national origin of users. The Yahoo! case was important in this respect, since the group of experts involved, including Vint Cerf, indicated that in 70-90% of cases Yahoo! could determine whether sections of one of its websites hosting Nazi memorabilia were accessed from France.

This assessment helped the court come to a final decision, which requested Yahoo! to filter access from France to Nazi memorabilia. Since the 2000 Yahoo! case, the precision of geo-location has increased further through the development of highly sophisticated geo-location software.

Content control through search engines

The bridge between the end-user and Web content is usually a search engine, and filtering search results is therefore often used as a tool to prevent access to specific content. The risk of filtering of search results, however, doesn’t come only from the governmental sphere; commercial interests may interfere as well, more or less obviously or pervasively. Commentators have started to question the role of search engines (particularly Google, considering its dominant position in users’ preferences) in mediating user access to information and to warn about their power of influencing users’ knowledge and preferences.

This issue is increasingly attracting the attention of governments, which call for increased transparency from Internet companies regarding the algorithms they employ in their search engines. German chancellor Angela Merkel spoke out about this risk, claiming that: ‘Algorithms, when they are not transparent, can lead to a distortion of our perception, they can shrink our expanse of information.’

Automated content control

For Internet companies, it is often difficult to identify illegal content among the millions of content inputs on their platforms. One possible solution can be found in artificial intelligence mechanisms to detect hate speech, verbal abuse or online harassment. However, relying on machine learning to make decisions as to what constitutes hate speech opens many questions, such as whether such systems would be able to differentiate between hate speech and irony or sarcasm.

Content policy on social media

With the development of social media and Web 2.0 platforms – blogs, document‑sharing websites, forums, and virtual worlds – the difference between the user and the creator has blurred. Internet users can create large portions of web content, such as blog posts, videos, and photo galleries. Identifying, filtering, and labelling ‘improper’ websites is becoming a complex activity. While automatic filtering techniques for texts are well developed, automatic recognition, filtering, and labelling of visual content are still in the early development phase.

One approach, sometimes taken by governments in an attempt to manage user-generated content that they deem objectionable, is to completely block access to platforms such as YouTube and Twitter throughout the country, or even to cut Internet access completely, hindering all communication on social network platforms (as was the case, for example, during some of the Arab Spring events). However, this can seriously infringe on the right to free speech, and violates the potential of the Internet in other areas (e.g. as an educational resource).

As the debate of what can and cannot be published online is becoming increasingly mature, social media platforms themselves have started to formalise their policies of where they draw the border between content that should or should not be tolerated. For example, Facebook’s Statement of Rights and Responsibilities specifies:

‘We can remove any content or information you post on Facebook if we believe that it violates this statement or our policies.’

Yet, the implementation of such policies sometimes leads to unintended consequences, with platforms removing legitimate content.

Legal instruments and policy initiatives

The national legal framework

The legal vacuum in the field of content policy provides governments with high levels of discretion in deciding what content should be blocked. Since content policy is a sensitive issue for every society, the adoption of legal instruments is vital. National regulation in the field of content policy could bring a more predictable legal situation beneficial for the business sector, ensure a better protection of human rights for citizens, and reduce the level of discretion that governments currently enjoy.

However, as. the border between justified content control and censorship is delicate and difficult to enshrine in legislation, this tension is increasingly being resolved in the courtroom, for example regarding the role of social media outlets in terrorist activities.

International initiatives

In response to the increased sophistication with which terrorists manage their activities and promote their ideologies online, multilateral forums have started addressing ways to limit harmful content (e.g. the G7, the UN Security Council, and the United Nations Office on Drugs and Crime). At the regional level, the main initiatives have arisen in European countries with strong legislation in the field of hate speech, including anti-racism and anti-Semitism. European regional institutions have attempted to impose these rules on cyberspace.

The primary legal instrument addressing the issue of content is the Council of Europe Additional Protocol to the Convention on Cybercrime (2003), concerning the criminalisation of acts of racist and xenophobic nature committed through computer systems. On a more practical level, the EU adopted the European Strategy to Make the Internet a Better Place for Children in 2012.

The Organization for Security and Co-operation in Europe (OSCE) is also active in this field. Since 2003, it has organised a number of conferences and meetings with a particular focus on freedom of expression and the potential misuses of the Internet (e.g. racist, xenophobic, and anti-Semitic propaganda, and content related to violent extremism and radicalisation).

The role and responsibility of intermediaries

The private sector is playing an increasingly important role in content policy. Internet Service Providers, as Internet gateways, are often held responsible for the implementation of content filtering. In addition, Internet companies (such as Facebook, Google, and Twitter) are becoming de facto content regulators. Google, for example, has had to decide on more than half a million requests for the removal of links from search results, based on the right to be forgotten. These companies are also increasingly involved in cooperative efforts with public authorities in an attempt to combat illegal online content.

Archive of previous content

In 2023, most initiatives on content governance will try to find a balance between the legal status of social media platforms and their social roles. Legally speaking, these are private companies with very little legal responsibility for the content they publish. Societally speaking, these companies are public information utilities that impact people’s perception of society and politics. Twitter’s founder Jack Dorsey described Twitter as ‘the public conversation layer of the internet’.

Currently in the USA, tech platforms are not responsible for the content they host (as per Section 230 of the US Communication Decency Act). Although there are calls from both parties in the US Congress to revisit this arrangement, content governance in the USA is still in the hands of tech companies.

The most important thing to happen in the coming year will be how Musk’s policy experiment with Twitter turns out. If he is successful, he may show that a self-regulation model for content governance is possible. If he fails, it will be a sign that the US Congress has to step in with public regulation, most likely by revising Section 230.

In the EU, content governance has shifted towards public regulation. The Digital Service Act (DSA) introduced new, stricter rules that social media companies will have to follow. Their implementation will start in 2023.

Similar to GDPR and data regulation, many countries are likely to take inspiration from the EU’s DSA approach to content governance.

On a multilateral level, UNESCO will host the conference Internet for Trust: Regulating Digital Platforms for Information as a Public Good in February 2023 as the next step in developing content governance around its Guidance for regulating digital platforms: A multistakeholder approach.