The evolution of the EU consumer protection law: Adapting to new challenges in the digital era

What is EU consumer law?

The first mention of consumer law in the EU was in the context of competition law in 1972 when policymakers started to pave the way to protect consumers in policy. Despite the lack of a legal treaty basis, many regulatory initiatives started to take shape to protect consumers (food safety, prevention of doorstep selling, and unfair contract terms). 

The first treaty-based mention of a specific consumer protection article was in the 1992 Maastricht treaty. Nowadays, the EU consumer law is one of the most and better developed substantive fields of the EU law.

As contained in the Consolidated Version of the Treaty on the Functioning of the European Union (the treaty that regroups all previous European Union treaties before 2009), Article 169 specifically refers to consumer protection. Article 169(1) reads as follows:

‘In order to promote the interests of consumers and to ensure a high level of consumer protection, the Union shall contribute to protecting the health, safety and economic interests of consumers, as well as to promoting their right to information, education and to organise themselves in order to safeguard their interests.’

Given its history, it has long been established that consumer law purports to guarantee and protect the autonomy of the individual who appears in the market without any profit-market intentions.  Beyond the goals set out in Article 169 TFEU, four main directives govern areas of consumer law, the 1985 Product Liability Directive, the 1993 Unfair Terms in Consumer Contracts Directive, the 2011 Consumer Rights Directive, and the subject of this analysis, the 2005 Unfair Commercial Practices Directive.

Since then, there have been numerous amendments to the EU’s consumer protection legislative framework. The main amendment in consumer law includes the adoption of the Modernisation Directive.

EU flags in front of European Commission

Adopted on 27 November 2019, it amended four existing directives, the UCPD, the Price Indication Directive 98/6/EC, the Unfair Contract Term Directive 93/13/EEC, and the Consumer Rights Directive 2011/83/EU. Even more recently, there have been specific proposals for amendments to the UCPD concerning environmental advertising, known as greenwashing, in line with furthering the European Union’s Green Deal.

What is UCP?

An unfair commercial practice (UCP) is a misleading practice (whether deliberate actions or omissions of information), aggressive or prohibited by law (blacklisted in Annex I UCPD). A UCP interferes with consumers’ free choice to determine something for themselves and affects their decision-making power.

Prohibited UCPs are explained in Article 5 of the UCPD.  It outlines that a UCP will be prohibited if it is contrary to professional diligence and materially distorts the average consumer’s economic behaviour. The EU clearly outlines and recalls that there are two main categories of UCPs, with examples for both:

  • First, misleading practices through action (giving false information) or omission (leaving out important information).
  • Second, aggressive practices aimed at bullying consumers into buying a product.

Some examples of UCPs are bait advertising, non-transparent search results ranking, free claims about cures, false green claims or greenwashing, certain game ads, false offers, and persistent unwanted calls. There is no exhaustive list of what a UCP may be, especially in the digital context where technology is rapidly changing the way we behave towards one another.

This is especially evident in the case of the use of AI. AI is a buzzword that is often impossible to avoid nowadays. Computer Science Professor at Standford University, Dr Fei-Fei Li, said that ‘AI is everywhere. It’s not that big, scary thing in the future. AI is here with us.’ 

AI is used in UCPs to improve and streamline emotional, behavioural, and other types of targeting. Data can be collected using AI (scraping website reviews or analysing consumer trends), and this information can be leveraged against consumers to influence their decision-making powers, ultimately furthering the commercial goals of traders, potentially to the detriment of the interests of consumers.

EU consumer protection

When influencing a consumer’s decision-making powers, AI will often employ measures to deceive and manipulate users to get them to influence their decision-making, thus breaching the UCPD. However, these violations often go unnoticed since most people are unaware of UCPD or dark patterns.

Therefore, UCPs are practices that manipulate consumer choices in a certain way, and the advancement of AI widens the gap between consumers and their freedom to decide what they want without them even knowing it.

What is the UCPD?

As part of consumer law and as already stated, this analysis will focus on the UCPD and its recent amendments.

The origin of the UCPD

The UCPD was not the original legislation governing the protection of UCP in the EU. The first law relating to UCPs was adopted in 2005 and amended the 1984 Misleading and Comparative Advertising Directive. Its scope grew from amendment to amendment, and at its core, the directive has always been based on the prohibition of practices contrary to the requirements of professional diligence as contained in Article 2(h) UCPD:

Professional diligence ‘means the standard of special skill and care which a trader may reasonably be expected to exercise towards consumers, commensurate with honest market practice and/or the general principle of good faith in the trader’s field of activity’.

The UCPD was introduced to establish a fully harmonised legal framework for combatting unfair business-to-consumer practices across member states. This entailed introducing legislation harmonising different pre-existing laws to form a cohesive and understandable legal framework. This harmonisation not only combined existing legislation whilst introducing some key amendments but also provided legal certainty by having one centralised document to consult when dealing with unfair commercial practices in the EU.

One of the major drawbacks from a member state’s perspective is that the UCPD has a full harmonisation effect (meaning that member states cannot introduce more or less protection through national legislation efforts). It implied that member states could not introduce the measures they deemed to be necessary to protect consumers against UCP. Member states do have some discretion to implement UCP national legislation in certain sectors such as contract law, health and safety aspects of products, and legislation on regulated professions, but for the most part, they cannot introduce their own pieces of legislation concerning UCPs.

The goals and objectives of the UCPD are twofold. First, it aims to contribute to the internal market by removing obstacles to cross-border trade in the EU. Secondly, it seeks to ensure high consumer protection by shielding consumers from practices that distort their economic decisions and by prohibiting unfair and non-transparent practices.

The UCPD has a blacklist in Annex I with all the prohibitions it includes. A trader cannot employ any of the practices listed in Annex I, and if they do, they are in breach of the UCPD. There is no need to assess the practice, the potential economic distortion or the average consumer. If a trader engages in a practice listed in Annex I of the UCPD, that behaviour is strictly prohibited.

Past amendments to the UCPD

Before the UCPD was implemented, EU member states had their own national legislations and practices regarding consumer law and specifically, UCP. However, this could cause issues for traders trying to sell goods to consumers as they had to consult many legal texts.

By consolidating all of these rules, changing some and adding new ones, the EU could codify UCP in a single document. This helps promote fairness and legal certainty across the EU. The UCPD has been amended several times since it was first published in the Official Journal of the European Union.

These amendments have covered several changes to enhance consumer protection and include the following: marketing of dual-quality products, individual redress, fines for non-compliance, reduced full harmonisation effect of the directive, and information duties in the online context. In essence, these amendments aim to improve the state of consumer law and protect consumers in the EU. Below is a summary of these amendments in more detail.

Marketing of dual quality products: dual quality refers to the issue of some companies selling products in different member states under the same (or similar) branding and packaging but with different compositions. There is currently no explanation of any objective justifications for the marketing of dual-quality products to be allowed under the directive, as there is no explanation of any possible objective criteria.

The directive’s preamble (non-binding but still influenceable) refers to certain examples where the marketing of dual-quality products is permitted. This can be permitted by national legislation, availability or seasonality of raw materials, voluntary strategies to improve access to healthy and nutritious food, and offering goods of the same brand in packages of different weights or volumes in different geographical markets.

Individual redress: a key aspect of these amendments is setting up individual remedies for consumers that did not exist previously. This harmonises remedy efforts across the EU, as many member states did not have individual consumer remedies. Article 11(a) of the directive will propose minimum harmonising remedies, meaning that member states can introduce legislation to further consumer protection.

Fines: the amendments introduced penalties and fines changed compared to the previous UCPD. The new amendments set out criteria for imposing penalties. It is a long list in article 13(2) of the directive. In addition to these criteria, the new amendment proposed that 4% of the EU’s global annual turnover should be the maximum fine for widespread infringement.

Reduced full harmonisation: the amendments also introduced limits to the somewhat controversial full harmonisation of the UCPD. They limited the harmonisation in 2 cases. The first concerns commercial excursions known as ‘Kaffeabrten‘ in Germany. These are low-cost excursions for the elderly where UCP sales occur, such as deception and aggressive sales tactics.

The second concerns commercial practices involving unsolicited visits by a trader to a consumer’s home. If member states wish to introduce legislation to this effect, they must inform the European Commission, which has to inform traders (as part of the information obligation) on a separate, dedicated website.

Recent amendments to the UCPD

The UCPD is not an entrenched directive that cannot be amended. This is evident from its amendment in 2019 and the more recent 2024 amendments.  The new proposal introduces two amendments that would add to the existing list of practices considered misleading if they cause or are likely to cause the average consumer to make a transactional decision they would not otherwise make in the context of environmental matters.

  • The first amendment concerns environmental claims related to future environmental performances without clear, objective, and publicly available commitments.
  • The second amendment relates to irrelevant advertising benefits for consumers that do not derive from any feature of the product or service.

Additionally, new amendments to the ‘blacklist’ in Annex I have been proposed. A practice added to the blacklist entails it to be considered as unfair in all circumstances. These amendments relate to environmental matters associated with the European Green Deal and aim to reduce the effect of ‘greenwashing’. These amendments include:

  • Displaying a sustainability label that is not based on a certification scheme or not established by public authorities.
  • Making a generic environmental claim for which the trader is not able to demonstrate recognised excellent environmental performance relevant to the claim.
  • Making an environmental claim about the entire product or the trader’s business when it concerns only a certain aspect of the product or a specific activity.
  • Claiming, based on the offsetting of greenhouse gas emissions, that a product has a neutral, reduced or positive impact on the environment in terms of greenhouse gas emissions.

The focus of the new amendments is evidently to reduce environmental misconceptions that consumers may have about a product, as businesses greenwash products to mislead them into choosing them. This aims to protect consumers in the EU so that they can make an informed choice about whether a product contributes to environmental goals or not without being manipulated or misled into believing that it is because of the use of an environmental colour (green) or an ambiguous title (sustainable).

Final thoughts

The level of consumer law protection in the EU is ever-evolving, always aiming to reach higher and higher peaks. This is reflected in the EU’s efforts to amend and strengthen the legislation that protects us consumers.

Past amendments aim to clarify doubtful areas of consumer law, such as what information should be provided and where member states can legislate on UCPs, reducing the effect of full harmonisation. These amendments also introduced new and important notions such as redress mechanisms for individual consumers along with criteria for fines.

 Architecture, Building, Office Building, City, Urban, High Rise, Condo, Housing, Handrail, Metropolis, Apartment Building, Window

The more recent amendments target trader’s actions towards misleading greenwashing practices. Hopefully, these greenwashing amendments will help consumers make their own informed choices and help make the EU more sustainable by cracking down on the use of misleading, sustainable, and unfair commercial practices.

Given that amendments only took place in 2024, it is unlikely that there will be any new amendments to the UCPD any time soon. However, in the years to come, there are bound to be new proposals, potentially targeting the intersection of AI and unfair commercial practices.

BRICS alliance targets AI innovation and collaboration

Russia has unveiled plans to create an AI alliance with BRICS countries Brazil, China, India, and South Africa along with other interested nations. President Vladimir Putin made the announcement at a major AI conference in Moscow, highlighting the initiative as a key step to challenge the dominance of the United States in the rapidly advancing field of AI.

The AI Alliance Network will promote joint research, technology development, and regulation among member nations. Despite Western sanctions that have hampered Russia’s access to essential AI hardware like microchips, domestic leaders like Sberbank and Yandex are driving innovation with generative AI models such as GigaChat and YandexGPT.

Russia also has ambitious plans to integrate AI across its economy, targeting a contribution of 11.2 trillion roubles to GDP by 2030 and training 80% of its workforce in AI skills. While the country currently lags behind global leaders like the US and China in AI development, this alliance could mark a turning point in its technological aspirations.

Nevada adopts blockchain for election certification

Nevada has introduced blockchain technology into its electoral certification process to enhance transparency and security. Secretary of State Francisco Aguilar announced the move, emphasising that blockchain would make altering or counterfeiting certification documents significantly more difficult. The system aims to ensure election integrity, positioning Nevada as a leader in adopting emerging technology for secure elections.

The blockchain system will serve as an immutable ledger to record the certification process, providing a trustworthy and transparent official election record. While details about the implementation remain limited, it’s unclear whether Nevada has developed its blockchain or is relying on existing platforms. Certification by state and national archives is the next step, with Nevada joining states like Alaska and Georgia, which have piloted similar initiatives.

This development follows allegations of election misconduct during the 2020 presidential race. Although the 2023 indictments of six Republican certifiers were later dismissed, the state’s focus on bolstering election integrity highlights its commitment to avoiding future controversies.

Australian Federal Police leverage AI for investigations

The Australian Federal Police (AFP) is increasingly turning to AI to handle the vast amounts of data it encounters during investigations. With investigations involving up to 40 terabytes of data on average, AI has become essential in sifting through information from sources like seized phones, child exploitation referrals, and cyber incidents. Benjamin Lamont, AFP’s manager for technology strategy, emphasised the need for AI, given the overwhelming scale of data, stating that AI is crucial to help manage cases, including reviewing massive amounts of video footage and emails.

The AFP is also working on custom AI solutions, including tools for structuring large datasets and identifying potential criminal activity from old mobile phones. One such dataset is a staggering 10 petabytes, while individual phones can hold up to 1 terabyte of data. Lamont pointed out that AI plays a crucial role in making these files easier for officers to process, which would otherwise be an impossible task for human investigators alone. The AFP is also developing AI systems to detect deepfake images and protect officers from graphic content by summarising or modifying such material before it’s viewed.

While the AFP has faced criticism over its use of AI, particularly for using Clearview AI for facial recognition, Lamont acknowledged the need for continuous ethical oversight. The AFP has implemented a responsible technology committee to ensure AI use remains ethical, emphasising the importance of transparency and human oversight in AI-driven decisions.

AI tool helps detect lung cancer

Dianne Covey, a 69-year-old retired hospital worker from Farncombe, credits an AI tool with helping to save her life after it helped diagnose her lung cancer in a few hours. She visited her GP with a persistent cough, and her chest X-ray was analysed by Annalise.ai, a technology that flags abnormalities for urgent review. The swift diagnosis caught her cancer at Stage 1, offering a positive prognosis.

‘I never really understood artificial intelligence, but now I think it might have saved my life,’ said Ms. Covey. ‘The early diagnosis has given me a second chance at life.’ She is the first patient at the Royal Surrey NHS Foundation Trust to benefit from the AI system, which prioritises X-rays needing immediate attention and enhances accuracy by identifying tiny anomalies often missed in manual reviews.

The Annalise.ai tool is currently being used across five UK NHS trusts in Surrey, Sussex, and Frimley, enabling radiographers to streamline cancer diagnoses. By accelerating and refining the diagnostic process, this technology has the potential to revolutionise early detection, giving countless patients a fighting chance against life-threatening diseases.

Google accelerates renewable energy for AI

Google has announced a $20 billion partnership with Intersect Power and TPG Rise Climate to build renewable energy projects, battery storage, and grid upgrades for its data centres. The initiative includes wind, solar, and battery storage facilities, each paired with 1-gigawatt-scale data centres to meet growing energy demands for AI technology. The first phase is expected to be operational by 2026.

The plan aims to address grid bottlenecks, with Google funding required upgrades to accelerate connectivity. This strategy highlights renewables’ speed over nuclear options, which have longer timelines for implementation.

Industry experts predict a shortfall in energy for AI-focused data centres by 2027, underscoring the urgency for alternative power sources. While Google also invests in nuclear energy projects, renewables are expected to dominate in the near term.

Luke Littler becomes UK’s top trending athlete

Luke Littler, a 17-year-old darts sensation, has made history as the youngest player to reach the World Darts Championship final and later became Google’s most-searched athlete in the UK for 2024. Dubbed “The Nuke,” Littler’s breakout year began with a record-setting performance in January and culminated in major victories, including the Grand Slam of Darts and the Premier League Darts title.

His meteoric rise saw him ranked fourth globally and trending higher on Google than figures like the prime minister and the King. Littler’s nine-dart finish at the Bahrain Darts Masters and his string of high-profile wins captured global attention, drawing millions of viewers and sparking widespread online interest.

Reflecting on his remarkable success, Littler said, “It’s been an amasing year for me personally and for darts as a sport. Being recognised in Google’s Year in Search is a huge honor and shows how much the sport is growing.” His achievements highlight a banner year for young athletes breaking boundaries and captivating audiences worldwide.

Alphabet bets big on AI for search

Alphabet, the parent company of Google, is doubling down on AI to reshape its core search business, which generates the majority of its $300 billion annual revenue. At the Reuters NEXT conference in New York, Alphabet President Ruth Porat described AI as a ‘generational opportunity’ for the company, with tools like AI-generated query overviews aiming to make search more intuitive. However, challenges such as AI ‘hallucinations,’ where incorrect information is generated, remain a key hurdle.

Beyond search, Alphabet is channeling its AI expertise into healthcare advancements. Porat highlighted innovations like AlphaFold, which predicts protein structures to aid drug discovery, and AI tools that could prevent blindness or enhance the doctor-patient relationship by reducing screen time for medical professionals. These efforts reflect the company’s broader commitment to applying technology for societal benefits.

Alphabet’s financial investments in AI are substantial, with $50 billion projected in capital expenditures for 2024, including data centres and chips. Porat emphasised the need for these investments to yield tangible returns while shaping the future of both technology and human connection.

Bank of England explores privacy tech for digital pound

The Bank of England is exploring how emerging privacy technologies, such as zero-knowledge proofs (ZK-proofs), could enhance data privacy in a potential digital pound. In its report ‘Enhancing the Privacy of a Digital Pound,’ the bank suggests these technologies may limit data sharing, giving users greater control over their information while maintaining privacy between the central bank and payment intermediaries.

Following its 2023 public consultation, the Bank of England, alongside HM Treasury, assured the public that personal data would remain inaccessible to both the government and the central bank. Collaborating with MIT’s Digital Currency Initiative, the bank continues to research privacy-enhancing technologies while acknowledging the challenges of balancing privacy with regulatory requirements.

The digital pound initiative began in 2020 and has since undergone detailed evaluations. While no decision has been made on launching the currency, the central bank emphasises the need to adapt to declining cash use and advancements in payment technologies.

Are AI safety institutes shaping the future of trustworthy AI?

Summary

As AI advances at an extraordinary pace, governments worldwide are implementing measures to manage associated opportunities and risks. Beyond traditional regulatory frameworks, strategies include substantial investments in research, global standard setting, and international collaboration. A key development has been the establishment of AI safety institutes (AISIs), which aim to evaluate and verify AI models before public deployment, among other functions.

In November 2023, the UK and the USA launched their AI Safety Institutes, setting an example for others. In the following months, Japan, Canada, and the European Union followed suit through its AI Office. This wave of developments was further reinforced at the AI Seoul Summit in May 2024, where the Republic of Korea and Singapore introduced their institutes. Meanwhile, Australia, France, and Kenya announced similar initiatives.

Except for the EU AI Office, all other AI safety institutes established so far need more regulatory authority. Their primary functions include conducting research, developing standards, and fostering international cooperation. While AISIs have the potential to make significant advancements, they are not without challenges. Critics highlight issues such as overlapping mandates with existing standard-making bodies like the International Organization for Standardization that may create inefficiencies and the risk of undue industry influence shaping their agendas. Others argue that the narrow focus on safety sidelines broader risks, such as ethical misuse, economic disruption, and societal inequality. Some also warn that this approach could stifle innovation and competitiveness, raising concerns about balancing safety with progress.

Introduction

The AI revolution, while built on decades-old technology, has taken much of the world by surprise, including policymakers. The EU legislators, for instance, have had to scramble to update their advanced legal drafts to account for the rise of generative AI tools like ChatGPT. The risks are considerable, ranging from AI-driven disinformation, autonomous systems causing ethical dilemmas, potential malfunctions, and loss of oversight to cybersecurity vulnerabilities. The World Economic Forum’s Global Cybersecurity Outlook 2024 reports that half of industry leaders in sectors such as finance and agriculture view generative AI as a major cybersecurity threat within two years. These concerns, coupled with fears of economic upheaval and threats to national security, make clear that swift and coordinated action is essential.

The European Union’s AI Act, for instance, classifies AI systems by risk and mandates transparency along with rigorous testing protocols (among other requirements). Other regions are drafting similar legislation, while some governments opt for voluntary commitments from industry leaders. These measures alone cannot address the full scope of challenges posed by AI. In response, some countries have created specialised AI Safety Institutes to fill critical gaps. These institutes are meant to provide oversight and also advance empirical research, develop safety standards, and foster international collaboration – key components for responding to the rapid evolution of AI technologies.

In May 2024, a significant advancement in global AI safety collaboration was achieved by establishing the International Network of AI Safety Institutes. This coalition brings together AI safety institutions from different regions, including Australia, Canada, the EU, France, Japan, Kenya, the Republic of Korea, Singapore, the UK, and the USA. 

In November 2024, the International Network of AI Safety Institutes convened for its inaugural meeting, marking an important step in global collaboration on AI safety. Discussions centred on advancing research, developing best practices for model testing, promoting global inclusion and knowledge-sharing, laying the foundation for future initiatives ahead of the AI Action Summit in Paris in February 2025.

The first wave of AI safety institutes, established primarily by developed nations, has centred on safeguarding national security and reinforcing democratic values. As other countries establish their institutes, whether they will replicate these models or pursue alternative frameworks more attuned to local needs and contexts remains unclear. As in other digital policy areas, future initiatives from China and India could potentially serve as influential models. 

Furthermore, while there is widespread consensus on the importance of key concepts such as ‘AI ethics,’ ‘human oversight,’ and ‘responsible AI,’ their interpretation often varies significantly. These terms are frequently moulded to align with individual nations’ political and cultural priorities, resulting in diverse practical applications. This divergence will inevitably influence the collaboration between AI safety institutes as the global landscape grows increasingly varied.

Finally, a Trump presidency in the USA, with its expected emphasis on deregulation, a more detached US stance toward multilateral institutions, and heightened focus on national security and competitiveness, could further undermine the cooperation needed for these institutes to achieve meaningful impact on AI safety.

Overview of AI safety institutes

The UK AI Safety Institute

Established: In November 2023, with a mission to lead international efforts on AI safety governance and develop global standards. Backed by £100 million in funding through 2030, enabling comprehensive research and policy development.

Key initiatives:
– In November 2024, the UK and the US AI safety institutes jointly evaluated Anthropic’s updated Claude 3.5 Sonnet model, testing its biological, cyber, and software capabilities. The evaluation found that the model provided ‘answers that should have been prevented’ when tested on jailbreaks or actions that produce a response from a model that is intended to be restricted.

– Researched and created structured templates, such as the ‘inability’ template, to demonstrate AI systems’ safety within specific deployment contexts.

– Released tools like Inspect Evals to evaluate AI systems.
Offers up to £200,000 in grants for researchers advancing systemic AI safety.

– Partnered with institutes in the US and France to develop safety frameworks, share research insights, and foster talent exchange.

– Expanded globally with a San Francisco office and published major studies, such as the International Scientific Report on Advanced AI Safety.

The UK AI Safety Institute, launched in November 2023 with £100 million in funding through 2030, was created to spearhead global efforts in AI safety. Its mission centres on establishing robust international standards and advancing cutting-edge research. Key initiatives include risk assessments of advanced AI models (so-called ‘frontier models’) and fostering global collaboration to align safety practices. The institute’s flagship event, the Bletchley Park AI Safety Summit, highlighted the UK’s approach to tackling frontier AI risks, focusing on technical and empirical solutions. Frontier AI is being described as follows in the Bletchely declaration

‘Particular safety risks arise at the ‘frontier’ of AI, understood as being those highly capable general-purpose AI models, including foundation models, that could perform a wide variety of tasks – as well as relevant specific narrow AI that could exhibit capabilities that cause harm – which match or exceed the capabilities present in today’s most advanced models. Substantial risks may arise from potential intentional misuse or unintended control issues relating to alignment with human intent. These issues are in part because those capabilities are not fully understood and are, therefore, hard to predict. We are especially concerned by such risks in domains such as cybersecurity and biotechnology and where frontier AI systems may amplify risks such as disinformation.

However, this narrow emphasis has drawn criticism, questioning whether it sufficiently addresses AI’s broader, everyday challenges.

At the 2024 Stanford AI+Policy Symposium, Oliver Ilott, Director of the AI Safety Institute, articulated the UK’s vision for AI governance. He underscored that AI risks are highly context- and scenario-specific, arguing that no single institution could address all the challenges AI presents. ‘Creating such an entity would be like duplicating government itself,’ Ilott explained, advocating instead for a cross-governmental engagement where each sector addresses AI risks relevant to its domain. This approach highlights the UK’s deliberate choice to concentrate on ‘frontier harms’ – the most advanced and potentially existential AI threats – rather than adopting the broader, risk-based regulatory model championed by the EU.

The Bletchley Park AI Safety Summit reinforced this philosophy, with participating countries agreeing on the need for a ‘technical, empirical, and measurable’ understanding of AI risks. Ilott noted that the ‘core problem for governments is one of ignorance,’ cautioning that policymakers risk being perpetually surprised by rapid AI advancements. While high-profile summits elevate the political discourse, Ilott stressed that consistent technical work between these events is critical. To this end, the UK institute has prioritised building advanced testing capabilities and coordinating efforts across the government to ensure preparedness.

The UK’s approach diverges significantly from the EU’s more comprehensive, risk-based framework. The EU has implemented sweeping regulations addressing various AI applications, from facial recognition to general-purpose systems. In contrast, the UK’s more laissez-faire policy focuses narrowly on frontier technologies, promoting flexibility and innovation. The Safety Institute, with its targeted focus on addressing frontier risks, illustrates the UK’s approach. However, this narrow focus may leave gaps in governance, overlooking pressing issues like algorithmic bias, data privacy, and the societal impacts of AI already integrated into daily life.

Ultimately, the long-term success of the UK AI Safety Institute depends on the government’s ability to coordinate effectively across departments and to ensure that its focus does not come at the expense of broader societal safeguards. 

The US AI Safety Institute

Established: In 2023 under the National Institute of Standards and Technology, with a US$10 million budget, with a focus on empirical research, model testing, and safety guidelines.

Key initiatives:
– In November 2024, the US Artificial Intelligence Safety Institute at the US Department of Commerce’s National Institute of Standards and Technology announced the formation of the Testing Risks of AI for National Security Taskforce, which brings together partners from across the US government to identify, measure, and manage the emerging national security and public safety implications of rapidly evolving AI technology. 

– Conducted joint pre-deployment evaluations (Anthropic’s Claude 3.5 model).

– Launched the International Network of AI Safety Institutes to foster international collaboration, with an inaugural convening in San Francisco in November 2024.

– Issued guidance documents, requested input on chemical/biological AI risks, and formed a consortium with over 200 stakeholders to advance AI safety.
Signed agreements with entities like Anthropic and OpenAI to enhance research and evaluation efforts.

– Expanded leadership and outlined a strategic vision for global cooperation, aligning with the Biden administration’s AI Executive Order.

The US AI Safety Institute, established in 2023 under the National Institute of Standards and Technology with a US$10 million budget, is a critical component of the US’s approach to AI governance. Focused on empirical research, rigorous model testing, and developing comprehensive safety guidelines, the institute has sought to bolster national and global AI safety. Elizabeth Kelly, the institute’s director, explained at the 2024 AI+Policy Symposium, ‘AI safety is far from straightforward and filled with many open questions.’ She underscored the institute’s dual objective of addressing future harms while simultaneously mitigating present risks, emphasising that ‘safety drives innovation’ and that a robust safety framework can fuel healthy competition.

Kelly highlighted the collaborative nature of the US approach, which involves working closely with agencies like the Department of Energy to leverage specialised expertise, particularly in high-stakes areas such as nuclear safety. The institute’s priorities include fundamental research, advanced testing and evaluation, and developing standards for content authentication, like watermarking, to combat AI-generated misinformation. According to Kelly, the institute’s success hinges on building ‘an AI safety ecosystem larger than any single government,’ underscoring a vision for broad, cross-sectoral engagement.

The institute’s strategy emphasises a decentralised and adaptive model of governance. By leveraging the expertise of various federal agencies, the US approach aims to remain nimble and responsive to emerging risks. Similar to the UK approach, this model contrasts the European Union’s AI Office, where AI Safety is just one of the five specialised units supported by two advisory roles. The EU AI Office distinguishes itself from other AI Safety Institutes by adopting a centralised and hierarchical model with a strong focus on compliance and harmonisation across the EU member states. Being part of a centralised structure, the AI Safety unit may face delays in responding to rapidly emerging challenges due to its reliance on more rigid decision-making processes.

The US model’s flexibility supports innovation but may leave gaps in areas such as ethical governance and long-term accountability. The Institute operates under a presidential order, making its directives susceptible to shifts in political priorities. The election of Donald Trump for a new mandate introduces significant uncertainty into the institute’s future. Given Trump’s history of favouring deregulation, his administration could alter or dismantle the institute’s initiatives, reduce funding, or pivot away from stringent AI oversight. Such a shift could undermine progress in AI safety and lead to inconsistencies in governance, particularly if policies become more relaxed or innovation-focused at the expense of rigorous safety measures.

A repeal of Biden’s AI Executive Order appears likely, signalling shifts in AI policy priorities. Yet, Trump’s earlier AI executive orders emphasised civil liberties, privacy, and trustworthy AI alongside innovation, and it is possible that his future policy initiatives could maintain this balance.

Ultimately, the future of the US AI Safety Institute will depend on whether it can secure more permanent legislative backing to withstand political fluctuations. Elon Musk, a tech billionaire entrepreneur and a prominent supporter of Trump,  advocates extensively to shift the focus of the AI policy debate to existential AI risks, and these efforts might also impact the work of the US AI Safety Institute.

Japan’s AI Safety Institute

Established: In 2024, under the Council for Science, Technology, and Innovation, as part of the G7 Hiroshima AI Process.

Key initiatives:
– Conducts surveys, evaluates AI safety methods, and develops standards while acting as a central hub for collaboration between industry, academia, and AI safety-related organisations in Japan.

– Addresses a wide range of AI-related issues, including social impact, AI systems, data governance, and content, with flexibility to adapt to global trends.

– Focuses on creating safety assessment standards, exploring anti-disinformation tools, cybersecurity measures, and developing a testbed environment for AI evaluation.

– Engages in global collaboration with the AI safety institutes in the UK and USA to align efforts and share expertise.

The Japan AI Safety Institute plays a central role in the nation’s AI governance strategy, aligning its efforts with Japan’s broader commitments under the G7 Hiroshima AI Process. Operating under the Council for Science, Technology, and Innovation, the institute is dedicated to fostering a safe, secure, and trustworthy AI ecosystem.

Akiko Murakami, Executive Director of the institute, emphasised at the 2024 AI+Policy Symposium the need to ‘balance innovation and regulation,’ underscoring that AI safety requires both interagency efforts and robust international collaboration. Highlighting recent progress, she referenced the agreement on interoperable standards reached during the US-Japan Summit in April 2024, underscoring Japan’s commitment to global alignment in AI governance.

Murakami explained that the institute’s approach stands out in terms of integrating private sector expertise. Many members, including leadership figures, participate part-time while continuing their roles in the industry. This model promotes a continuous exchange of insights between policy and practice, ensuring that the institute remains attuned to real-world technological advancements. However, she acknowledged that the institute faces challenges in setting traditional key performance indicators due to the rapid pace of AI development, suggesting the need for ‘alternative metrics’ to assess success beyond conventional safety benchmarks.

The Japan AI Safety Institute’s model prioritises flexibility, real-world industry engagement, and collaboration. The institute benefits from up-to-date expertise and insights by incorporating part-time private sector professionals, making it uniquely adaptable. This hybrid structure differs significantly from the centralised model of the US AI Safety Institute, which relies on federal budgets and agency-specific mandates to drive empirical research and safety guidelines. Japan’s model is also distinct from the European Union’s AI Office, which, besides the AI Safety Unit, has broad enforcement responsibilities of the AI Act across all member states and from the UK’s primary focus on frontier risks.

Zooming out from the AI safety institutes and examining each jurisdiction’s broader AI governance systems reveals differences in approaches. The EU’s governance is defined by its top-down regulatory framework, exemplified by ex-ante regulatory frameworks such as the AI Act, which aims to enforce uniform risk-based oversight across member states. In contrast, Japan employs a participatory governance model integrating government, academia, and industry through voluntary guidelines such as the Social Principles of Human-Centric AI. This strategy fosters flexibility, with stakeholders contributing directly to policy developments through ongoing dialogues; however, the reliance on voluntary standards risks weaker enforcement and accountability. The USA takes an agency-driven, sector-specific approach, emphasising national security and economic competitiveness while leaving the broader AI impacts less regulated. The UK is closer to the US approach, with an enhanced focus on frontier risks addressed mostly through empirical research and technical safeguards. 

Japan’s emphasis on international collaboration and developing interoperable standards is a strategic choice. By actively participating in global efforts and agreements, Japan positions itself as a key player in shaping the international AI safety landscape. 

While the Hiroshima AI Process and partnerships like the one with the USA are central to Japan’s strategy, they also make its success contingent on stable international relations. If geopolitical tensions were to rise or if global cooperation were to wane, Japan’s AI governance efforts could face setbacks. 

Singapore’s AI Safety Institute 

Funding:  $50 million grant, starting from October 2022.

Key initiatives:
– Focuses on rigorous evaluation of AI systems, including generative AI, to address gaps in global AI safety science.

– Develops frameworks for the design, development, and deployment of safe and reliable AI models.

– Researches and implements methods to ensure the accuracy and reliability of AI-generated content.

– Provides science-based input for AI governance and contributes to international AI safety frameworks.

– Works with other AI safety institutes, including those in the USA and UK, to advance shared goals in AI safety and governance.

– Led the launch of the ASEAN Guide on AI Governance and Ethics to address regional AI safety needs cohesively and interoperably.

Unlike the US and the UK that established new institutions, Singapore repurposed an existing government body, the Digital Trust Centre. At the time of this writing, not enough information is publicly available to assess the work of the Centre. 

Canada’s AI Safety Institute

Established: November 2024, as part of Canada’s broader strategy to ensure the safe and responsible development of AI. Funding: C$50 million.

Key initiatives:
– CAISI operates under Innovation, Science and Economic Development Canada (ISED) and collaborates with the National Research Council of Canada (NRC) and the Canadian Institute for Advanced Research (CIFAR).

– It conducts applied and investigator-led research through CIFAR and government-directed projects to address AI safety risks.

– Plays a key role in the International Network of AI Safety Institutes, contributing to global efforts on AI safety and co-developing guidance for responsible AI practices.

– Supporting Canada’s Pan-Canadian Artificial Intelligence Strategy, the Artificial Intelligence and Data Act (Bill C-27), and voluntary codes of conduct for advanced AI systems.

As of this writing, more publicly available information is needed to evaluate the work of the Institute, which was only recently established.

European Union’s AI Office

Established: January 2024, the European Commission has launched an AI innovation package to support startups and SMEs in developing trustworthy AI that complies with EU values and rules. The AI office was part of this package. Funding: €46.5 million, setup funding.

Key Initiatives:
– Contributing to the coherent application of the AI Act across the member states, including the set-up of advisory bodies at EU level, facilitating support and information exchange.

– Developing tools, methodologies, and benchmarks for evaluating capabilities and reach of general-purpose AI models, and classifying models with systemic risks.

– Drawing up state-of-the-art codes of practice to detail out rules, in cooperation with leading AI developers, the scientific community and other experts

– Investigating possible infringements of rules, including evaluations to assess model capabilities, and requesting providers to take corrective action.

– Preparing guidance and guidelines, implementing and delegated acts, and other tools to support effective implementation of the AI Act and monitor compliance with the regulation.

The EU AI Office stands out as both an AI safety institute, through its AI Safety Unit, and a regulatory body with broad enforcement powers under the AI Act across EU member states. The AI Safety Unit fulfills the typical functions of a safety institute, conducting evaluations and representing the office internationally in meetings with its counterparts. It is not clear whether the AI Safety Unit  will have the necessary resources, both in terms of personnel and funding, to perform similar model testing as its UK and US counterparts. 

Republic of Korea’s AI Safety Institute

Established: November 2024, to ensure the safe use of artificial intelligence technology.

Key initiatives:
– Preemptively addresses risks like misuse, technical limitations, and loss of control to enhance AI reliability.

– Provides guidance to reduce AI side effects, such as deepfakes, and supports companies in navigating global regulations and certifications.

– Participates in international efforts to establish AI safety norms and align with global frameworks.

– Partners with 24 domestic organisations to strengthen AI safety research and create a secure R&D environment.

– Collaborates with companies like Naver, LG, and SK Telecom to promote ethical AI practices and manage potential risks.

As of this writing, insufficient publicly available information exists to evaluate the work of the Institute, which was only recently established.

Conclusion 

The AI safety institutes are beginning their journey, having only established their first basis for collaboration. While early testing efforts offer a glimpse of their potential, it remains to be seen whether these actions alone can effectively curb deploying AI models that pose significant risks. Diverging priorities, including national security concerns, data-sharing policies, and the further weakening of multilateral systems, could undermine their collective effectiveness.

Notably, nations such as India, Brazil, and China have yet to establish AI safety institutes. The governance models these countries propose may differ from existing approaches, setting the stage for a competition between differing visions of global AI safety. 

Building trust between the institutes and the AI industry will be critical for meaningful collaboration. This trust could be cultivated through transparent engagement and mutual accountability. Equally, civil society must play an active role in this ecosystem, acting as a watchdog to ensure accountability and safeguard the broader public interest.

Finally, the evolving geopolitical landscape will profoundly impact the trajectory of these initiatives. The success of the AI safety institutes will depend on their ability to adapt to technical and policy challenges and how effectively they navigate and influence the complex global dynamics shaping AI governance.