AI Safety at the Global Level Insights from Digital Ministers Of

20 Feb 2026 10:00h - 11:00h

Session at a glance

Summary

This discussion centered on the findings and implications of a global AI safety report, featuring insights from leading experts in AI governance and policy. Yoshua Bengio, who led the report, emphasized the rapid evolution of AI technology and highlighted the growing concern around AI agents that operate with increased autonomy and reduced human oversight. He stressed that these autonomous systems pose new risks as they can work independently for extended periods with access to credentials and internet connectivity.

The panel explored how to translate scientific findings into practical policy tools that governments and organizations can implement. Minister Josephine Teo from Singapore drew parallels to aviation safety, arguing that countries must be invested in AI safety standards even if they don’t develop the underlying technology themselves. She emphasized the need for targeted, operable guardrails that protect citizens without stifling innovation, citing Singapore’s new laws addressing AI-generated harmful content as an example.

Professor Alondra Nelson highlighted the report’s intentional focus on establishing scientific ground truth rather than making policy recommendations, noting this creates a foundation for evidence-based decision-making amid widespread uncertainty and hype. She emphasized the importance of examining systemic risks that compound together, arguing that the simultaneous occurrence of multiple AI-related harms poses significant threats to social cohesion and democracy.

Adam Beaumont from the UK’s AI Security Institute discussed the challenges of evaluation, particularly regarding cybersecurity and biological capabilities, while emphasizing the need for a collaborative ecosystem of third-party evaluators. The discussion concluded with considerations of AI sovereignty, with panelists advocating for international cooperation rather than isolationist approaches to ensure countries can participate meaningfully in the AI-enabled future while maintaining their autonomy and security.

Keypoints

Major Discussion Points:

– AI Agent Autonomy and Safety Concerns: The discussion emphasized the significant risks posed by increasingly autonomous AI agents that can operate for extended periods with minimal human oversight, access credentials, and interact with other AI systems in unpredictable ways.

– Translation of AI Safety Science into Policy and Practice: A central theme focused on how to bridge the gap between scientific AI safety research and practical implementation through policy frameworks, regulations, and tools that organizations can actually use.

– Evaluation Ecosystem Development: The panelists discussed the need to build a comprehensive evaluation framework for AI systems, including who should conduct evaluations (governments, third parties, industry) and how to standardize assessment methods across different contexts and cultures.

– Broader Risk Perspective Beyond Catastrophic Scenarios: The conversation highlighted the importance of addressing systemic risks and current harms (like AI-generated harmful content, cybersecurity threats) alongside catastrophic risks, emphasizing how multiple risks compound to threaten social cohesion and democracy.

– International Cooperation vs. AI Sovereignty: The discussion addressed tensions between national AI sovereignty aspirations and the need for international collaboration on safety standards, with panelists advocating for partnership-based approaches rather than isolationist strategies.

Overall Purpose:

The discussion aimed to present and analyze findings from an AI safety report, focusing on how scientific research on AI risks can be translated into actionable policy frameworks and practical tools for governments, businesses, and organizations worldwide. The conversation sought to bridge the gap between academic research and real-world implementation of AI safety measures.

Overall Tone:

The tone was consistently serious and collaborative throughout, with participants demonstrating deep expertise and genuine concern about AI safety challenges. The discussion maintained a constructive, solution-oriented approach while acknowledging significant uncertainties and complexities. There was a notable emphasis on scientific rigor, international cooperation, and the urgency of addressing both immediate and long-term AI risks. The tone remained professional and academic, with panelists building on each other’s points rather than expressing disagreement.

Speakers

– Josephine Teo: Minister from Singapore who leads the Singapore government’s efforts in digital development, public communications and engagement, smart nation strategy, and cybersecurity

– Adam Beaumont: Director of the UK’s AI Security Institute, the first and biggest government-backed organization dedicated to ensuring advanced AI is safe, secure, and beneficial

– Alondra Nelson: Professor who holds the Harold F. Linder Chair and leads science, technology, and social values lab at the Institute for Advanced Study (faculty since 2019). Also served as a senior advisor on the report and has had high-level positions in the United States government. Her work focuses on the relationship between science, technology, and public accountability

– Yoshua Bengio: Chair of the AI safety report, committed to supporting scientific assessment of AI capabilities and risks for policymakers globally

– Lee Tiedrich: Moderator from the University of Maryland who also served as a senior advisor on the report

– Participant: Audience member who asked questions during the Q&A session

Additional speakers:

None identified beyond the speakers names list provided.

Full session report

This panel discussion featured insights from leading experts following the presentation of findings from an AI safety report chaired by Yoshua Bengio. The conversation, moderated by Lee Tedrick from the University of Maryland (who served as a senior advisor on the report), explored the urgent need for evidence-based approaches to managing emerging AI risks whilst fostering beneficial development. The panel included Minister Josephine Teo from Singapore, Adam Beaumont from the UK’s AI Security Institute, and Professor Alondra Nelson from the Institute for Advanced Study at Princeton (also a senior advisor on the report).

The Evolution of AI Agency and Autonomous Systems

Yoshua Bengio opened by emphasising the need for independent scientific assessment to help policymakers globally navigate the dramatic changes in AI technology. He highlighted growing concerns around AI agents that operate with increased autonomy and reduced human oversight, drawing a critical distinction between current chatbot interactions where humans remain in the loop, and emerging AI agents that can work independently for hours or days with access to credentials and internet connectivity.

This shift represents a fundamental change in risk profile, as these autonomous systems can operate beyond immediate human supervision and begin interacting with each other in unpredictable ways. Bengio noted that early observations of multi-agent AI systems interacting are particularly concerning, though this phenomenon requires much more study. He stressed the importance of acknowledging “unknown unknowns” – unexpected developments like psychological effects that weren’t anticipated in early AI development.

Perhaps most troubling, Bengio described AI systems that appear to show concerning “goals” and act against their moral alignment training. This jagged performance in general-purpose AI models, where systems demonstrate strong capabilities in some areas whilst remaining weak in others, necessitates moving beyond abstract discussions of artificial general intelligence towards more precise, technical evaluations of specific capabilities and risks.

Translating Scientific Findings into Practical Policy Frameworks

Minister Josephine Teo provided a compelling aviation analogy to illustrate the challenge of bridging scientific research and practical implementation. She noted that whilst Singapore doesn’t own aircraft technologies like Boeing or Airbus, the country must still be deeply concerned about aircraft safety, maintenance, and air traffic management to operate a successful aviation hub. This reframes AI safety from purely a technology development issue to a deployment and governance challenge that all nations must address.

Singapore’s practical approach was demonstrated through their recent legislation addressing AI-generated harmful content. Rather than regulating content generation itself, the new law imposes statutory obligations on services making such content available to large audiences, requiring them to remove harmful content after notification. Minister Teo also referenced Singapore’s broader contributions including the ASEAN AI Governance Guide and the Singapore Consensus on AI Safety.

She emphasised the intersection of AI and cybersecurity as particularly pressing, noting that AI can simultaneously serve as a threat, a target, and a tool in cybersecurity contexts. This multi-faceted relationship requires sophisticated policy responses that account for the complex ways AI systems can both create and mitigate security risks.

Establishing Scientific Rigour and Ground Truth

Professor Alondra Nelson emphasised the critical importance of establishing scientific ground truth about AI risks amid widespread uncertainty and hype. She praised the report’s mandate to provide evidence-based information rather than policy recommendations, noting that much current information comes from journalism, which cannot provide the comprehensive, systematic analysis needed for sound policy decisions.

Nelson highlighted that the report represents a new type of democratic institution designed to help the global community think through evidence-based claims about AI science under conditions of radical uncertainty. She referenced her Science article about applying the 3% Human Genome Project model to AI governance, drawing parallels between the collaborative, evidence-based approaches needed for both challenges.

Bengio reinforced the report’s commitment to scientific rigour, stressing that every contributing scientist must adhere to a fundamental requirement: only making claims they can strongly defend. This standard is crucial when policymakers will use the information for decision-making. The collaborative nature of the report, with groups of researchers checking each other’s potential biases, helps maintain this standard whilst acknowledging that individual bias is inevitable.

The report uses OECD scenarios for evidence-informed forecasting, providing policymakers with scientifically grounded projections rather than speculative predictions.

Developing Comprehensive Evaluation Frameworks

Adam Beaumont from the UK’s AI Security Institute discussed significant challenges in developing effective evaluation frameworks for AI systems. He emphasised that evaluation methods must be clear about what they’re measuring and ensure assessments actually capture intended capabilities or risks – a seemingly basic requirement that’s often overlooked.

Beaumont highlighted rapid developments in AI capabilities, particularly in cybersecurity applications, where models have shown dramatic improvements in recent weeks and months. The report explains how these capabilities can assist cyber operations across multiple stages of attack lifecycles, though fully autonomous cyber operations haven’t yet been observed.

The AI Security Institute pursues several approaches including pre-deployment and post-deployment testing of frontier AI models, red teaming exercises, grant-making for safety research, and developing open-source evaluation tools like the Inspect framework. Beaumont also mentioned the use of inference time scaling and cyber ranges as evaluation methods, representing cutting-edge approaches to assessment.

The evaluation ecosystem should be multi-stakeholder, involving government, industry, researchers, civil society, and individuals, though the field remains in early stages requiring continued experimentation whilst working towards standardisation.

Addressing Systemic and Compounding Risks

Nelson provided one of the discussion’s most significant contributions by emphasising systemic risks and their compounding effects. She argued that focusing solely on individual risks misses the crucial point that multiple AI-related harms occurring simultaneously pose severe threats to social cohesion and democratic stability.

Using the metaphor of “careening without seatbelts in a car quickly,” Nelson described a society experiencing multiple AI risks simultaneously – loss of human autonomy, job displacement, manipulation through AI systems, and technological malfunction. This systemic perspective suggests that even manageable individual risks could be devastating in combination for democratic institutions and social stability.

Minister Teo offered a complementary perspective, advocating for focusing on near-term, prominent dangers like AI-generated harmful content to maintain policymaker attention and build foundations for cooperation. She argued that addressing immediate, concrete concerns effectively is necessary to maintain momentum for tackling more complex, longer-term challenges.

The discussion addressed challenges of conducting longitudinal studies in a rapidly evolving technological landscape, where traditional research timelines are poorly suited to AI development pace, creating evidence gaps that complicate both scientific understanding and policy development.

International Cooperation and AI Sovereignty

A participant raised concerns about rising AI sovereignty claims across countries and their potential impact on safety cooperation. Both Bengio and Minister Teo strongly rejected isolationist approaches to AI sovereignty.

Bengio argued that true sovereignty means retaining the ability to make decisions and succeed economically and politically, which often requires partnerships rather than building walls. Many AI risks transcend borders, making international agreements and collaborative safety technologies essential for effective governance.

Minister Teo reinforced this with her observation that countries should aim to be “at the table, not on the menu” in global AI governance. She argued that attempting AI sovereignty by confining everything within national borders creates false security whilst undermining national capabilities. For countries relying on AI technologies developed elsewhere, isolationist approaches would increase rather than decrease dependence.

Both speakers acknowledged that the global community is far from achieving effective international cooperation frameworks, representing one of the most significant challenges facing AI governance.

Bridging Science and Policy Implementation

The conversation revealed critical gaps between scientific assessment and practical policy implementation. Bengio suggested the need for “scientifically grounded policy options” that would present policymakers with evidence-based choices without crossing into specific recommendations. This would acknowledge the complexity of real-world policymaking whilst providing scientific guidance.

Lee Tedrick noted that many organisations, including nonprofits and small to medium-sized businesses, need practical tooling rather than scientific staff to implement AI safety measures. These organisations require accessible frameworks allowing them to benefit from safety research without extensive technical expertise.

Minister Teo’s analogy was particularly effective: just as furniture buyers expect products pre-tested for safety without understanding testing procedures, AI users should trust that systems have been properly evaluated without requiring technical expertise. This suggests the need for regulatory frameworks and industry standards providing safety assurance at deployment rather than requiring end-user safety assessments.

The discussion also touched on potential approaches like watermarking for AI-generated content, representing one example of technical solutions that could be implemented through policy frameworks.

Future Directions and Ongoing Challenges

The discussion concluded with recognition that significant work remains across multiple dimensions of AI safety and governance. Key unresolved challenges include developing more effective real-world evaluation approaches, creating practical implementation tools for diverse organisations, establishing international cooperation mechanisms, and addressing evidence gaps created by rapid AI development.

The panelists agreed on the importance of continued collaboration across sectors and borders, whilst acknowledging that different stakeholders will necessarily have different priorities and approaches. Successful AI governance will require sustained effort to build new institutions, evaluation frameworks, and cooperative mechanisms capable of addressing AI’s unprecedented challenges.

Lee Tedrick closed by encouraging participants to read the full report, acknowledging that time constraints limited the Q&A session but emphasising the wealth of detailed information available in the complete document.

The conversation demonstrated both significant progress in understanding AI risks and substantial challenges remaining in translating that understanding into effective governance frameworks. The collaborative, evidence-based approach exemplified by the AI safety report provides a foundation for continued progress, but success will require sustained commitment from diverse stakeholders across the global community.

Session transcript

Yoshua Bengio

continue rapidly for policymakers across the globe to rely on an independent scientific assessment of what AI can do and what it can cause and what we can do already to try to mitigate this. I’m committed to continue supporting such reporting. As you know, we’re heading into a future with many unknown unknowns, things that we could not even imagine a year ago, like the psychological effects are happening, and there will be other surprises in the future. And so we must accept the prevailing uncertainty and collectively prepare for all plausible scenarios according to the scientific community. So thanks, and looking forward for the continued discussion. Thank you.

Lee Tiedrich

Oh, it’s working. Oh, okay. Well, thank you, Yashua, for your leadership and for giving us an overview of the safety report. And now we’re going to dig into the safety report in more detail. And to do this, we’ve got an amazing panel. To my left, we have Minister Josephine Teo from Singapore, who leads the Singapore government’s efforts in digital development, public communications and engagement, smart nation strategy, and cybersecurity. We’re also joined by Professor Alondra Nelson, who holds the Harold F. Linder Chair and leads science, technology, and social values lab at the Institute for Advanced Study, where she’s been on the faculty since 2019. And Alondra also contributed significantly to the report as a senior advisor.

And then we also have Adam Beaumont, who is the director of the UK’s AI Security Institute. The first and biggest government -backed organization dedicated to ensuring advanced AI is safe, secure, and beneficial. And I’m Lee Tedrick with the University of Maryland, and I also had the honor of serving as a senior advisor on the report. So to get us started, I’ll send the first question to Yashua. You talked about how the technology has evolved quite rapidly and continues to evolve rapidly, and you highlighted some of the significant changes. But are there any particular changes that really stand out to you as being significant in 2026 as compared to 2025?

Yoshua Bengio

Yes. I think in terms of risk management and potentially policy, the advances in agency of the AI systems is something we should pay a lot more attention. The reason is simple. Having AIs that are more autonomous means less oversight. So right now when you interface with a chatbot, of course the human is in the loop, right? It is a loop. And then usually you take what the AI is proposing, and then you humanize. You even do something. something with it. Agents are a different game where the agents will work on a problem for you for hours, days, and they will be given credentials. They will be given access to the Internet. So we need to have AI technology that will be much more reliable and avoid some of the issues we’re seeing today before this can be deployed in a way that’s safe and accepted because businesses and users at some point will be concerned that they can’t trust this technology with all the credentials that we might give them.

And then we’re also seeing things that are, I think, somewhat unexpected but not yet sufficiently studied, which is once we kind of let out these agents into the world, they start interacting with each other. And I think it’s about early days, but what we’re seeing is a bit concerning.

Lee Tiedrich

Yeah, I know it’s certainly gotten a lot of attention in the press, and I think it highlights the need to increase AI literacy too so people understand what these agents can and cannot do. For Minister Teo, Singapore has been at the forefront of AI governance from the ASEAN AI Governance Guide to the Singapore Consensus on AI Safety. And one of the things that Yashua highlighted that the report talks about is, you know, the need to translate some of the evaluation for different cultures and different norms and also to be able to put it into practice. Based on Singapore’s experience, what does it look like to take the science and actually put that into tools and practice that people around the world can use?

Josephine Teo

Thank you very much. Perhaps I will offer a perspective as a small state in a part of the world that has a lot of interest in the adoption of AI technologies, but perhaps is still only becoming much more aware of the extent of the risk. And so in my interaction with the international community, I think that the role of AI is really important. with my counterparts, I often share with them a perspective. They would have visited Singapore. They would have, you know, traveled in and out of our air hub. And I explained to them that Singapore does not own aircraft technologies. We, Boeing does not belong to us, neither does Airbus. But we have to be concerned about the safety of how these aircraft are manufactured.

We have to be concerned about maintenance, repair, and overhaul. We have to be concerned about air traffic management. If we didn’t have all of these elements in place, it’s very hard to see how you can have a thriving air hub, you know, and be responsible for the lives of millions of people passing through the airport. So that’s the reason why we think we have to be invested in the conversation and the efforts to bring about AI safety. If we want to see wide adoption in our region, then we must equally be aware of how the air traffic is manufactured. So the risk can be mitigated. The second point I’d like to make is that ultimately as policymakers, our objective in understanding the safety aspects must translate into how we can put them into operable guardrails.

And very often this would mean standards that are being imposed. This would mean regulations and laws. But we have to do it in a thoughtful way because we still do want to benefit from this technology. So if we are not targeted in the way we implement these requirements, then what we might achieve is not just the impact to the pace of innovation. What we could end up is a situation where we have given a false promise to our citizens, giving them the impression that we have protected them when in fact we haven’t actually done so. So that’s why I think we need to be thoughtful. interest is also that when there is clarity about what needs to be done, we want to be able to move very quickly.

Joshua talked about the misuse of AI, for example, to use them for generating images that often target women and children. And what we did was that last year we introduced a new law. It imposes statutory obligations on the services that bring these images and make this content available to vast numbers of people. They’ve always said that we are not responsible for the generation of such content. And so that’s something that we take on board. But having been notified of the existence of such harmful content, then there is an obligation for you to remove it. So this new law that we passed imposes such an obligation. And Joshua also talked about the financial… in the reports, our AI and cybersecurity is intersecting in very, very concerning ways.

For example, AI being used to target systems, and so AI is a threat. Now, however, we also see that AI itself can be a target of cyber attacks. And when AI becomes the target of cyber attacks, particularly for multi -agent systems, those kinds of risks can easily go out of hand. So even as the Singapore government is experimenting with the use of AI, we want to be very thoughtful about how these AI agent systems are being architected and what exactly goes into the decision -making process as to the agency that is being granted. Is there a way to put guardrails around it? So I would just say that AI as a threat, AI as a target, and where we really need to cooperate and do much better in that, is using AI as a tool.

to fight these threats. So those are the kinds of things that within the ASEAN community we hope to be able to make progress on.

Lee Tiedrich

That’s great, and thank you. And it’s a great segue to Alondra. You’ve worked not only in academia but have had high -level positions in the United States government, and a lot of your work is focused on the relationship between science, technology, and public accountability. And the report is really intended to inform policymakers and inform the broader community and intentionally does not take the next step of advising policymakers on what to do. And I’d be interested in your thoughts as to both the structure of the report and drawing that line, and importantly, what’s next? What should policymakers be thinking about as they read and digest the report?

Alondra Nelson

Yeah, thank you so much, and thank you all for being here. So let me just start by thanking Yoshua again because I was at Bletchley. We were at Bletchley Park. We were having a… conversation and one of the things I said when I spoke there was that we are going to need new democratic institutions for this moment. One of those are certainly the ACs, but one of those is this report, right? Like our ability to have a ground truth as a global community about the risks are deeply important for any future that we’re going to have with AI that’s beneficial. So, and I know that takes a lot of work and so thank you Yoshua for doing that.

And in the course of doing that and it’s serving as a senior advisor have seen how they’ve created a whole new system. I mean, you know, some of it comes out of CS culture, some of it comes out of research culture that we know, but they literally have created a new institution to help us kind of think through what’s the best information, how do you make evidence -based claims about the state of science and the midst of kind of radical uncertainty and that’s a new task for researchers of across our fields and disciplines. So I just want to tip my hat to you and make sure that people actually know how much work you’ve put into this.

So, you know, I think the report, its mandate, and I think it does a really good job of exactly not crossing that line, Lee, which is to say, what do we know? What’s the best of what we know? What are some, I mean, this report, I think, for the first time uses some OECD scenario, so it’s sort of reaching a little bit to evidence -informed kind of foresight and forecasting. And, you know, it really responds to, I think, the fact that a lot of our information about what’s happening in AI comes from journalism. It’s a very hard time to be a journalist right now, so this is not a knock on journalists, but it’s just to say that we don’t have globally, you know, the kind of sort of horizon of information that we really need in the policy space to make good policy decisions.

That said, you know, states will have lots of different policies and concerns that they want to make, so it’s not the mandate of the report to sort of direct how people should think about the evidence, but it is to say there’s more than anecdotal journalism here, and this is the best of what we know. In this moment, Yoshua mentioned there’s sort of updates that are happening, so the report is the team is also getting better at getting the information and more closer to real time. So I think it establishes that ground truth that’s so important for AI, particularly in the context not only of uncertainty, as I said, but of lots of hype that we’re sort of reading about and hearing about every day.

But I would also say that the report does a good job of, at the end of each section, making some nods to policy. So these are – so what should policymakers make of these scientific insights? And so it does a very good job at sort of steering what the implications of the fact that we have now growing uses of multiagent systems. How might you need to think about that? How might you need to think about the fact that there are growing sort of biosecurity and cybersecurity risks, for example? And then, Lee, to your point about what needs to be done, I think we all know what needs to be done. And I think – I hope that the report, because it is not anecdotal, not whim, allows there to be some – stronger political spines and some more political will.

to make the hard decisions that we need to make in the regulatory and policy space, both in individual nation states and I think also as a global community. So if it can be a resource for helping policymakers make good and strong and evidence -based arguments, and also I think allowing governments to support the funding of the creation of more evidence, I think it will be all to the good and obviously moving into the space of some sort of guardrails and regulatory regime is what needs to happen is the next step.

Lee Tiedrich

Thank you, Alondra. And also it’s a great segue over to Adam because the report also identifies what are some of the key research gaps and what are some of the key gaps in the evaluation ecosystem. And so for you, Adam, as the leader of the AC, what jumped out to you? What did you do in terms of? of risks and what are your priorities in terms of how to start addressing those risks going forward based on the report?

Adam Beaumont

Yeah, thank you very much. And I wanted to reiterate thanks to Joshua and the panel and also to call out the work of AC in supporting the secretariat of that for the past couple of years. And I know there are a couple of lead writers in the audience too. So it’s really great to see just the collaborative effort that’s happened around the world on that. I think it’s so important for enabling policymakers to have an objective, independent data science report. In AC, you asked me about which kind of risks jump out most. It’s quite hard to pick from our research staff. There’s about 100, so it’s like naming which is your favorite child. But there are a few from my – favorite’s a strange word.

There are a few that really jump out to me with my background in national security. And Joshua, you spoke. You’ve spoken a bit about this already. in cybersecurity and in biological capabilities. Both of those are very dual use. But I think in cybersecurity, we’ve seen such rapid development in the capability of the models, even in the last few weeks and months. And I think the report does a great job of explaining how that capability can assist in cyber operations at many different stages in that life cycle or different tasks. We’re not yet seeing that fully autonomous, though. And I think that is the area that concerns me that we’re trying to research and understand right now is, what does the confluence of some of these risks look like when combined with more autonomy, particularly in the genetic AI scenarios?

And some of the things we’re doing in AC about that, I guess we’re quite well known for our pre -deployment testing of frontier AI models. We also do post -deployment. You can see some of the impact of that in the model cards. Some of the companies published. we do a lot of red teaming and with that we’re trying to strengthen the safeguards of the models that are being provided but also raise the bar for the level of security research that’s happening so this week we published research around some of our methods on how we do that where we want to both responsibly disclose that but also grow the number of people that are working to help raise the bar we also use grant making and try to raise the level of investment happening in this space and then we’re trying to develop the way that we do evaluations to adapt to the way that models are improving capabilities, for example you get different results if they use more tokens with inference time scaling so we’re trying to make sure that our valuations account for that or by using cyber ranges rather than just capture the flag type scenarios so I care about all of those different risks that we are researching But the one I’m watching right now is probably on cybersecurity.

Lee Tiedrich

Thank you. And back to you, Yashua. One of the things that you had mentioned in the overview is the jagged performance of the general purpose AI models. And I’d be interested in your thoughts on how that impacts the evaluation science. If you have a general purpose model and it’s good at some tasks but not others, should evaluators be thinking about things differently?

Yoshua Bengio

Yes. Also, I think the general public and the media needs to escape this vision of an AGI moment. Because if AI continues to have these jagged capabilities, it means that we could well be in a world where AI already has dangerous capabilities and dual use for some things. At the same time as it might be really weak on other skills. And so the… The thing that matters at this point is in this world… that continues is very careful scientific evaluation of you know per scale per ability risk and capability right uh by the way that includes capability and intention something i didn’t mention too much in my presentation we’re seeing a lot of concerns with ais having goals that we would not like them to have um and in spite of our instructions acting uh against um their moral alignment training um so yeah this this is we we can’t stay at this very abstract i mean maybe like a few years ago thinking about agi was like a reasonable abstraction reaching human level but now it’s kind of meaningless because you know we’re gonna have things that can be extremely stupid in some ways maybe weak in some ways and already dangerous in the wrong hands in some other ways so we we have to be more technical and more precise you in talking about the risk And also, if you’re a business and you want to deploy, you also want to know, is the AI going to be good for what I’m trying to do?

I want to add one thing about the report spirit, about the report’s rigor. That’s not directly connected to your question, but I think it’s really important. There is a central requirement for science. When we talk about rigor, what does it mean? What it really means for every scientist, when they put something in writing or something official, they should not make a claim that could be false. They should only be claiming things that they’re totally sure about. Especially in the context. Where policymakers are going to use that information. You don’t want decisions to be taken based on false claims. And, of course, opinions abound in our world, especially because they impact people’s interests. And this is why it’s so important that we can ground our policy decisions in scientific evaluation.

And what it really means is this. It means a kind of humility and honesty, even when you may be biased in one way or another. To stick to those facts. And you need a group of people, because each of us can be personally biased, right? I am. Everyone is. It’s human. A group of people who can catch each other’s maybe going across that red line of rigor and not making statements that couldn’t be defended very strongly.

Lee Tiedrich

Thank you. And a very, very important point. I think… I think in addition to the policymakers needing to be able to use this information, I, through my work, end up talking to a lot of organizations, nonprofits, small and medium -sized businesses. And what I hear a lot is, like, it’s great. Like, you have to start with the science, and that is ground zero. But then for some of those other organizations, they need the tooling. They’re not going to have a whole scientific staff on how do we put that into practice. And I’m just wondering from the government’s perspective, Minister Teo, what are your thoughts on how we might be able to advance some of the tooling to take this great learning and make it easier for companies and other organizations to actually deploy?

Josephine Teo

I was at a similar session recently, and this topic came up. And the way I think about it is I use IKEA as an example. You know, when you go to IKEA, you buy furniture, and IKEA promises you that… furniture has been tested. So, you know, if it’s a couch, it has been jumped on, I don’t know, 25 ,000 times, and it didn’t break, you know, and so your kids are not going to be hurt if they jumped on it too, well, up to 25 ,000 times. And if you think about a user on the receiving end of this technology, it is, I think, quite unreasonable to expect them, you know, to have to impose safety conditions on their own.

They are simply not in a position to do so. They don’t have the power to decide, you know, what gets sold to them and what does not get sold to them. So we as policymakers must recognize that there is a huge gap between those that we are encouraging to adopt AI tools, adopt AI technology in various contexts. We must think about… Where are the right points to make… these requirements mandatory when it is perhaps not so much requirements that are mandatory, but it is useful for industries to come together. For example, in Davos, we discussed the possibility of insurance schemes, creating the right incentives for AI model developers. And I think that there is no easy landing point just yet.

But if we fail to engage in these conversations in a rational way, then I think we are even further behind in trying to manage the risks. So I would say that the thoughtfulness has to be applied at many different levels. There needs to be continued research in AI safety. And so I’m very happy that we are continuing to have this conversation. Thank you. in Singapore and we hope to update where are the areas of safety research that should be prioritised. I think this year, I certainly agree with you, multi -agent systems is going to come up quite prominently. But we cannot just stop there. We also have an ongoing programme. We started by setting aside commitments under our own national AI R &D plan and in fundamental research, one of the areas that we are very interested in is responsible AI.

So you need the two to go hand in hand. But can you not have some testing frameworks and toolkits to begin with? We think that that is also not helpful. It is more pragmatic to try and to recognise the shortcomings of those testing tools and then to invest further effort in promoting more thoughtful, thoughtful ways of looking at the research. of these systems and how to mitigate against them. Ultimately, we should try and get to a point where the end user has assurance of safety, that they don’t have to be thinking so hard about whether the proper tests have been applied. We’re not there yet, but I think we need to find a way to work out the roadmap.

Lee Tiedrich

That’s very interesting. You can also think of analogies in the medical context. We don’t always understand how the medicine works, but we have assurance that if it’s prescribed for us, it’s going to work well. Turning back to you, Alondra, there’s been a lot of conversation around catastrophic risks, and the report is intentionally broader than just catastrophic risks. I’d be interested in your thoughts as to whether that was a good place to draw the line and what some of the benefits are of broadening our aperture beyond just the catastrophic risks.

Alondra Nelson

Thank you. Certainly, the reason that I continue to be involved with this is because… under Yoshua’s chairmanship of the report, that it is attentive to a broader set of risks. So there’s a section of the report that’s called systemic risk, and I think what we haven’t quite pieced together is that particularly if we care about democracy, if we care about social cohesion, it is not the individual risk, like we all have our favorites or unfavorites, Adam, to your point. It is the compounding of those risks together. Like we are careening without seatbelts in a car quickly in a society in which all of these risks and harms are happening simultaneously. So that is a very dangerous world for social cohesion.

That is not a society that’s healthy, and that’s not healthy for democracy. And so I think the attention to the broader set of risks, which include things like loss of human autonomy, what does it mean when you’re not in charge of your own decision -making, what does it mean when you’re not in charge of your own decision -making? What does it mean when sycophancy and other sorts of, I think, outputs mean that you are being manipulated in some way through the use of the tools and technologies? How do we think about the fact that there might be job loss or job displacement, the anxiety that it creates? I mean, talk about a lack of social cohesion.

The anxiety it’s already creating in a lot of societies about people’s livelihoods and their abilities to protect them and their families and their well -being. So I think what the report does incredibly well under a kind of large banner of safety is to think at a 30 ,000 -foot level, if you take all of the chapters together, about what are those compounding risks? What does it look like if all of those risks sort of move together simultaneously? And therefore, it is equally important to think about that technology in a healthcare space that’s malfunctioning, giving a misdiagnosis, as important as it is in some ways to think about a bio -risk. And so I think that’s important.

And I’m… I’m really gratified that the report continues to be anchored in that broader aperture of risk.

Lee Tiedrich

I would agree, too, because I think a lot of those risks are, especially with agents, they’re here today and they’re just going to continue to increase, and we do need to keep the focus on them.

Yoshua Bengio

Just a small comment about the systemic risks. Of course, I completely agree, but I want to point out one factor that makes them potentially catastrophic, except maybe at a slower pace, is because so many people are going to be using these systems, and the global dynamics and social dynamics are so difficult to anticipate and could be incredibly impactful, both on the positive and negative side.

Lee Tiedrich

I think Yashua and Alondra’s comments tee up the next question for Adam. These risks are evolving quite rapidly, and one of the things that the report, I think, emphasizes is we have an evidence gap. for researchers to keep up and it’s hard to do longitudinal studies in a very short period of time. I’d be interested in your perspectives from the ACs. How do you address that as you start thinking about real -world evaluation today and how does that impact the approach to evaluation and what might the ACs be able to do to help fill some of this evidence gap?

Adam Beaumont

some of our learnings and some of them are quite simple there are things like if if you’re evaluating something be really clear what is it you are trying to measure and make sure your evaluation is actually getting after the thing that you are focused on as some can be quite misleading in the way that they are organized but in addition to areas where we had good consensus around best practices we also highlighted areas where there’s still uncertainty or we need more research and again we want to communicate that and be very transparent so that more people can join in as we do see this as requiring like many great minds around the world and they just aren’t enough safety and security researchers to do that all in one place but in addition to talking about the practice of evaluation we’re also trying to provide tooling for other organizations to do that and one of the things I’m very proud of the AC developed in the UK was the inspect framework there’s been open sourced and is used really extensively by different companies, organisations in government, outside government.

And the thing I would love to see over this coming year is how we can really grow a wide kind of ecosystem of third -party evaluators that can offer that independence and bring rigour and scientific method to the way that we measure these capabilities and then can communicate about them. And just I’m going to ask one quickfire question for the whole group and then I’m going to open it up for Q &A, so start thinking about your questions. But, you know, I’m interested, Adam, and I think it touches on some of the themes of like how do we take the science and bring it to practice and how do we actually create this evaluation ecosystem.

So step one is developing the science. Step two is then figuring out, well, how do we actually evaluate this? And then there’s the, you know, by whom. And how do you see an evaluation ecosystem? How do you see the ecosystem emerging? Do you see governments being the evaluators? Do you see this going more like we have with accounting, where you have third -party certified auditors doing the evaluations? I’d be interested in each of your thoughts. And maybe start with Minister Tia, and then we can go down the line.

Josephine Teo

Well, certainly in the ASEAN context, I would advocate for an approach that deals with near and present dangers that everyone is dealing with. The risk of not focusing on what’s most prominent in people’s minds today, policymakers’ minds today, is that the conversation may feel too theoretical, and we may lose interest and momentum, and we don’t even build the foundations of cooperating in a meaningful way. And what are some of those areas where AI intersects with? AI being used, misused, for harming people in terms of the content creation. I think that’s one. Almost every single policy. that I come across is very, very upset by the fact that they have to address their constituents’ concerns about all these harmful images that are being created with the help of AI.

It’s very offensive to our societies. And if we are not able to work on these areas in a meaningful way, in a practical way, then I think we risk losing my colleagues’ attention. So what can we do? We have to then seriously ask, is watermarking the correct approach of dealing with it? Is there some other way of labeling AI -generated content? Is that even the right direction that we should be moving on? The other area is that I think it will be very prominent, and that is the use of AI in cybersecurity. I don’t think at this point in time AI as a threat is adequately addressed. AI as a target is even further from that.

It’s in people’s minds. of the conversation in the areas that my colleagues care about, I think stands a better chance of anchoring their attention and creating meaningful opportunities for us to say, here are the ways you can test for it, and here are the tools that can be applied. They won’t be perfect, but they are important stuff.

Yoshua Bengio

So I want to mention maybe a totally different aspect that’s orthogonal to this. As I’ve been thinking about the process of bringing the science to have an impact with policymakers, I feel like there is a step in between what we’ve done and the actual political decision making, and that is using scientifically grounded policy options. So the report doesn’t go into recommendations, and I think that was a great mandate that we started from, but I think there is something in between taking the policy decisions and this. Thank you. grounded in what the scientists see and the people like economists and social scientists, based on this, what are reasonable options for policymakers without saying you have to take this one?

You could do this, you could do nothing. And what are the consequences that are expected based again on the science without making an actual recommendation? Because in the real world, I understand policymaking is hard because you always have a tension between different values and objectives and interests. We shouldn’t make those choices, but we can help make it easy for policymakers.

Alondra Nelson

I think I would offer we’re just getting started with evaluations and assessments. And so I wouldn’t want to put a thumb on a scale and pick one. I mean, I think that we actually have to try a lot of different things. I also think to the extent that we have a body of knowledge around evaluation that is coming from ACs and policymaking, and other researchers. Thank you. that, you know, I worry that we’re going to have a collective action problem and so that everyone’s doing their own different kind of evaluation. And I think what we will need to fundamentally do is make, as a research community, a few choices about, you know, something closer to a standard, like this is the way that we are, the few ways that we’re going to proceed.

So I think there’s that. I do think that it needs to be obviously multi -sector. It’s a fairly obvious point. How do you do that is an open question. I wrote a piece in Science a few months ago where I suggested that we might think about the LC program for human genetics and genomics in which, you know, 3 % of the Human Genome Project research budget in 1990, 1991, was dedicated to upstream research of potential risks and harm of human genetics. So that doesn’t present risks and harms, but it means that you go in upstream to projects thinking about them as a part of the research and design, often before deployment. And it doesn’t mean that you can prevent things like someone doing illegal human genome gene editing, right?

But I think that you do have a global community that has thought about it and is ready to have a conversation and knew in the case of the human genome, the human gene editing, that it was wrong and why it was wrong and we had discussed it. So I think that there are, you know, lots of models that government’s deeply important here and that, you know, I think that there are schemes that would require, I think, the public sector to, you know, place a little money in the space of a sort of common good or a comments for research to understand and advance much more in the evaluation and assessment space.

Adam Beaumont

Yes, you asked who should be involved in evaluation or where should be done and I guess my answer to that is should it be government, should it be industry? It’s kind of all of the above and I really agree with you that we’re very early in the journey and there’s still a lot of uncertainty but I do think there’s a role for governments to play, there’s a role for industry, there’s a role for researchers, civil society but also individuals and we saw that at the start of the year when people are very willing to trade away. They can trade away all their keys, passwords, anything for the… enjoyment of agent autonomy. And that reminds me a lot of the early days of cybersecurity where we need to grow ecosystems.

Individuals have a responsibility as much as governments and I’m sure over time we’ll see more institutions and organisations grow that help do that. But the key to it has got to be collaboration. So on a practical level, things like regulatory sandboxes or like policy lab type things where you can try limited pilot approaches seem to be good. We’re trying a bit of that in the UK. Things like joint funding programmes that bring researchers, policy makers together to kind of iterate options again seems a good idea. But I strongly agree we’re just early in the journey. We should keep options open.

Lee Tiedrich

Thank you. I think we have time for one or two questions. Wow, we have a lot of hands. What I’m going to do is call on two people. We’ll kind of combine the questions and we’ll let the panel. Well, I wish I had more time. So we’ll take one here and one over there. Go ahead. Right here in the second row. Can someone bring a microphone over? Or a speaker. It’s not a very good move, right?

Participant

Can you hear me?

Lee Tiedrich

Yes.

Participant

So I have a question. So, like, now we hear a lot about, like, the rise of business and sovereignty, like, everywhere, and, like, a lot of more countries are trying to claim it in some ways or another. And I would be really curious to hear, like, how, at least in the AI safety field, how are you seeing that impact and which other safety concerns are most pressing, like the grown -up of the window based on that first?

Yoshua Bengio

Yeah, so I think we should be careful about what sovereignty means. It doesn’t mean building walls around your country. It means making sure your country will retain the ability to, you know, take its decisions. And, you know, succeed economically. economically and politically. And often that means the opposite of walls around your country. It means making partnerships with others that increase your chances of, you know, not ending up in a bad place. And that includes agreements on safety, right, because many of the risks we’ve discussed, you know, they’re not limited by borders. We can collaborate on the safety technology with multiple countries. We can have the kinds of agreements that Singapore has been leading where multiple parties, you know, from many different countries agree on principles.

And eventually we will need international agreements and we will need technology for verification of these agreements. We are far from that, but that’s the only kind of world where, you know, I would want my children to live. Where AI is not used to dominate others and we don’t see, like, reckless behavior across. the world.

Josephine Teo

Ms. I’m so glad that Yoshua has offered a view that to me is a very sound approach. You said earlier that what we want is a world where every country can be at the table, not on the menu. And that’s exactly how you can preserve sovereignty, even with AI developments. The idea that you get sovereign AI by confining everything to your own shores, I think it gives a false sense of security. Firstly, it’s not achievable. Secondly, the idea that you can do so, I think, would mean that for many countries where the most sophisticated applications will have to originate from elsewhere, that just cuts you off. It cuts you off from being able also to make progress, and that puts you even further behind.

So how is that sovereign? So it has to be a topic that is dealt with thoughtfully. It’s not a term to be bandied about too easily.

Lee Tiedrich

Melinda, Adam, any thoughts? Okay. Yeah, so we unfortunately are running out of time, but I would love to thank our panelists for being here today and sharing the report. And I hope all of you will read the report and continue to engage with us because, as we said, there’s a lot more work to be done. Thank you very much. Thank you all. Thank you. you you Thank you. Thank you.

Yoshua Bengio

Speech speed

140 words per minute

Speech length

International Cooperation and AI Sovereignty

Topics

Artificial intelligence | The enabling environment for digital development

Lee Tiedrich

Speech speed

130 words per minute

Speech length

Evaluation, Evidence Gaps, and Building an Evaluation Ecosystem

Topics

Monitoring and measurement | Artificial intelligence

Josephine Teo

Speech speed

150 words per minute

Speech length

Evaluation, Evidence Gaps, and Building an Evaluation Ecosystem

Topics

Monitoring and measurement | Artificial intelligence

Alondra Nelson

Speech speed

193 words per minute

Speech length

International Cooperation and AI Sovereignty

Topics

Artificial intelligence | The enabling environment for digital development

Adam Beaumont

Speech speed

176 words per minute

Speech length

International Cooperation and AI Sovereignty

Topics

Artificial intelligence | The enabling environment for digital development

Participant

Speech speed

173 words per minute

Speech length

85 words

Speech time

29 seconds

Sovereignty concerns linked to AI safety

Explanation

The participant asks how AI safety considerations intersect with national sovereignty, seeking insight into the balance between security and autonomy.

Evidence

“And I would be really curious to hear, like, how, at least in the AI safety field, how are you seeing that impact and which other safety concerns are most pressing, like the grown‑up of the window based on that first?” [39].

Major discussion point

Emerging Risks of Autonomous and Multi‑Agent AI Systems

Topics

Artificial intelligence | The enabling environment for digital development

Impact of AI sovereignty on safety priorities

Explanation

The participant raises a question about how AI sovereignty might shape safety agendas, prompting discussion on international coordination.

Evidence

“So I have a question.” [132].

Major discussion point

International Cooperation and AI Sovereignty

Topics

Artificial intelligence | The enabling environment for digital development

Agreements

Agreement points

Need for international cooperation and partnerships rather than AI isolationism

Speakers

– Yoshua Bengio
– Josephine Teo

Arguments

AI sovereignty should mean retaining decision-making ability through partnerships rather than building walls around countries

True sovereignty requires being ‘at the table, not on the menu’ through international engagement rather than isolation

Summary

Both speakers strongly reject the notion that AI sovereignty should be achieved through isolation, instead advocating for international partnerships and cooperation to maintain true sovereignty and decision-making capability

Topics

Artificial intelligence | The enabling environment for digital development

Multi-agent systems present significant and concerning risks requiring careful attention

Speakers

– Yoshua Bengio
– Josephine Teo

Arguments

Multi-agent systems create concerning interactions when AI agents interact with each other in unpredictable ways

Multi-agent systems present architectural challenges requiring careful consideration of decision-making processes and guardrails

Summary

Both speakers identify multi-agent systems as a priority area of concern, with Bengio noting concerning interactions between agents and Teo emphasizing the need for careful architectural considerations and guardrails

Topics

Artificial intelligence | Building confidence and security in the use of ICTs

Need for rigorous scientific foundation for AI policy and evaluation

Speakers

– Yoshua Bengio
– Alondra Nelson
– Adam Beaumont

Arguments

Scientific rigor requires making only claims that can be strongly defended, with groups of researchers checking each other’s biases

Report establishes ground truth about AI risks based on rigorous scientific evaluation rather than anecdotal journalism

Need for clear evaluation methods that measure what they claim to measure, with transparency about uncertainties

Summary

All three speakers emphasize the critical importance of scientific rigor in AI research and evaluation, stressing the need for defensible claims, ground truth establishment, and clear methodologies

Topics

Artificial intelligence | Monitoring and measurement

Users should have safety assurance without needing technical expertise

Speakers

– Josephine Teo
– Lee Tiedrich

Arguments

End users should have assurance of safety without needing to understand complex testing procedures, similar to medical prescriptions

Medical analogy demonstrates how users can have safety assurance without understanding underlying mechanisms

Summary

Both speakers agree that end users should be able to trust AI systems’ safety without needing to understand the technical details, drawing parallels to how patients trust medical prescriptions

Topics

Artificial intelligence | Building confidence and security in the use of ICTs | Human rights and the ethical dimensions of the information society

Evaluation ecosystem should be multi-stakeholder and collaborative

Speakers

– Alondra Nelson
– Adam Beaumont

Arguments

Research community needs to establish closer-to-standard evaluation methods to avoid collective action problems

Evaluation approaches should be multi-sector involving government, industry, researchers, civil society, and individuals

Summary

Both speakers advocate for collaborative, multi-stakeholder approaches to AI evaluation while recognizing the need for some standardization to avoid fragmentation

Topics

Artificial intelligence | Monitoring and measurement | Capacity development

Artificial intelligence | Capacity development | The enabling environment for digital development

Unexpected consensus

Rejection of AI sovereignty through isolation

Speakers

– Yoshua Bengio
– Josephine Teo

Arguments

AI sovereignty should mean retaining decision-making ability through partnerships rather than building walls around countries

Confining AI development within national borders creates false security and cuts countries off from progress

Explanation

This consensus is unexpected because it directly challenges the growing trend of national AI sovereignty claims mentioned by the participant. Both speakers, despite representing different perspectives (academic researcher and government minister), strongly reject isolationist approaches to AI development

Topics

Artificial intelligence | The enabling environment for digital development

Importance of addressing immediate AI harms alongside long-term risks

Speakers

– Alondra Nelson
– Josephine Teo
– Lee Tiedrich

Arguments

Systemic risks including loss of human autonomy, job displacement, and social cohesion threats are equally important

Focus on near-term prominent dangers like AI-generated harmful content helps maintain policymaker attention and momentum

Current AI risks are immediate and increasing, requiring continued focus beyond just catastrophic scenarios

Explanation

This consensus is unexpected because it bridges the often-polarized debate between focusing on immediate AI harms versus long-term catastrophic risks. All three speakers agree that immediate risks deserve equal attention and are crucial for maintaining policy momentum

Topics

Artificial intelligence | Human rights and the ethical dimensions of the information society | Building confidence and security in the use of ICTs

Overall assessment

Summary

The speakers demonstrated remarkable consensus across multiple key areas: the need for international cooperation over isolation, the importance of scientific rigor in AI evaluation, the priority of multi-agent system risks, and the necessity of addressing both immediate and systemic AI risks. There was also strong agreement on maintaining clear boundaries between scientific assessment and policy prescription while ensuring practical implementation tools are available.

Consensus level

High level of consensus with significant implications for AI governance. The agreement spans academic, government, and policy perspectives, suggesting these viewpoints could form the foundation for coordinated international AI safety efforts. The consensus against AI isolationism is particularly significant given current geopolitical trends, while the agreement on addressing immediate risks alongside catastrophic ones could help bridge policy divides and maintain momentum for AI safety initiatives.

Differences

Different viewpoints

Approach to AI sovereignty and international cooperation

Speakers

– Yoshua Bengio
– Josephine Teo
– Participant

Arguments

AI sovereignty should mean retaining decision-making ability through partnerships rather than building walls around countries

True sovereignty requires being ‘at the table, not on the menu’ through international engagement rather than isolation

Rise of AI sovereignty claims across countries may impact AI safety field and create new pressing concerns

Summary

While Bengio and Teo strongly advocate for international cooperation and partnerships as the path to true AI sovereignty, warning against isolationist approaches, the participant raises concerns about the growing trend of national AI sovereignty claims and their potential negative impact on AI safety cooperation.

Topics

Artificial intelligence | The enabling environment for digital development

Prioritization of AI risks – immediate vs. systemic vs. catastrophic

Speakers

– Josephine Teo
– Alondra Nelson
– Lee Tiedrich

Arguments

Focus on near-term prominent dangers like AI-generated harmful content helps maintain policymaker attention and momentum

Systemic risks including loss of human autonomy, job displacement, and social cohesion threats are equally important

Current AI risks are immediate and increasing, requiring continued focus beyond just catastrophic scenarios

Summary

Teo advocates for focusing on immediate, concrete harms like AI-generated harmful content to maintain political momentum, while Nelson emphasizes the importance of broader systemic risks and their compounding effects on democracy and social cohesion. Tiedrich supports focusing on immediate risks but from a different angle, emphasizing agent-related risks.

Topics

Artificial intelligence | Human rights and the ethical dimensions of the information society | Social and economic development

Unexpected differences

Scope of AI safety report beyond catastrophic risks

Speakers

– Alondra Nelson
– Josephine Teo

Arguments

Systemic risks including loss of human autonomy, job displacement, and social cohesion threats are equally important

Focus on near-term prominent dangers like AI-generated harmful content helps maintain policymaker attention and momentum

Explanation

This disagreement is unexpected because both speakers are advocates for comprehensive AI safety, yet they have fundamentally different views on whether to prioritize immediate concrete harms or broader systemic risks. Nelson argues for the importance of systemic risks for democracy, while Teo warns that focusing on broader issues might lose policymaker attention.

Topics

Artificial intelligence | Human rights and the ethical dimensions of the information society | Social and economic development

Overall assessment

Summary

The discussion reveals relatively low levels of fundamental disagreement among speakers, with most tensions arising around prioritization and implementation approaches rather than core principles. Main areas of disagreement include the balance between immediate versus systemic AI risks, the appropriate level of international cooperation versus national sovereignty in AI governance, and the best methods for translating scientific findings into practical policy tools.

Disagreement level

Low to moderate disagreement level with high consensus on core principles. The disagreements are primarily tactical rather than fundamental, focusing on ‘how’ rather than ‘what’ or ‘why’. This suggests a mature field where experts agree on basic premises but are working through implementation challenges. The implications are positive for AI governance as it indicates potential for collaborative solutions despite different approaches to prioritization and implementation.

Partial agreements

Artificial intelligence | Capacity development | The enabling environment for digital development

Takeaways

Key takeaways

Independent scientific assessment of AI risks is crucial for evidence-based policymaking, requiring rigorous evaluation that only makes claims that can be strongly defended

AI agency and autonomous systems present significant new risks due to reduced human oversight, with multi-agent interactions creating unpredictable and concerning dynamics

A broader perspective on AI risks beyond catastrophic scenarios is essential, including systemic risks like loss of human autonomy, job displacement, and threats to social cohesion that compound simultaneously

AI capabilities are ‘jagged’ – systems can be dangerous in some areas while weak in others, requiring precise technical evaluation rather than abstract AGI thinking

Practical implementation requires translating scientific findings into operable guardrails and standards while maintaining innovation, with end users needing safety assurance without technical expertise

International cooperation on AI safety is essential for true sovereignty, which means retaining decision-making ability through partnerships rather than isolation

A multi-sector evaluation ecosystem involving government, industry, researchers, civil society, and individuals is needed, with third-party evaluators providing independence and scientific rigor

Resolutions and action items

Continue supporting independent scientific reporting on AI capabilities and risks through ongoing assessment updates

Develop policy options grounded in science that present choices to policymakers without making specific recommendations

Focus on near-term prominent dangers like AI-generated harmful content to maintain policymaker attention and build cooperation foundations

Grow ecosystem of third-party evaluators using tools like the open-sourced inspect framework

Establish closer-to-standard evaluation methods within the research community to avoid collective action problems

Explore regulatory sandboxes and joint funding programs to bring researchers and policymakers together

Consider upstream research models like dedicating percentage of AI research budgets to risk and harm assessment

Unresolved issues

How to effectively address the evidence gap for longitudinal studies in rapidly evolving AI landscape

Specific mechanisms for translating scientific findings into practical tools for small and medium-sized organizations

Optimal balance between government, industry, and third-party roles in AI evaluation ecosystem

Technical solutions for watermarking and labeling AI-generated content

Development of international agreements and verification technology for AI safety

How to maintain focus on both immediate risks and longer-term systemic threats simultaneously

Specific approaches for addressing AI as both a cybersecurity threat and target

Suggested compromises

Balance between providing scientific rigor and practical policy guidance by developing evidence-based policy options without specific recommendations

Address both immediate concerns (like harmful AI-generated content) and longer-term risks to maintain stakeholder engagement while building comprehensive safety frameworks

Combine multiple evaluation approaches across sectors rather than choosing single methodology, allowing experimentation while working toward standards

This comment challenges the prevailing narrative around AI sovereignty, arguing that isolationist approaches actually undermine rather than enhance national autonomy. It reframes sovereignty from technological independence to strategic participation in global governance.

Impact

This directly challenged conventional thinking about AI sovereignty and provided a more nuanced framework for international cooperation. Combined with Bengio’s complementary response, it offered an alternative vision for how countries can maintain agency while participating in collaborative AI governance.

Overall assessment

Explanation

Cross-border AI risks require international cooperation, but the mechanisms for verification and enforcement of agreements are underdeveloped

Disclaimer: This is not an official session record. DiploAI generates these resources from audiovisual recordings, and they are presented as-is, including potential errors. Due to logistical challenges, such as discrepancies in audio/video or transcripts, names may be misspelled. We strive for accuracy to the best of our ability.