From principles to practice: Governing advanced AI in action

Summary

This discussion focused on moving from AI governance principles to practical implementation, featuring experts from academia, government, and industry across different global regions. The conversation was framed as “AI governance speed dating” to explore various stakeholder perspectives on what approaches are working and what needs improvement in governing rapidly advancing AI technology.

Brian Tse opened by identifying four key frontier AI risks: misuse by malicious actors, accidents and malfunctions, potential loss of control scenarios, and systemic societal risks including labor market disruption. He emphasized the need for multi-stakeholder collaboration to address these global-scale challenges. Chris Meserole highlighted the promise of Frontier AI frameworks and voluntary commitments made at the Seoul AI Summit, noting that some risks require entirely new risk management instruments due to their potential for rapid, large-scale emergence.

Yaqin Zhang discussed practical implementation challenges, particularly the lag between rapid technological development and regulatory responses, advocating for continued R&D investment in risk identification and enhanced global collaboration. Juha Heikkila explained the EU’s approach through the AI Act, emphasizing trust as fundamental to AI adoption and describing mechanisms like the Code of Practice that allow for flexibility and updates while maintaining compliance frameworks.

Udbhav Tiwari provided a concrete example of Signal’s response to Microsoft’s Recall feature, illustrating how companies must sometimes take protective action independently when existing governance frameworks prove insufficient. He stressed the importance of addressing current real-world harms rather than focusing solely on hypothetical future risks.

The panelists’ recommendations for scaling effective governance included binding regulations for powerful AI systems, standardized frontier AI frameworks, honest conversations about limitations, safety-by-design approaches, and international cooperation on defining unacceptable AI outcomes. The discussion concluded with calls for developer agency, content identification systems, streamlined initiatives, and transparency mechanisms to build the trust necessary for responsible AI adoption and innovation.

Keypoints

## Major Discussion Points:

– **Frontier AI Risk Categories**: Four key types of risks were identified: misuse (lowering barriers for malicious actors in cyber attacks and dangerous pathogen design), accidents and malfunctions (like AI hallucinations in medical diagnosis), potential loss of control scenarios (where AI systems may deceive humans or evade oversight), and systemic societal risks (particularly profound labor market impacts).

– **Governance Approaches and Frameworks**: Discussion of various regulatory models including voluntary commitments (like the Seoul AI Summit frameworks), legislative approaches (EU AI Act), and industry self-regulation. Emphasis on the need for new risk management instruments specifically designed for rapidly emerging AI risks, particularly those that could scale quickly before harms become apparent.

– **Implementation Challenges Across Jurisdictions**: Participants highlighted the tension between rapid technological advancement and regulatory lag, with different regions (China, EU, US) developing divergent approaches. Key challenges include the need for regulatory clarity, threshold definitions, and coordination across different governance models.

– **Trust and Transparency as Core Requirements**: Strong emphasis on building societal trust through transparency, honest conversations about AI limitations, developer accountability, and “privacy by design” approaches. The discussion included concrete examples like Signal’s response to Microsoft’s Recall feature to protect user privacy.

– **International Cooperation and Standardization Needs**: Call for global alignment on defining “red lines” for unacceptable AI outcomes, standardizing risk management frameworks, streamlining proliferating initiatives, and establishing norms for transparency in frontier AI development.

## Overall Purpose:

The discussion aimed to explore practical approaches for translating AI governance principles into actionable practices, bringing together diverse stakeholders from academia, government, and industry to share lessons learned and identify effective governance models that can keep pace with rapid AI advancement.

## Overall Tone:

The conversation maintained a collaborative and constructive tone throughout, characterized by mutual respect among panelists despite representing different sectors and regions. The tone was serious and urgent given the stakes involved, but remained optimistic about the potential for international cooperation. The moderator’s “AI Governance Speed Dating” framing added a light touch while keeping the discussion focused and dynamic. All participants demonstrated a shared commitment to responsible AI development, even while advocating for different approaches.

Speakers

– **Sasha Rubel** – Head of Public Policy for Generative AI at AWS, Moderator

– **Ya Qin Zhang** – Chair Professor at Tsinghua University

– **Juha Heikkila** – Advisor for the International Aspects of Artificial Intelligence at the European Commission

– **Udbhav Tiwari** – Vice President of Strategy and Global Affairs at Signal

– **Brian Tse** – Role/title not specified, but appears to be involved in AI safety research and policy

– **Chris Meserole** – Role/title not specified, but appears to be involved with the Frontier Model Forum and AI governance

Additional speakers:

– **Unnamed moderator/host** – Introduced the panel at the beginning and concluded the session

# AI Governance Speed Dating: From Principles to Practice – Discussion Report

## Introduction and Context

This panel discussion, described as “AI governance speed dating,” brought together experts from academia, government, and industry to explore the transition from AI governance principles to practical implementation. The conversation was moderated by Sasha Rubel, Head of Public Policy for Generative AI at AWS, and featured Ya Qin Zhang (Chair Professor at Tsinghua University), Juha Heikkila (Advisor for the International Aspects of Artificial Intelligence at the European Commission), Udbhav Tiwari (Vice President of Strategy and Global Affairs at Signal), and Brian Tse.

## Frontier AI Risks Framework

Ya Qin Zhang established a comprehensive framework for understanding AI risks, identifying four critical categories requiring immediate attention:

**Misuse by Malicious Actors**: AI systems that lower barriers for malicious activities, particularly in cyber attacks and dangerous pathogen design. Zhang emphasized these represent immediate threats with current AI capabilities.

**Accidents and Malfunctions**: AI systems producing incorrect or harmful outputs due to technical failures. Zhang provided a concrete example from Beijing, where medical prescription regulations were implemented after AI systems began making inappropriate medical recommendations.

**Potential Loss of Control Scenarios**: Situations where AI systems may deceive humans or evade oversight mechanisms, representing perhaps the most concerning category of risks.

**Systemic Societal Risks**: Broader societal impacts, particularly profound labor market disruption that could create systemic challenges requiring coordinated responses.

Zhang noted the regulatory gap with a striking analogy: “we have more regulations on the food safety of dumplings than [on AI systems] today.”

## Voluntary Commitments and Standards Development

Brian Tse highlighted developments in frontier AI governance, particularly voluntary commitments from international forums. He noted that some AI risks require entirely new risk management instruments due to their potential for rapid, large-scale emergence before harms become apparent.

A critical challenge Tse identified is the timeline mismatch between AI development and standards creation. Current formal standards processes, such as ISO SC402001, typically require “four to five years to develop”—incompatible with rapid AI advancement. He emphasized the need for accelerating formal standards development while maintaining rigor.

Tse recommended that firms publish their risk management frameworks and provide systematic implementation updates for public transparency, noting that “15 or 16 companies” have signed commitments at recent international summits.

## European Union’s Legislative Approach

Juha Heikkila provided insights into the EU’s approach through the AI Act, emphasizing trust as fundamental to AI adoption. The EU’s framework aims to build trust through clear requirements and accountability mechanisms that can adapt to technological changes.

Heikkila identified a critical gap in current approaches: the lack of systematic follow-up and effectiveness measurement. He emphasized that “for trust, which has been identified here as one of the key issues, it is actually important to know how and if and how, and to what extent, whatever measures and recommendations are there, were put into practice and how effective they are.”

He also addressed the challenge of proliferating AI governance initiatives, noting that multiple overlapping requirements create excessive burdens on developers and advocating for streamlined coordination between different governance frameworks.

## Academic Research Perspective

Ya Qin Zhang emphasized the critical lag between rapid technological development and regulatory responses, creating windows of vulnerability where harmful applications can emerge before appropriate safeguards are in place.

Zhang advocated for a “five-stage process: testing, auditing, verification, monitoring, and mitigation” and emphasized the importance of “ID, ID, ID” for registration and identification systems. She highlighted China’s regulatory approach, including generative AI registration requirements and the Beijing medical prescription regulations as examples of proactive governance.

From a research perspective, Zhang called for increased investment in R&D for identifying critical risks, developing appropriate benchmarks, and creating early warning systems. She argued that “safety must be foundation of product design, not an afterthought, with academic-industry-policy collaboration.”

## Industry Implementation Challenges

Udbhav Tiwari provided concrete examples of real-world implementation challenges through Signal’s experience. His most significant example was Signal’s proactive response to Microsoft’s Recall feature, which takes periodic screenshots of users’ screens. Signal deployed a feature preventing screenshots when the application is active to protect user privacy.

This example illustrated gaps in current governance frameworks, with Tiwari noting that “AI regulation has existed for some time, voluntary efforts have existed for quite some time, but yet Microsoft could deploy a lot of technology” that created privacy risks for users.

Tiwari emphasized the importance of addressing current, real-world harms: “it is just as important to recognize the very real harms that are occurring to real people with very real consequences that are taking place today because of these AI systems.”

He also criticized declining industry transparency, noting that model cards have become less informative over time due to industry hype, occurring precisely when greater openness is needed for effective governance.

## Key Recommendations

The discussion generated several specific recommendations:

**International Coordination**: Developing international standards for frontier AI risk management within accelerated timelines and establishing global norms for risk management transparency.

**Regulatory Mechanisms**: Implementing binding regulations for the most powerful AI systems while maintaining voluntary frameworks for lower-risk applications, and creating adaptive regulatory mechanisms that can be updated more quickly than full legislative revisions.

**Technical Standards**: Scaling registration and identification systems for AI-generated content globally and implementing content provenance and watermarking systems.

**Accountability and Monitoring**: Establishing follow-up mechanisms to measure governance effectiveness and requiring systematic publication of risk management frameworks and implementation updates.

## Ongoing Challenges

Several critical issues remain unresolved:

– Balancing rapid technological advancement with necessary governance frameworks across different regional approaches

– Lack of consensus on what constitutes “intolerable risks” and appropriate risk thresholds globally

– Tension between transparency needs and proprietary commercial development

– Ensuring inclusive governance that gives meaningful voice to underrepresented communities

## Conclusion

The discussion revealed both promise and challenges in translating AI governance principles into practice. While speakers shared common goals around safety, trust, and international cooperation, they demonstrated different approaches to implementation, particularly regarding the balance between voluntary and binding frameworks, and between addressing current harms versus preparing for future risks.

The conversation highlighted the urgent need for governance frameworks that can keep pace with technological development while addressing both immediate and long-term risks. Success will likely require hybrid approaches combining voluntary flexibility with regulatory accountability, maintaining collaborative approaches while navigating significant implementation challenges ahead.

Sasha Rubel: AI. I’m not hearing the round of applause. I’m calling for to welcome our guests here. Thank you. Yajing Zhang, Chair Professor at Tsinghua University. Yuhua Heikila, who’s Advisor for the International Aspects of Artificial Intelligence at the European Commission. And Udvav Tiwari, who’s Vice President of Strategy and Global Affairs at Signal. In a conversation moderated by Sasha Rubel, who’s Head of Public Policy for Generative AI at AWS. We wish you all a very fruitful conversation. Thank you. Thank you everybody for the warm welcome. Over the next half hour, we are going to deep dive into questions related to how to move from principles to practice in governing AI. That is the official title. The unofficial title could have been called AI Governance Speed Dating, where over the next 30 minutes or so, we will hear different perspectives from different stakeholders and regions of the world in terms of their approaches of what they are seeing working, what they are seeing not working, what they think needs to be scaled. And if you feel like swiping right on all of the above, you can. So I want to make sure that we allocate as much time as possible to the meat of these conversations and lessons learned that all of you are going to share. And I want to kick it off in terms of framing the conversation with you, Brian. Can you share what you’re seeing in terms of key frontier AI risks and what we should be paying attention to

Brian Tse: right now? First of all, it’s a great honor to be on this panel today. To ensure that AI could be used as a force for good, we have to start by identifying and managing some of the global scale challenges that we face as humanity. Drawing from the International AI Safety Report and some of the latest research, I think there are at least four types of frontier AI risk. Number one is misuse. As frontier AI systems achieve human expert level performance on areas like scientific reasoning, coding, and even philology, there’s also the potential for lowering the barrier for malicious actors to launch cyber attacks or even design dangerous pathogens. For example, recent studies have shown that teams of AI agents are able to exploit zero-day vulnerabilities in real-world settings in open-source softwares. So one measure could be providing early and differential access for developing countermeasures to make sure that we have defensive capabilities on the upper hand over the attackers. Number two is accidents and malfunctions. While hallucinations from chatbots could be quite amusing, it could be dangerous in use cases like medical diagnosis. So recently the Beijing government has implemented regulations prohibiting AI from writing medical prescriptions automatically. So classifying some applications as high-risk and making sure that we have robust public safety regulations is critical. Number three is potential loss of control scenarios. As Professor Geoffrey Hinton has mentioned in the summit, we don’t currently have robust mechanisms to ensure that digital superintelligence could be remained under human control. In fact, there is a growing body of evidence showing that frontier AI systems have the tendency to deceive humans or even try to evade human oversight under particular conditions. Now of course, frontier AI systems still struggle to bypass know-your-customer checks and will struggle to automatically replicate in the wild. But I think for catastrophic risk, even if they are low probability or of unknown probability, we should take precautionary measures. The fourth one is systemic risk at the societal level. One particular example is that as general purpose AI outperform humans on many of the economically valuable tasks, the impact on the labor market will be profound. And these gradual and diffuse impacts cannot be managed by any single organization. So we need a multi-signal approach, both at the national and the global level, to ensure that no one will be left behind. And I think that’s what makes the ITU platform so valuable. Thank you.

Sasha Rubel: Brian, thanks so much for that introductory overview. And I think your emphasis on the need for multi-stakeholder collaboration and dialogue and alignment is one of the reasons why this summit is so important, but also why it’s absolutely crucial to get these different groups in the room to agree on what does risk look like, what are we talking about, and what kind of governance is most effective. It’s a challenge, and all of you know this firsthand in the work that you’re doing, to think through governance models that address risks in a technology field where in reality advancements are moving super quickly. And Chris, you’ve done a lot of work at the Frontier Model Forum trying to think through very concretely what kind of governance models or approaches, whether that’s regulatory, institutional, or voluntary, can actually address the fast pace of the development of this technology. What do you believe is showing the most promise in terms of these different approaches that you’re seeing, particularly as it

Chris Meserole: concerns possible global alignment? Well, first of all, it’s great to be here and just, you know, a wonderful chance to speak to such a great multi-stakeholder audience for a really important conversation that hopefully we can make a lot of progress together with. You know, as far as kind of which approaches might have most promise, one thing that I would really flag would be the Frontier AI frameworks and the kind of voluntary commitments that a set of companies made in Seoul at the Seoul AI Summit in May of last year, where they committed to, you know, again, there were about 15 or 16 companies, I believe, globally that signed these commitments, and the commitment was to identify what they considered intolerable risks and, like, their plan for managing those risks. And, you know, over the last year and a half, most of the companies that made that commitment have spent a lot of time trying to flesh out what are now called Frontier AI frameworks or what others might call a safety and security framework. And the goal of this is to try and make sure that we have a risk management framework that is fit for purpose for the specific kinds of risks that were just alluded to. You know, Brian, I think, did a great job of outlining the full array of risks that might come with Frontier AI. Some of those risks, I think, can be managed by adapting existing risk management instruments, but there’s other risks that we might actually need an entirely new type of risk management instrument for, and in particular, there’s a class of risk that, if it appears, it may appear very quickly and at, you know, extreme scale very quickly in a way that, like, the harms don’t materialize until it’s too late to do anything about them. In which case, you need a risk management instrument that’s able to identify those issues in advance or as far in advance as possible, and that’s what Frontier AI frameworks are really designed to do, and I think there’s been a tremendous amount of progress over the last year in identifying which risks fall into that category. I think there’s growing consensus that, like, bio risks, advanced cyber risks, autonomous R&D capabilities, those kinds of capabilities really require, I think, a fundamental new type of risk management instrument that we’re starting to see a lot of. I would say right now we’re in the kind of emerging experimentation phase of what those should look like, but hopefully over time we’ll

Sasha Rubel: increasingly institutionalize them going forward. Chris, I love how you underline that a lot of the risks that we’re going to have to pay attention to are probably going to emerge really quickly, and the question of, okay, how do we see what already exists that can be adapted, and then where do we see the need to completely invent a new approach and framework to risk, and I want to turn, Yachin, to you because I’m curious, in this kind of approach of seeing where the risks are, what might emerge, what exists already, and what we might need to develop, what do you see as the practical challenges when implementing AI governance approaches, especially as models become more and more powerful, and how can we actually overcome some of these challenges?

Ya Qin Zhang: For me? All right. Nice to be here. There are a lot of impedances in implementation. I will just list three. First is just the rapid technological development events versus the governance regulatory lag, and if you look just in the last six months or a year, the tremendous AI safety. In the UK, EU has different guidelines, China has different guidelines, and the US has very different guidelines as well. Obviously, China is quite open and very inclusive in terms of how to approach the thing with companies. We’re very close with industries, defining how you collect data, how do you do testing, how do you register, and how do you release the big models. How do we overcome that? Probably time is not enough, but let me just mention one thing. We need to continue to invest in R&D in terms of how we identify those critical risks, catastrophic risks, to come up with the red lines, the benchmarks, the thresholds, and the warning system. Let me also just, I will talk later if I have time, the five-stage process, testing, auditing, verification, monitoring, and also the mitigation. But overall, I think the most important thing is for all of us to come together for enhanced global collaboration in a forum like this.

Sasha Rubel: Yatin, I love your emphasis on needing to invest in R&D, and also the need to have regulatory clarity in light of the pace of the technological advancement, but also the context that you provide in terms of different countries going ahead and developing different approaches to questions around risk, and also to AI governance. I want to turn to you now, at the forefront of a lot of these conversations in the EU as the EU implements the EU AI Act and the secondary legislation and guidelines related to its implementation, what are you seeing as practical challenges when implementing AI governance, again, especially as these models are becoming more and more powerful? There were conversations during the EU AI Act about thresholds, systemic risk, guidelines that are currently underway with the Code of Practice for general purpose AI. What do you see as the challenges and also the opportunities to overcome them?

Juha Heikkila: Thank you. Thank you very much. It’s indeed a great pleasure to be here and to be a member of this panel. So, first of all, the term governance is used in different ways. So, we have different kinds of international initiatives, principles, codes of conduct, recommendations, etc., on the one hand. Then, on the other hand, we have indeed either soft guardrails or hard law within jurisdictions. So, those have, of course, slightly different consequences, different implications in terms of compliance and monitoring compliance and enforcement, and the possibility to do that. So, this is actually one challenge in and of itself, which is not necessarily even dependent on, though it is complicated by, the progress of this technology. It is, of course, a well-known fact, as you said, that the European Union has decided to regulate AI. For us, it’s key to ensure trust, and we felt that for ensuring trust legislation was necessary, because trust is really the sine qua non for the take-up of this technology, which is a sine qua non for the benefits of AI to materialize, and we do believe that there are really very many benefits in this technology. So, in the AI Act, we have introduced a mechanism of enabling updates in a way which can keep up with this technology, so we have the high-risk areas defined in an annex, and that can be updated much more quickly and much more easily than having a full revision of legislation. On the other hand, you mentioned the Code of Practice. This is a process which is producing an instrument, a tool, to help facilitate compliance with the provisions on general-purpose AI models, and this process, it’s about to be published, this Code of Practice. It has been conducted, led by independent chairs and co-chairs, with strong multi-stakeholder support. We’ve had more than 1,000 stakeholders involved in this, and indeed, this highlights one of the aspects that was mentioned by Brian at the very beginning, this multi-stakeholder aspect is very important in these processes. But the Code of Practice is aimed at taking into account best practice, current industry practice, to the extent that it is feasible, and also to make it as future-proof as possible. But there is also some flexibility in implementing it, so those who sign up to this Code of Practice, they can in some cases choose the way, for example, how to put in place mitigation measures to comply with this Code of Practice, and this then obviously would be a presumption of compliance with the relevant provisions. It also refers to the state of art in a number of places, so that can be interpreted flexibly to take into account where we are. And of course, it is possible also for us to review it and revise it as necessary and clarify concepts if it’s required. Thank you.

Sasha Rubel: Thank you, Juha. I again appreciate your emphasis on the question of trust, because one of the things that we’re seeing in conversations with organizations across the world, as a blocker to AI adoption, is actually the question of trust, and how do we build trust both in the technology by having an approach of responsibility by design, but also between stakeholders that need to come together and work together to design these kinds of governance approaches. I want to turn to you, Udbav, on that specific issue, because I would be curious to hear from you at Signal what are you seeing in terms of the practical challenges in implementing governance, but also building trust? Signal has really been at the forefront of making sure that trust is the center of the ways in which you operate as an industry counterpart. How are you implementing AI governance inside of Signal, and what do you see as both the challenges, again, but also opportunities to build that trust that’s necessary?

Udbhav Tiwari: Thank you. Maybe I’ll start with an example. The purpose of the example is to illustrate how Signal, as a player in the market, is being both impacted and affected by AI, but is in turn also shaping how AI is deployed in practice, and maybe use that to talk about governance and what’s working and what’s not. Quite recently, Signal rolled out a feature on its desktop installations, where if you run Windows 11 and you try to take a screenshot on Windows 11 of the Signal application, all you will get is a blank screen. The reason we deployed that feature was because Microsoft had deployed Recall, a feature that takes periodic screenshots of your screen, stores them, runs an on-device model that analyzes those screenshots and answers questions that you might have about, say, when did I visit this website, or when did I go to this particular page, or when did I last look up this person’s profile? The reason we had to do that was because people use Signal because of the fact that they trust it with their most private information, and not everyone who uses Signal would necessarily understand. I’m here to talk about the summer of 2019. We understand the fact that the recall feature would be taking screenshots of the screen every three seconds and storing them in perpetuity to be able to answer questions that they might have. If you look at signal as a player in this market, AI regulation has existed for some time, voluntary efforts have existed for quite some time, but yet Microsoft could deploy a lot of technology to prevent that from happening, and that’s why we had to take action in our own hands, because we had to protect the privacy of our users against the kinds of technologies and context in which they are deployed, which are quite different from the context of many other AI systems, because a lot of the people who use signal are particularly high-risk individuals. Which is why connecting that to governance, I actually think that the solution to a lot of this governance conversation is all of these things need to happen in parallel. I don’t think signal is under pressure. I don’t think that a law will suddenly come into place by early next year that will prevent things like recall from being deployed without users understanding what it really means. I don’t think it means that only voluntary efforts are enough, because I think we’ve seen a lot of precedence from social media and a lot of other places in the industry that self-regulatory efforts have not really worked in practice, and I think ultimately acknowledging that as much as we talk about advanced AI, it is just as important to recognize the very real harms that are occurring to real people with very real consequences that are taking place today because of these AI systems. And unless we choose to address them rather than wondering whether AI will control Skynet three years from today, I think we’re going to end up in a scenario where some very real harms will occur and the trust that the society has on these technologies, which is already falling quite a bit, because it’s not exactly a very popular technology, if you talk to an artist and talk to a lot of other individuals, will keep decreasing. So if we want to earn society’s trust, I think we all need to both work together but also ensure we prevent today’s harms from occurring rather than focusing too much on what might

Sasha Rubel: be possible tomorrow. I love that call to action, Udbhav. Yeah. I love that call to action. It’s clear that many of you in the room also love that call to action in terms of making sure that we’re thinking through and starting with and working backwards from the impact that this has on individuals and users and communities, but I also think your emphasis on privacy by design and working in parallel are equally important. How do we embed in the approach from the beginning and the ways in which we develop this technology in parallel to efforts of governance and regulation and voluntary commitments privacy and responsibility by design in order to build that trust to encourage responsible adoption. We’ve heard a lot of different kinds of models and lessons learned from the five of you, and I want to shift a little bit to think through concretely if there was one element of governance that you would suggest be widely adopted or better coordinated across jurisdictions and stakeholder groups, what would it be and why? Some of you talked about R&D, voluntary commitments like in Seoul, legislative initiatives like the EU AI Act, internal governance approaches like what’s happening at Signal. What is the model of AI governance that you would want to push forward and make sure that there’s more cooperation on a global level that we can develop, as you said, very elegantly in parallel at the same time as the development of this technology? Brian, I want to start off with you. Thank you. Given the profound impact from front AI systems in the coming years, it’s quite striking that we have more regulations on the

Brian Tse: food safety of dumplings than they are today, and so I think we need more binding regulations, especially for the most powerful AI systems. Just to share two regulatory measures from my region, so the first is pre-deployment registration and licensing. Currently in China, all generative AI would have to be registered with the government before they are deployed to the public, going through a set of safety and security assessments, and this type of registration and licensing could adapt to different regional needs as well as to adapt to emerging challenges as our understanding of the threat landscape deepens. The second would be post deployment transparency. For example, having content provenance and watermarking on AI-generated content to make sure that users and society are able to differentiate the realities between those generated by AI versus humans. I think that’s critical, and that’s something that China has

Sasha Rubel: also implemented through a national standard requiring both developers as well as platforms to have explicit labeling as well as implicit metadata. Thank you.

Chris Meserole: Chris, let’s turn to you. What would be your recommendation in terms of something that’s working to scale and how? I think I would build on the comments I was making earlier in the sense that I think one of the things that hopefully will happen over the next 12 months is an increasing push to start standardizing frontier AI frameworks or risk management systems. There’s a lot of voluntary activity around them. That means there’s a lot of different types of frameworks that have been published, but I think coordinating globally around what a robust frontier AI risk management instrument should look like will be incredibly important. In doing that, though, I think we need to identify how we’re going to create formal standards on a very quick timeline, because I think the things like SC402001 at ISO, which is a great AI risk management standard, I think it took four or five years to develop, and I think we need to really be able to provide something with that kind of rigor, but in a much

Sasha Rubel: shorter timeline. I love that emphasis on the question of standards in terms of their importance in providing clarity, particularly as we think through and hear from organizations that want to scale, but in a safe way. Udbhav, over to you.

Udbhav Tiwari: I would say that two main things. The first is there is, I think, a recognition that the rate at which some of these technological items are being deployed is far faster than the rate at which they’ve really been understood, even sometimes by the organizations that develop and deploy them, and without that understanding, I think it’s very, very hard for us to come out, to come to the conclusion that if we follow process X or standard Y, all of these risks in these systems will be addressed, because in many ways it’s those risks themselves that are not very clearly understood, and the reasons they aren’t really understood is because of the fact that they often happen in incredibly proprietary and closed settings with very little interaction with the outside world, and the second is that even when it does happen, the consequences so much, like every time there’s a big model release, like everybody, that’s all everyone talks about for that day, like, on the internet, like this new model has come out, that people tend to be a lot more risk-averse on talking about the limitations and challenges of the kind of models that have been developed and things that they should not be used for. So if you just look at the model cards for all the prominent model providers over the last three to four years and just analyze the amount of information that used to be present in them and the amount of information that is present in them today for when they come out, we’ve actually seen a regression, and the reason we are seeing that regression is because there is a greater consequence to that information because of all the hype that has really built up around the industry. So I really think that we need to make sure that there’s space and opportunity for people to be able to be clear about faults and drawbacks just as much as there is approaches to what the benefits and progress that this technology is making, because unless we have an honest conversation about that, both from a deployment but also from an addressing problems perspective, I think, independent of whether it’s a self-regulatory body or an actual legislation, we will not be able to have an honest conversation, and that’s what’s really important.

Sasha Rubel: I love that emphasis on the need to have honest conversations. I think in parallel to the question around honest conversations, you also implicitly underlined the need to be asking the right questions. What kind of questions do we need to be asking? And a lot of times very hard questions with regards to both misuse but also missed use and beneficial use of this technology. Yaxin, over to you.

Ya Qin Zhang: I thought the National AI Safety Institute and a lot of the NGOs have played a very constructive and positive role in the last couple of years. you know, the assess model, you know, the work with the industries to monitor and publish the results. So that was great. Let me also mention one thing, and, you know, I’m glad to see top academics and thought leaders like Yashua Bengio and a lot of the AI pioneers are working with industry leaders and policy makers, and to make this work, you know, we have to do this in the very beginning, right? By design and the safety is a foundation of the product.

Sasha Rubel: It’s not an afterthought. I love that. Safety is the foundation and not an afterthought. It’s again one of the reflections that comes up more and more in conversations is how do we move from an approach of responsibility and safety that from bolted on to built in to by design? How do we ensure approaches that are by design safe and trustworthy? And I think Juha, you’ve spent a lot of time thinking about that as well in the work that you’re doing on the EU AI Act. What’s a kind of key lesson that you’ve taken away in terms of conversations on AI governance in the EU? Well, I think there are a number of lessons that could be drawn from that and could be related, but I think, you know, these are complex issues and they don’t actually, complex issues rarely lend themselves to simple solutions as much as we’d like to have simple solutions.

Juha Heikkila: But I think overall one thing that maybe is worth highlighting is the need for follow-up and update mechanisms. I mean, update we have already talked about, but follow-up as well, because for trust, which has been identified here as one of the key issues, it is actually important to know how and if and how, and to what extent, whatever measures and recommendations are there, were put into practice and how effective they are. So I think that this is one aspect that is important, of course, in the context of legislation that requires this kind of a mechanism that is kind of inbuilt, but it would be required also then for other sort of softer approaches.

Sasha Rubel: I want to build on your last statement, Juha, and shift to a kind of closing call to action and reflections in the couple of minutes that we have left, and ask all of you, looking ahead over the next 12 months, and 12 months in the time of AI is probably in dog years, 12 years, what’s one concrete step, is it a norm, a protocol, collaboration that’s needed, a standard you’d like to see move forward to make AI governance more effective in practice? A lot of you have talked about the need for international collaboration and alignment, the need for gaps analysis, the need for multi-stakeholder conversations. What’s one thing that you would like to see on the international agenda prioritized in terms of an approach that you’re seeing work that will move the needle on AI governance and particularly trustworthy and safe AI? Chris, I want to start with you.

Chris Meserole: I would just pick up on Juha’s point about trust and say that one of the things that really needs to happen globally is a norm that’s developed where firms that are developing frontier AI models, like genuinely kind of the newest, most powerful models, that they publish a risk management framework about how they’re going to handle the most extreme risks associated with their technology. And then that they also publish, this is a point that was also made, that they publish kind of systematic updates on how they’re implementing it so that the public can be aware of the activity that they’re doing and can kind of judge for themselves exactly how trustworthy these products are. Great call to action. Brian, over to you.

Brian Tse: I think a concrete step I would like to see is an international effort to define what we view as red lines or unacceptable outcomes from advanced AI systems. And based on those red lines, we can also better define a set of risk threshold and early warning indicators, as Professor Zhang Aqin has mentioned. And what is acceptable and what is safe should not be left to the AI industry alone. And as hopefully we have learned from past financial crises and environmental disasters, some of the unchecked risk-taking by a few elites could actually lead to systemic risk on a global level, including for the Global South and unprotected communities worldwide. So I think this is an area where the international community and civil society have to come together

Sasha Rubel: and have a strong and decisive voice. I love that call to action of coming together. Juha, what would you like to see over the next 12 months?

Juha Heikkila: Well, 12 months is a long time in AI, but it can also be quite a short time for some things. And I’m thinking of one thing in particular, which is the fact that we had kind of a proliferation of initiatives, which also means that there’s been a proliferation of monitoring, reporting, et cetera, requirements, and this can put burden on developers. And I think that maybe some streamlining of initiatives would be quite useful. And for that, however, 12 months is quite a short time. So it really depends on what you look at, whether that’s a long or a short time perspective. Great call to action on streamlining.

Sasha Rubel: Yachin?

Udbhav Tiwari: For the last three years, I’ve been a strong advocate for registration, identification of AI-generated contents, models, and particular agents. I’m glad to see some of this is happening in China, and I hope that can scale to the rest of the world. It’s ID, ID, ID. ID, ID, ID. Udbav, over to you. For me, I’d say it would definitely be developer agency, very similar to the recall example that I mentioned earlier. I think it’s vital that application providers be able to make decisions for their users in order to protect them from risks that AI systems could pose. Privacy is one of them. It’s okay for a user to be able to override those protections, but right now I think there’s been a pretty symbiotic relationship between operating system providers and applications that run on them. I think we’re going to see that radically change with the deployment of AI systems, and in order to make sure that if something is sensitive, the user doesn’t necessarily need to be able to menu dive deep into settings in order to be able to protect their settings, but an application should be able to protect the privacy of the user on their behalf, like we did with recall. It should become much more of a norm, because without that, we’re not going to have 7 billion people being able to protect themselves to the level that they deserve.

Sasha Rubel: One of the key takeaways that I take from this conversation and the great insights of all five of you is really this need to invest not only in cooperation and research as AI advances in terms of what’s possible with regards to safety, but also making sure that that cooperation is across regions and jurisdictions and across stakeholder groups where transparency and responsibility is prioritized, particularly, and many of you have underlined this, as we become very, very aware in the importance of centralizing responsibility, because at the same time, responsibility drives the trust that’s missing, which will drive adoption, which will ultimately drive innovation and beneficial use for all. This has been the most enjoyable AI governance speed dating ever with the five of you. Thank you very much for your insights and for sharing your learnings about what’s working and what’s not. Chris, Yuhua, Udbav, Yachin, and Brian, I hope, as many of you underlined the importance of conversations and asking the right questions, that this is the beginning of a conversation and not the end going forward, and hope that all of you can interact with some of the audience in the room about how to take those call to actions forward with a great group at AI for good. Thank you very much, all, and have a great rest of your morning. Many thanks. Many thanks to you, Sacha, and many thanks to all your guests. I agree with you. This was a very engaging, very dynamic conversation around how we can move to practice, of course. Many thanks to all of you for your insights this morning. Thank you. And we swiftly move on to our…

Agreement points

Multi-stakeholder collaboration is essential for effective AI governance

Speakers

– Brian Tse
– Ya Qin Zhang
– Juha Heikkila
– Sasha Rubel

Arguments

Multi-signal approach, both at the national and the global level, to ensure that no one will be left behind

Enhanced global collaboration through forums and multi-stakeholder approaches essential

EU AI Act implements trust-based legislation with update mechanisms and Code of Practice for multi-stakeholder compliance

Multi-stakeholder collaboration and dialogue essential for AI governance alignment on risk identification and effective governance

Summary

All speakers agree that addressing AI governance challenges requires bringing together diverse stakeholders from different regions, sectors, and expertise areas to work collaboratively on solutions

Topics

Legal and regulatory | Development

Trust is fundamental for AI adoption and requires transparency and responsibility

Speakers

– Juha Heikkila
– Udbhav Tiwari
– Sasha Rubel

Arguments

Trust is fundamental for AI technology adoption and benefit realization, requiring legislative frameworks

Need for honest conversations about AI limitations and drawbacks, not just benefits and progress

Trust is fundamental blocker to AI adoption and requires responsibility by design and stakeholder cooperation

Summary

Speakers consistently emphasize that trust is the cornerstone of successful AI deployment, requiring transparency about limitations and proactive responsibility measures

Topics

Human rights | Legal and regulatory

Safety and responsibility must be built into AI systems from the beginning, not added later

Speakers

– Ya Qin Zhang
– Sasha Rubel

Arguments

Safety must be foundation of product design, not an afterthought, with academic-industry-policy collaboration

Need to move from responsibility and safety being ‘bolted on’ to ‘built in’ to ‘by design’ in AI development

Summary

Both speakers advocate for embedding safety and responsibility considerations from the initial design phase rather than treating them as afterthoughts

Topics

Legal and regulatory | Human rights

Need for transparency and identification of AI-generated content

Speakers

– Brian Tse
– Ya Qin Zhang

Arguments

Content provenance and watermarking essential for differentiating AI-generated from human-generated content

Registration and identification systems for AI-generated content, models, and agents should scale globally

Summary

Both speakers emphasize the importance of clearly marking and identifying AI-generated content to maintain transparency and help users distinguish between AI and human-created content

Topics

Human rights | Legal and regulatory

Similar viewpoints

Both speakers recognize that frontier AI systems pose unprecedented risks that require entirely new approaches to risk management, particularly for risks that can emerge rapidly at scale

Speakers

– Brian Tse
– Chris Meserole

Arguments

Four types of frontier AI risks: misuse, accidents/malfunctions, loss of control scenarios, and systemic societal risks

Need for new risk management instruments for risks that appear quickly at extreme scale before harms materialize

Topics

Cybersecurity | Legal and regulatory

Both advocate for international coordination to establish common standards and boundaries for AI development, emphasizing the need for global alignment on what constitutes acceptable AI behavior

Speakers

– Chris Meserole
– Brian Tse

Arguments

Need for standardizing frontier AI frameworks globally with formal standards on quick timelines

International effort required to define red lines and unacceptable outcomes from advanced AI systems

Topics

Legal and regulatory | Infrastructure

Both speakers emphasize the importance of addressing current, real-world AI harms rather than focusing solely on hypothetical future risks, and the need for ongoing monitoring and adaptation of governance measures

Speakers

– Udbhav Tiwari
– Juha Heikkila

Arguments

Real harms occurring today with AI systems affecting high-risk individuals, as demonstrated by Microsoft Recall feature

Follow-up and update mechanisms necessary for measuring effectiveness of governance measures

Topics

Human rights | Legal and regulatory

Unexpected consensus

Need for rapid development of formal standards despite traditional slow timelines

Speakers

– Chris Meserole
– Juha Heikkila

Arguments

Need for standardizing frontier AI frameworks globally with formal standards on quick timelines

Streamlining proliferating initiatives to reduce burden on developers while maintaining oversight

Explanation

Despite coming from different regulatory contexts (voluntary frameworks vs. EU legislation), both speakers recognize the tension between the need for rigorous standards and the urgency of AI development timelines, advocating for accelerated but coordinated approaches

Topics

Legal and regulatory | Infrastructure

Importance of developer agency and proactive protection measures

Speakers

– Udbhav Tiwari
– Ya Qin Zhang

Arguments

Developer agency crucial for applications to protect users from AI system risks without requiring user technical expertise

Safety must be foundation of product design, not an afterthought, with academic-industry-policy collaboration

Explanation

Unexpected alignment between a privacy-focused application developer and an academic policy expert on the need for proactive, built-in protections rather than reactive measures or user-dependent safeguards

Topics

Human rights | Cybersecurity

Overall assessment

Summary

Strong consensus on fundamental principles including multi-stakeholder collaboration, trust as prerequisite for adoption, safety by design, and transparency requirements. Speakers from different regions and sectors align on the need for international coordination while recognizing current governance gaps.

Consensus level

High level of consensus on core principles with complementary rather than conflicting approaches. This suggests a mature understanding of AI governance challenges and potential for coordinated global action, though implementation details may vary by jurisdiction and stakeholder type.

Different viewpoints

Regulatory approach – binding vs voluntary governance

Speakers

– Brian Tse
– Chris Meserole

Arguments

China’s approach includes pre-deployment registration/licensing and post-deployment transparency requirements

Frontier AI frameworks and voluntary commitments from Seoul AI Summit showing promise for identifying intolerable risks

Summary

Brian advocates for binding regulations with mandatory pre-deployment registration and licensing, citing China’s regulatory framework, while Chris emphasizes voluntary commitments and frameworks as showing the most promise for governance

Topics

Legal and regulatory

Focus on current vs future AI risks

Speakers

– Udbhav Tiwari
– Brian Tse
– Chris Meserole

Arguments

Real harms occurring today with AI systems affecting high-risk individuals, as demonstrated by Microsoft Recall feature

Four types of frontier AI risks: misuse, accidents/malfunctions, loss of control scenarios, and systemic societal risks

Need for new risk management instruments for risks that appear quickly at extreme scale before harms materialize

Summary

Udbhav emphasizes addressing immediate, real harms happening today rather than focusing on speculative future risks, while Brian and Chris focus more on frontier AI risks and potential future catastrophic scenarios

Topics

Human rights | Cybersecurity | Legal and regulatory

Unexpected differences

Transparency in AI model information

Speakers

– Udbhav Tiwari
– Chris Meserole

Arguments

Need for honest conversations about AI limitations and drawbacks, not just benefits and progress

Firms should publish risk management frameworks and systematic implementation updates for public transparency

Explanation

While both advocate for transparency, Udbhav criticizes the AI industry for becoming less transparent over time (noting regression in model card information), while Chris advocates for more systematic transparency through published frameworks. This suggests disagreement on whether current industry transparency efforts are sufficient or moving in the right direction

Topics

Human rights | Legal and regulatory

Overall assessment

Summary

The speakers show broad agreement on fundamental goals (safety, trust, international cooperation) but significant disagreement on implementation approaches, particularly regarding regulatory mechanisms (binding vs voluntary), risk priorities (current vs future), and transparency methods

Disagreement level

Moderate disagreement with high implications – while speakers share common objectives, their different approaches to governance (regulatory vs voluntary, immediate vs future-focused) could lead to fragmented global AI governance if not reconciled. The disagreements reflect deeper philosophical differences about the role of government regulation, industry self-governance, and risk prioritization in AI development

Partial agreements

Similar viewpoints

Both speakers recognize that frontier AI systems pose unprecedented risks that require entirely new approaches to risk management, particularly for risks that can emerge rapidly at scale

Speakers

– Brian Tse
– Chris Meserole

Arguments

Four types of frontier AI risks: misuse, accidents/malfunctions, loss of control scenarios, and systemic societal risks

Need for new risk management instruments for risks that appear quickly at extreme scale before harms materialize

Topics

Cybersecurity | Legal and regulatory

Both advocate for international coordination to establish common standards and boundaries for AI development, emphasizing the need for global alignment on what constitutes acceptable AI behavior

Speakers

– Chris Meserole
– Brian Tse

Arguments

Need for standardizing frontier AI frameworks globally with formal standards on quick timelines

International effort required to define red lines and unacceptable outcomes from advanced AI systems

Topics

Legal and regulatory | Infrastructure

Both speakers emphasize the importance of addressing current, real-world AI harms rather than focusing solely on hypothetical future risks, and the need for ongoing monitoring and adaptation of governance measures

Speakers

– Udbhav Tiwari
– Juha Heikkila

Arguments

Real harms occurring today with AI systems affecting high-risk individuals, as demonstrated by Microsoft Recall feature

Follow-up and update mechanisms necessary for measuring effectiveness of governance measures

Topics

Human rights | Legal and regulatory

Key takeaways

AI governance requires parallel development alongside technology advancement, not sequential implementation

Trust is fundamental for AI adoption and must be built through transparency, responsibility by design, and legislative frameworks

Four critical frontier AI risks need addressing: misuse, accidents/malfunctions, loss of control scenarios, and systemic societal risks

Multi-stakeholder collaboration across regions, jurisdictions, and stakeholder groups is essential for effective AI governance

Current governance approaches show promise but need standardization and coordination to avoid fragmentation

Safety must be foundational to AI product design rather than an afterthought

Real harms from AI systems are occurring today and require immediate attention alongside future risk prevention

Developer agency is crucial for protecting users from AI risks without requiring technical expertise from end users

Resolutions and action items

Develop international standards for frontier AI risk management frameworks within 12 months

Create global norms requiring firms to publish risk management frameworks and systematic implementation updates

Establish international effort to define red lines and unacceptable outcomes from advanced AI systems

Scale registration and identification systems for AI-generated content, models, and agents globally

Streamline proliferating AI governance initiatives to reduce developer burden while maintaining oversight

Implement follow-up and update mechanisms to measure effectiveness of governance measures

Invest in R&D for identifying critical risks, benchmarks, thresholds, and warning systems

Enable developer agency for applications to protect users from AI system risks proactively

Unresolved issues

How to balance rapid technological advancement with necessary governance regulatory frameworks across different regions

Lack of consensus on what constitutes ‘intolerable risks’ and appropriate risk thresholds globally

Insufficient understanding of AI system risks due to proprietary and closed development settings

Gap between voluntary commitments and binding regulations for the most powerful AI systems

How to ensure Global South and unprotected communities have voice in defining AI safety standards

Coordination challenges between different regional approaches (EU, China, US) with varying guidelines

How to maintain honest conversations about AI limitations amid industry hype and competitive pressures

Suggested compromises

Flexible implementation approaches within governance frameworks allowing companies to choose specific mitigation methods while meeting compliance standards

Adaptive regulatory mechanisms that can be updated more quickly than full legislative revisions

Combination of voluntary industry commitments with binding regulations for highest-risk AI systems

Multi-stakeholder processes that balance industry expertise with civil society and academic input

Presumption of compliance for companies that sign up to codes of practice while maintaining oversight mechanisms

State-of-the-art references in governance frameworks that can be interpreted flexibly as technology evolves

Drawing from the International AI Safety Report and some of the latest research, I think there are at least four types of frontier AI risk. Number one is misuse… Number two is accidents and malfunctions… Number three is potential loss of control scenarios… The fourth one is systemic risk at the societal level.

Speaker

Brian Tse

Reason

This comment provided a comprehensive taxonomic framework for understanding AI risks, moving beyond vague concerns to specific, actionable categories. It was particularly insightful because it connected theoretical risks to real-world examples (like Beijing’s medical prescription regulations) and emphasized the precautionary principle for catastrophic risks even when probabilities are unknown.

Impact

This foundational comment set the analytical framework for the entire discussion. It established a common vocabulary and structure that other panelists referenced throughout, with Chris Meserole specifically building on these risk categories when discussing frontier AI frameworks, and other speakers addressing different aspects of these four risk types.

Quite recently, Signal rolled out a feature on its desktop installations, where if you run Windows 11 and you try to take a screenshot… The reason we deployed that feature was because Microsoft had deployed Recall, a feature that takes periodic screenshots of your screen… If you look at signal as a player in this market, AI regulation has existed for some time, voluntary efforts have existed for quite some time, but yet Microsoft could deploy a lot of technology to prevent that from happening.

Speaker

Udbhav Tiwari

Reason

This concrete example was exceptionally thought-provoking because it demonstrated the real-world inadequacy of existing governance approaches. It showed how even well-intentioned AI features can create privacy risks that require defensive action by other companies, highlighting the gap between regulatory frameworks and practical implementation.

Impact

This comment fundamentally shifted the discussion from theoretical governance models to practical, immediate challenges. It introduced the concept of ‘developer agency’ as a necessary governance mechanism and emphasized focusing on current harms rather than speculative future risks. Sasha explicitly praised this as a ‘call to action’ that resonated with the audience.

I think ultimately acknowledging that as much as we talk about advanced AI, it is just as important to recognize the very real harms that are occurring to real people with very real consequences that are taking place today because of these AI systems. And unless we choose to address them rather than wondering whether AI will control Skynet three years from today, I think we’re going to end up in a scenario where some very real harms will occur.

Speaker

Udbhav Tiwari

Reason

This comment was particularly insightful because it challenged the panel’s focus on future frontier AI risks by emphasizing present-day harms. It introduced a critical tension between preparing for hypothetical catastrophic risks versus addressing immediate, tangible problems affecting real users today.

Impact

This comment created a notable shift in the discussion’s temporal focus. It influenced subsequent speakers to balance future-oriented governance with present-day concerns, and Sasha’s enthusiastic response (‘I love that call to action’) indicated it resonated strongly with the audience and moderator.

Given the profound impact from front AI systems in the coming years, it’s quite striking that we have more regulations on the food safety of dumplings than they are today.

Speaker

Brian Tse

Reason

This analogy was brilliantly thought-provoking because it used a mundane comparison to highlight the regulatory gap in AI governance. The contrast between extensive food safety regulations and minimal AI oversight effectively illustrated the urgency of the governance challenge.

Impact

This vivid comparison provided a memorable frame of reference that underscored the need for binding regulations. It supported Brian’s subsequent recommendations for pre-deployment registration and post-deployment transparency, making abstract regulatory concepts more relatable and urgent.

For trust, which has been identified here as one of the key issues, it is actually important to know how and if and how, and to what extent, whatever measures and recommendations are there, were put into practice and how effective they are. So I think that this is one aspect that is important… it would be required also then for other sort of softer approaches.

Speaker

Juha Heikkila

Reason

This comment was insightful because it identified a critical gap in AI governance: the lack of systematic follow-up and effectiveness measurement. It highlighted that having governance frameworks is insufficient without mechanisms to verify their implementation and impact.

Impact

This observation influenced the discussion’s focus on accountability and transparency. It connected to Chris Meserole’s later emphasis on publishing systematic updates and influenced the final recommendations around monitoring and reporting mechanisms.

Overall assessment

These key comments fundamentally shaped the discussion by establishing both the theoretical framework and practical urgency of AI governance challenges. Brian’s risk taxonomy provided the analytical foundation, while Udbhav’s Signal example grounded the conversation in immediate, tangible concerns. The interplay between these perspectives created a productive tension between preparing for future risks and addressing present harms. Juha’s emphasis on follow-up mechanisms added a crucial accountability dimension that influenced the final recommendations. Together, these comments moved the discussion from abstract principles to concrete, actionable governance approaches, with a notable shift toward emphasizing transparency, developer responsibility, and the need for mechanisms that can keep pace with technological development while addressing real-world impacts on users and communities.

How can we develop robust mechanisms to ensure that digital superintelligence remains under human control?

Speaker

Brian Tse

Explanation

This addresses the critical challenge of maintaining human oversight over advanced AI systems that may have tendencies to deceive humans or evade oversight, which is essential for preventing loss of control scenarios

How can we create formal standards for frontier AI risk management frameworks on a very quick timeline?

Speaker

Chris Meserole

Explanation

Current standards like ISO SC402001 take 4-5 years to develop, but the rapid pace of AI development requires standards with similar rigor but much shorter development timelines

How can we better understand AI systems and their risks when development happens in proprietary and closed settings?

Speaker

Udbhav Tiwari

Explanation

The lack of transparency in AI development makes it difficult to understand risks and consequences, which is essential for effective governance and risk management

How can we streamline the proliferation of AI governance initiatives to reduce burden on developers?

Speaker

Juha Heikkila

Explanation

Multiple overlapping initiatives create excessive monitoring and reporting requirements that burden developers, requiring coordination to make governance more efficient

How can we develop international red lines and unacceptable outcomes for advanced AI systems?

Speaker

Brian Tse

Explanation

Defining global standards for what constitutes unacceptable AI outcomes is crucial for preventing systemic risks that could affect communities worldwide

How can we invest in R&D to identify critical and catastrophic risks and develop appropriate benchmarks, thresholds, and warning systems?

Speaker

Ya Qin Zhang

Explanation

Better risk identification and early warning systems are needed to stay ahead of rapidly advancing AI technology and implement effective governance measures

How can we establish norms for developer agency to protect users from AI system risks?

Speaker

Udbhav Tiwari

Explanation

Application providers need the ability to make protective decisions for users without requiring users to navigate complex settings, especially as AI systems change the relationship between operating systems and applications

How can we scale AI content identification and registration systems globally?

Speaker

Ya Qin Zhang

Explanation

Registration and identification of AI-generated content, models, and agents is important for transparency and accountability, and successful implementations need to be scaled internationally

How can we create space for honest conversations about AI limitations and drawbacks alongside benefits?

Speaker

Udbhav Tiwari

Explanation

The current environment discourages discussion of AI limitations due to industry hype, but honest assessment of both benefits and risks is essential for effective governance