Advancing Scientific AI with Safety Ethics and Responsibility

20 Feb 2026 11:00h - 12:00h

Session at a glance

Summary

This discussion focused on AI governance and safety challenges in scientific research, particularly in biosecurity and biological applications, with emphasis on the unique needs of emerging economies and the Global South. The panelists examined whether AI safety should be approached as a governance problem, model design issue, or verification and compliance challenge.

Speaker 1, a biosecurity expert, highlighted how AI biodesign tools are fundamentally changing risk landscapes in life sciences by decoupling capabilities from traditional physical containment measures. They emphasized that over 1,500 biodesign tools now exist, shifting risks upstream to the design phase and requiring more decentralized oversight mechanisms rather than centralized authority-based approaches. The speaker stressed the need for adaptive oversight systems and capacity building in countries like India with vibrant but uneven scientific ecosystems.

Speaker 2 addressed the tension between open science and security, advocating for tiered access and contextual norms rather than blanket restrictions. They supported pre-deployment assessments with structured rubrics and emphasized that differentiated governance at capability levels is preferable to blanket access restrictions. The speaker warned against conflating open source development with danger, noting its importance for lower-resource settings.

Speaker 3 discussed institutional gaps, particularly AI readiness disparities between countries and the failure of Western-trained models when evaluated against Southeast Asian safety benchmarks. They emphasized the need for socio-cultural evaluations tailored to specific deployment environments and highlighted India’s initiatives including AI safety sandboxes and a planned Global South network for trustworthy AI.

The panelists agreed that safety measures must be systemic rather than purely technical, requiring integration of existing biosafety systems with AI evaluation frameworks. They emphasized the importance of cross-border collaboration, standardized data sharing protocols, and capacity building to address the global nature of these risks while respecting regional differences and resource constraints.

Keypoints

Major Discussion Points:

– Shifting Risk Landscape in Life Sciences: The discussion highlighted how AI biodesign tools and LLMs are fundamentally changing biosecurity risks by decoupling them from traditional physical containment measures. With over 1,500 biodesign tools available, risks are moving upstream to the design phase, requiring new governance approaches beyond traditional lab-based oversight.

– Balancing Open Science with Security: Panelists explored the challenge of preserving open science benefits while preventing dangerous capability diffusion. The consensus favored tiered access systems, contextual norms, and differentiated governance at capability levels rather than blanket restrictions, emphasizing that open source tools remain essential for lower-resource settings.

– Global South Perspectives and Adaptation: A significant focus was placed on how emerging scientific powers can shape AI governance rather than simply inheriting Western frameworks. The discussion emphasized the need for tailored approaches that consider local contexts, resources, and socio-cultural factors, with India’s sandbox strategies and incident reporting mechanisms cited as examples.

– Institutional Gaps and Capacity Building: The conversation identified critical gaps in AI readiness, technical capabilities, and regulatory coordination, particularly the need for cross-trained professionals in AI biosafety, decentralized oversight mechanisms, and integration of AI evaluation into existing biosafety systems.

– Systemic vs. Model-Centric Evaluation: Panelists argued for expanding beyond model-level assessments to broader socio-technical readiness measures, including capability uplift evaluation, incentive structures, cross-border risk diffusion, and integration with existing biosafety systems.

Overall Purpose:

The discussion aimed to explore governance frameworks for AI systems in high-risk scientific domains, particularly biosecurity, with a focus on how emerging scientific powers can develop appropriate oversight mechanisms while balancing innovation with safety concerns.

Overall Tone:

The discussion maintained a collaborative and constructive tone throughout, characterized by technical expertise and policy-oriented pragmatism. Panelists demonstrated mutual respect and built upon each other’s points, creating a nuanced conversation that avoided polarized positions. The tone remained consistently forward-looking and solution-oriented, with speakers acknowledging complexities while proposing concrete frameworks and approaches. The audience Q&A section maintained this engaged, scholarly atmosphere with thoughtful questions that extended the core themes.

Speakers

Speakers from the provided list:

– Moderator (Shyam): Session moderator facilitating the discussion on AI governance and biosecurity

– Speaker 1 (Suryesh): Biosecurity expert, works in the field of biosecurity and has experience in disarmament

– Speaker 2 (P.T.): AI safety researcher with expertise in biosecurity and AI-enabled biological tools, associated with RAND Europe research

– Speaker 3 (Geeta): Policy researcher at IIT Madras, works at CIRI (likely a research institute), focuses on AI governance, trustworthy AI, and AI safety evaluation frameworks

– Audience Member 1: Researcher in safety of AI at the University of York, focuses on psychological harms of AI

– Audience Member 2:

– Audience Member 3:

Additional speakers:

None identified beyond those in the provided speakers names list.

Full session report

This panel discussion examined the complex challenges of governing artificial intelligence systems in scientific research contexts, with particular emphasis on biosecurity applications and the unique needs of emerging economies in the Global South. The conversation brought together experts from biosecurity, AI safety, and policy backgrounds to explore whether AI governance should be approached as a data governance problem, model design issue, or verification and compliance challenge.

Fundamental Paradigm Shifts in Scientific Risk Management

The discussion opened with a crucial observation from Speaker 1, a biosecurity expert who explicitly noted they are “not an AI or AI safety expert” but work “in biosecurity.” They highlighted a fundamental transformation occurring in life sciences research, where traditional risk governance mechanisms have historically relied on physical infrastructure controls—laboratory facility inspections, material transfer protocols, and containment measures. However, the emergence of AI biodesign tools and large language models is fundamentally altering this landscape. Speaker 1 mentioned that “Rand also did a study on this, but there are more than probably 1,500 biodesign tools,” noting that these capabilities are becoming “partly decoupled from the physical containment measures which were usually used in the life sciences.”

This decoupling represents a critical paradigm shift where risks are moving “upstream to the design side” of scientific research. AI systems can now facilitate protein engineering, DNA sequence optimisation, and pathogen-host interaction modelling with unprecedented ease and accessibility. However, Speaker 1 emphasized an important limitation: the “digital to physical barrier” still exists, noting that “even if you have everything, you still can’t just develop, modify viruses without having a proper physical infrastructure and there are still some ways to control that.”

The implications extend beyond major research institutions to include “DIY kind of science” and “small-scale commercial activities which are not fully under the oversight mechanism.” As Speaker 1 emphasized, countries like India with “vibrant but uneven” scientific ecosystems require new approaches that move beyond centralized authority models, where “one authority sitting somewhere in Delhi trying to do everything” becomes inadequate when risks can emerge from distributed digital activities.

Technical Capabilities and Institutional Gaps

The conversation revealed significant gaps in current evaluation and oversight capabilities. Speaker 2, working with RAND Europe, cited research by “this organization called SECURE.Bio” showing that “ChachiPT-03” (likely referring to ChatGPT-4) “outperformed expert virologists by 94% at troubleshooting wet lab protocols.” This finding illustrates how rapidly AI capabilities are advancing and highlights the inadequacy of traditional oversight mechanisms designed for human-scale activities.

The fundamental differences between biological and nuclear security paradigms were explored in depth. Unlike nuclear materials, biological materials are “diffused, dual-use by nature, and nearly impossible to trace.” This distinction means that governance frameworks developed for nuclear oversight cannot be directly applied to biological AI systems.

Speaker 2 recommended implementing “six-monthly ritual” and “refresh cadence” for monitoring and assessment of risk on a continuous basis, using AI automation tools to increase efficiency. They proposed establishing an “AI safety or security institute model” with “formal relationship with the government” and “anchoring around biological weapons convention or the WHO.” However, they noted that such institutions would require “very significant investment from the government at multilateral level” to be effective.

The discussion also addressed the critical gap between AI governance and biosecurity communities. As Speaker 2 observed, “people in AI governance frameworks often think of biosurveillance as a niche edge case, and people in biosecurity frameworks think that AI governance is like a tool. These people don’t talk to each other. And that gap right there is where the risk happens.”

Balancing Open Science with Security Imperatives

A central tension explored throughout the discussion was how to preserve the benefits of open scientific collaboration whilst preventing dangerous diffusion of capabilities. Speaker 2 advocated for “tiered access and contextual norms” rather than binary restrictions, emphasizing that “once released, the danger is already out there” and becomes difficult to withdraw.

The discussion revealed a sophisticated understanding of this balance, with Speaker 2 arguing that “differentiated governance at capability level is always better than blanket restriction at access level.” They warned against conflating open source development with danger, noting that open source tools remain “necessary for lower resource settings” and are critical for innovation in developing countries.

Speaker 1 mentioned how “CEPI has developed this platform” where “agentic AI is being used to check if there is someone who is trying to jailbreak or someone who is trying to misuse the tool,” representing an innovative approach to using AI systems themselves for safety monitoring.

Global South Perspectives and Institutional Innovation

Speaker 3, from the Centre for Responsible AI (CRI) at IIT Madras, brought crucial insights about the inadequacy of Western-developed frameworks when applied to different geographical and cultural contexts. Citing recent research on Southeast Asian safety benchmarks, they revealed that “all these leading large language models have failed when they evaluated for more than 20 to 30 percent of the risk” in biological settings specific to Southeast Asian contexts.

The discussion highlighted significant disparities in AI readiness across regions, with India ranking third globally whilst countries like Indonesia rank around 49th. Speaker 3 emphasized the need for “participatory approaches” that consider “socio-cultural aspects that are relevant to the deployment environments” and bring end users into the requirements definition process from the outset.

India’s emerging leadership was highlighted through several concrete initiatives. The country is developing AI safety sandboxes for various sectors and creating frameworks that other developing countries can learn from. Speaker 3 described plans for a “Global South network for trustworthy AI” and mentioned the “techno-legal framework and guidelines that was recently published by METI.” They also referenced work with “CMC Wellure Hospital” on understanding healthcare worker perceptions.

Speaker 3 described CRI’s work on developing incident reporting frameworks that capture harms experienced by marginalized communities—impacts that “may be indirect but will never be recorded everywhere.” They also mentioned developing “small language models which will enable edge deployments at the low resource settings” as an alternative to large centralized models.

Systemic Versus Model-Centric Evaluation Approaches

A recurring theme was the inadequacy of purely technical, model-centric evaluation approaches. Speaker 1 argued that “policy evaluation must expand from model-centric assessment to socio-technical assessment,” warning that current approaches risk “auditing algorithms while ignoring the institutions that operationalise them.”

The heterogeneity of scientific ecosystems presents unique challenges for implementing safety measures. As Speaker 1 noted, “governance capacity, compliance culture and technical expertise varies widely in Indian institutions.” Simply importing safety frameworks developed in well-resourced Western institutions may result in measures that are “more performative than functional.”

The discussion emphasized the need for “proportionate capability-aware safeguards” that match specific institutional contexts. Speaker 1 highlighted limited awareness about AI safety among scientists themselves, noting that whilst there is some awareness of privacy issues, “safety and security is still a big gap in understanding of even the scientific experts.”

Cross-Border Coordination and Interoperability Challenges

Speaker 2 highlighted how countries are deploying AI-enabled biosurveillance systems using “incompatible data standards with very different legal regimes across borders.” The COVID-19 pandemic demonstrated how “data hoarding and incompatible reporting actually cost lives,” particularly affecting lower-resource settings.

Speaker 2 proposed three key elements for addressing these challenges: data standards harmonization through federated interoperability frameworks (mentioning “HL7FHIR” as an example of “federated healthcare interoperability resources”); legal safe harbors for cross-border data sharing during public health emergencies; and shared evaluation criteria embedded into national surveillance systems that account for different contexts.

An audience member raised important questions about “temporal modality” and how AI model performance degrades over time as data distributions change, highlighting additional implementation challenges that governance frameworks must address.

Implementation Challenges and Future Directions

The conversation acknowledged that traditional review mechanisms—periodic, paper-based, facility-centric measures—are “very much outdated in the era of AI.” New approaches must be “far more adaptive and quick” whilst maintaining rigorous safety standards.

The AI Impact Summit was highlighted as an example of forums needed to “jumpstart conversations and break the silos” between AI governance and domain-specific safety communities. Several concrete initiatives were identified as promising directions, including India’s AI safety sandboxes, the planned Global South network for trustworthy AI, and the integration of AI evaluation into existing biosafety systems rather than creating parallel structures.

The panel ultimately argued for a “web of prevention” approach where multiple complementary measures work together rather than relying on single solutions. The consensus emerged that whilst technical safeguards remain important, the most critical challenges lie in developing institutional capacity, fostering international cooperation, and ensuring that governance frameworks serve diverse global communities rather than simply protecting the interests of technologically advanced nations.

The discussion highlighted both the urgency of addressing AI governance challenges in scientific research and the complexity of developing effective, equitable solutions that account for the distributed nature of modern scientific research and the varying capabilities of institutions worldwide.

Session transcript

Moderator

Key area should we think about it as a governance data governance problem, problem in model design or should it be more on a verification or compliance angle.

Speaker 1

Thanks thank you very much Shyam for having me and good morning to everyone and welcome to this session. So I think okay let me maybe just start with saying that I’m not an AI or AI safety expert so whatever I say take it with a pinch of salt. My work is in biosecurity and that’s the angle I’ll come from. I think all of those things whether it’s a model evaluation and other things those are there and those are very very important factors and that those are the things that we need to keep in mind. But on top of that there is also a very important deep structural change that is happening. For example in the field of life sciences historically whatever risk and risk governance things that we had were very much linked to the physical infrastructure and lab facilities and facility inspection and material transfer control and things like that.

But that seems to have changed and seems to be changing very rapidly now with the kind of AI biodesign tools as well as LLMs that are emerging. So I think Rand also did a study on this, but there are more than probably 1 ,500 biodesign tools that are out there, and those are totally transforming how life sciences, but in general, science is done. Now, what kind of change that we are seeing is with these capabilities, now it’s much easier to engineer proteins, optimize DNA sequences to do things that we want, have better pathogen host modeling, interaction modeling, and things like that. Now, these capabilities are… because of AI becoming partly decoupled from the physical containment measures which were usually used in the life sciences.

So we have a lot of this risk landscape shifting a little bit more upstream to the design side when it comes to at least biological side of things. So yes, data governance, things matter. Model evaluation and red teamings are essential and we should be doing that. But also it is very important that especially for a country like India where we have a very vibrant scientific ecosystem but that is also very uneven. How we can use this AI -enabled science which is rapidly evolving into the existing mechanisms to some extent but also at the same time develop those capabilities, have more people with the core capabilities and more people with the core capabilities and more people with the core capabilities chemical security, AI nuclear security, and things like that.

So we need to train more people on those things. So integrating, again, going back to the life sciences, so integrating AI evaluation into biosafety system, strengthening the institutional readiness. Some places there are information, some labs and some institutions have information security labs or information security offices. How we can get them better prepared for these new emerging risks that are coming due to AI. Some places they have biosafety officers or biosecurity officers. How we can enable them better to address the AI risk is what the direction that we need to move towards. And have more adaptive oversight mechanism that is not only based on the, limited to this once in a while inspection that happens, but that goes more with the rapidly evolving things that we are seeing with the AI models coming up.

And I think, I think, So, just in terms of paradigm change that we are seeing and that you mentioned, is that there need to be more decentralized checks and balances and oversight mechanisms. If there is one authority sitting somewhere in Delhi and trying to do everything, that’s not going to work. So that is one of the things that we have to collectively think about. How do we decentralize these kind of oversight systems to some extent? For example, as I was saying, how we can empower the information security or biosecurity offices and create what in the field of disarmament where I have worked on called way of prevention. One measure is not enough. It’s not sufficient.

You need to have a number of measures in place which collectively can help prevent something bad from happening. Thank you.

Moderator

Thank you. That’s very insightful. And I think we’ve already touched on some areas that, you know, that would be follow -up questions. P.T., focusing a bit more on… open science where high risk domains, especially in biological data and AI capabilities, as Surya was mentioning. How do we preserve the benefits of open science while preventing the destabilizing diffusion of capabilities that we were just discussing about?

Speaker 2

Thank you. Thank you for having me today. So I guess like I would love to be able to give like a binary yes or no answer. Right. I think we all want to have that. But unfortunately, that’s not quite the case. So we need to find a way to balance the openness and also the restrictions as well. So I guess my answer here would be sort of like a tiered access and contextual norms. I think those are really important. And I think RAN Europe has done a really great job at establishing the global risk index on AI enabled biological tools. And also just generally looking into AI safety in general, where they do this thing where they call the pre -deployment assessment.

with structured rubrics. And I’m a huge fan of that because I think that when you release very frontier models and frontier tools, the danger is already out there once released. It’s really hard to withdraw the danger. But however, prevention, right? There’s this window before you release where you can do a pre -deployment assessment. So I think I’m a really huge fan of that and also the same way that I’m a big fan of KYC, know your customers. And I guess this principle also pretty much applies whereas in the case of biosecurity, where we differentially allow the development of medical countermeasures and also the defensive measures that is necessary for the research, but also don’t limit the researchers from actually innovating either.

And I guess my point here is that we’re not going to be able to do that. The non -safeguarded access, like private access to credential researchers where necessary for like defensive research is absolutely necessary. And then, you know, like open source tools, it’s necessary. Like we can’t turn away from being open source. Like any governance structure that conflates open source with danger makes a huge mistake because that also is a very critical development point, especially for lower resource settings. So we cannot afford to conflate that altogether. So I guess a very long way to answer this and then to summarize my answer is that differentiated governance at capability level is always better than blanket restriction at access level.

Yeah. Moderator

I think that’s a very structured answer and I think, you know, there’s a start of a very valid framework level conversation that’s already happening there. Geetha, turning to you, thinking more about institutional gaps in enabling some of the solutions that we are discussing, potential solutions, what are the most immediate gaps that you see in evaluating systems, technical capability, regulatory and coordination, largely from the policy angle that you work in?

Speaker 3

Thank you, Shyam. Good morning, everyone. So on the technical capabilities, right, the most fundamental thing I see is the AI readiness aspect of deployment. So in general, when we see India stands or ranks third globally, and when we see the Southeast Asian countries, I think Indonesia is around 49, and so there we see the gap, right? So whatever we do from the Western context or in the Indian context can never be catered to the AI readiness aspect of deployment. So I think it’s important to the needs, the unique needs of the Southeast. Asian countries and moreover what there is the end user perception where we see that we have to build lot of capacity for creating awareness among the end users who are actually going to use the products and from the policy perspective I would like to give you certain aspects where we think about the socio -cultural aspects that is relevant to the deployment environments.

So in general the large language models are usually trained on the western data and the very recent research work maybe I will cover a bit of both tech and policy here. So there is a Southeast Asia related benchmark, safety benchmark which says that all these leading large language models have failed when they evaluated for more than 20 to 30 percent of the risk. So in the biological settings so which means that we did not have enough safeguards which will protect people from encountering all these risks. And moreover, so this lets us know that we have to build in more sociocultural evaluations and assessments which will cater to the harms that is more particular to that particular deployment environment rather than just having a high level evaluation strategies.

And this cannot come just from the policy side, right? So we need to bring in all the participatory approach which will bring in the end users, the different stakeholders involved in using all these AI systems, be it model, right from the requirements definition, right? So when we assess whether we need an AI system or not, generally now there is a perception saying that for whatever we are going to build or the problem that we are going to solve, by default we assume. We assume that we need a large language model which will not care. which is not even possible to have it deployed in a low resource setting, right? So we need to think about small language models which will enable edge deployments at the low resource settings and also consider all the multicultural and socio -economic diversity that exist in these regions so that your model doesn’t hallucinate, is still fair and also establish some governance and accountability frameworks which will make the developers more accountable and also because having the developers more accountable will enable them considering more safeguards, right?

And also create more awareness about the main fundamental thing is that they will be expected to document whatever testing that has been gone through. And on the policy side, there is one more aspect which is the Indian government also endorses, right? The self -regulation. voluntary commitments on managing and mitigating risk that comes out of all these AI models. So I think we have to have a unified framework which can still be adaptable to different deployment settings.

Moderator

I think we are already getting a diversity of perspectives here and it is very useful to hear. Moving ahead and thinking about institutionalizing these kind of capabilities in scientific AI context, PT turning to you. Should independent evaluation and red teaming of AI systems from a technical kind of solution perspective for this problem that generate biological outputs, especially thinking biosecurity, given your perspective on this, should it become a norm and part of the global scientific specialist infrastructure? And if so, how would we go about that?

Speaker 2

I think we have to have a clear understanding of the role of the AI system and how it can And I think that is a key point. And I think that is a key point. And I think that is a key point. And I think that is a key point. And I think that is a key point. And I think that is a key point. And I think that is a key point. And I think that is a key point. So I guess a good example to use here is probably we’re thinking of nuclear weapons, right? Which falls under this organization called the International Atomic Energy Agency, the IAEA. Now, from my perspective, I think fissile materials, correct me if I’m wrong, they’re very scarce.

And they are, to a certain degree, technically trackable. And they are also, more than anything else, highly regulated. Whereas biology, on the other hand, is everything but that. It’s diffused, it’s dual -use by nature, and it’s also nearly impossible to trace. And also, most importantly, commercially available, right? And so in the recent study, actually, this was done by this organization called SECURE. Bio, where they actually tested frontier large language models against expert virologists. And it turns out that ChachiPT -03 actually outperformed expert virologists by 94 % at troubleshooting wet lab protocols. So that’s a very shocking number, right? And then, I mean, obviously you mentioned earlier that there’s a very concentrated effort that is happening between the US, UK, and China, like the global superpowers, basically.

And I guess there’s, we, in the recommendation from the RAND Europe that I was, you know, helping out with is that we recommended that governments and also independent researchers do this six -monthly ritual of monitoring and also assessment of risk on a continuous basis. And we also suggested, obviously, like using AI as an automation tool to increase the efficiency of this risk monitoring system. But I think, to your point, I think stuff like this, stuff like that is non -interactive methodology that doesn’t require, you know, researchers to actually query directly with the danger systems is actually already in and of itself a very meaningful, you know, safeguard. But that is not enough. You know, we need something that is much larger than that.

That is the integration into, like, you know, institutionalizing it. And I would argue that, like, a six -monthly, you know, ritual, that refresh cadence, for it to be delivered, it’s going to require a very significant investment from the government at multilateral level, right? And so we can’t go without any investment at all. So my suggestion would be to actually implement this AI safety or security institute model that we’ve been applying where largely… It is technically credentialed. It’s independent, but also has a very… formal relationship with the government. And something that I would caveat from the bio side is that for the institution to have some kind of anchoring around biological weapons convention or the WHO.

Because right now that relationship is not quite there yet. And I think, you know, back to my point of like pre -deployment assessment, I think that is definitely needed and then the result has to be shared then across the credential network with tiered confidentiality that rather than being kept, you know, as a proprietary to the different state. I think it’s kind of a

Moderator

That’s an interesting position, PT. Suryesh, thinking more about safety measures at large, how can we make sure that they remain rigorous and feasible within research ecosystems that you’re quite familiar with, you know, from a biosecurity angle, if you will, but largely also in the larger scientific ecosystem.

Speaker 1

Thanks Shyam. I think first, yeah first thing that we need to understand is how that ecosystem is and then see if certain measures will work there or not, right. One of the hallmarks of let’s say Indian scientific ecosystem is there is a lot of heterogeneity. There are some places which are really extremely well performing and there are other places who are not well resourced or have other all kind of challenges. So, understanding how the ecosystem is, what kind of regulation within the institutes that are there, what kind of administrative measures that are there, what kind of safety teams these kind of institutes might have, all of those things are extremely important, right. The governance capacity, compliance culture and technical expertise varies widely in Indian institutions.

And I believe this is true for many other countries in the global south as well. So it’s not something very unique. Particularly to India, we have challenges related to different kind of resources. And even when the resources are there, sometimes it’s also problematic to use them efficiently enough. Now, given that context, if we just import safety frameworks that are developed in a well -resourced place in a Western country or any developed countries, I don’t know if those would be a very good fit for the kind of system that we have here. So those might become more performative than functional to some extent. Another challenge that also P.T. mentioned to some extent is that the speed and scale of AI is huge, right?

And we need these traditional review mechanisms that institutes have for safety audits and all of those things are not going to work. We need something which is far more adaptive and quick. And also what we had traditionally is this periodic paper -based facility -centric kind of measures. And those are very much outdated in the era of AI that we live in. Now, so what… Now the question becomes, how do we design proportionate capability -aware safeguards that would be better matched for the challenges that we have? One of the major challenges, as I think a lot of us realize, is that there is limited awareness about AI safety when it comes to scientific issues, even among the scientists.

So a huge number, a large majority of scientists just don’t know what they are putting, let’s say in chat GPT might be harmful or what they are getting out of biodesign tools could be harmful to some extent. So there is some understanding about the privacy -related issues, but safety and security is still a big gap in understanding of even the scientific experts that are there. Now also regarding AI, I think there needs to be a tiered risk classification. So not everything is highly risky. There are certain biodesign tools, for example, that are trained in… in virus data. Those we’ll put in a higher risk category compared to something which is just working, let’s say, on certain animals which are not dangerous.

Now, also the safety measures, as I was mentioning earlier, as the risk has moved a bit upstream, it has come more on the design side, we should also have more safety measures moving upstream. And as Piti was mentioning that, you know, certain kind of evaluation that are before launching AI tools are necessary, but also integrating AI evaluation modules into grant review processes, creating cross -trained AI biosafety review panels, so panels specifically for AI biosafety at, from the bottom -up side, instead of having them from the top -down approach. Investing more in domestic evaluation capacity, having more AI safety institutes like Geeta’s home institute at IIT Madras. So we need a lot more of that. And lastly, I think what we have in the US and UK are these, a lot of AI safety work is being done there, right?

And as I was mentioning, importing that directly might not work. And we in the global south are largely the users and importer of this technology. So we have to see from the bottom up side, where do we put those safety measures? Do we, like when it comes to import, what kind of, when the data is being transferred, is there certain places where we can put those kind of safeguards? Also, how we can use some tech sovereignty measures in this context, right? That tech sovereignty measures are used for a number of things, but AI security is something, AI safety and security is something where those could also be used to some extent. So, yeah, I would stop here and then we can discuss.

Thank you.

Moderator

Thank you. And I think a lot of useful thoughts here for us to explore a bit more. I think we’ve… just crossed the mid mark and I’m going to use Geeta to kind of like bridge between the next two topics by combining two of your questions sorry for that so just as Surya just mentioned will the emerging scientific powers you know global south middle powers would they be able to shape governance in this context especially you know enable science or will they continue to inherit the frameworks and if they were to show leadership what would that look like in scientific AI and research ecosystems and you know you’ve already been working on some of this so I’m looking forward to kind of hearing concrete measures that you know are happening

Speaker 3

Sure. So in general what I think is definitely the emerging powers right they are putting on all efforts to bring in all the tools and frameworks that are required for governing these AI systems and for example, so India’s strategy towards all these emerging techs is that they are trying to create sandboxes which are highly essential for deploying or evaluating safety aspects for the models, right? So they do it for healthcare systems, they do it for ideology systems and whatever, right? So these type of tools and frameworks come from Indian settings will actually help the other underdeveloped countries to learn from the strategies that we use and then build something of their own or something which cannot go cross border can still happen through learning and collaboration, right?

So for example, we are going to launch a global south network for trustworthy AI and we are going to launch a global south network for trustworthy AI and we are going to launch a global south network for trustworthy AI which will enable all these mechanisms to happen, enable people to… develop and deploy AI systems which will be deployed in the low resource settings. And the other initiative which is going to give a very big leap in evaluating AI safety is coming up with an AI safety commons for the global south. That is part of the safe and trusted AI pillar that is one of the pillars in this impact summit and I think in another one or two years we will have safety commons which will help us evaluate and assess how these AI data models and systems work for different deployment settings.

Another important thing is that as Suresh mentioned about the audit frameworks. So when we come with, when we focus on the kind of risk and audit mechanisms that we have here, we still have it from an organization perspective and not from the end user perspective. So at CRI, we have come up with an incident reporting mechanism and a framework that caters to the Indian settings. So it tells you how to operationalize incident, AI incident reporting in the Indian settings, which is completely different from the Western settings. And here we have to get the harms that the people experience in the marginalized communities, which will never be recorded everywhere, right? So how do we enable all these things?

So since it is all about all these CERN -based systems, right, even those things will have certain impacts to the marginalized communities, which may be an indirect impact. But how do they are knowing about such things are happening to them, right? So those kinds of gaps we should mitigate by building more awareness, creating more AI literacy. And we should also be able to provide more privacy to all these people. The final thoughts about combining all these things is that we have to bring in some kind of collaborative work between the different stakeholders who are involved in developing and deploying these systems. And the governments have already given certain prompt knowledge about how to enable all these things through the techno -legal framework and guidelines that was recently published and the AI governance guidelines.

Which was recently published by METI. So the Southeast Asian countries can learn from the developing countries like India and then have curated a more tailored approach towards their unique needs. So that is what I think. So whoever has an opportunity or a willingness to have more things that will actually help them use or leverage these technologies can learn from whatever. Learn from. the mistakes as well as the experience that the other countries have, which is now openly available through all these summits.

Moderator

That’s very useful and I’m looking forward to following up on IIT Madras’s work in this front as well. Going to Suresh for kind of the last question in this series really, should, you know, safety measures, evaluations, primarily focus, where should the focus be at the model level? And you talked about upstream quite a bit. Should there be more broader socio -technical readiness measures, misuse considerations? Where do you think it should be?

Speaker 1

And also, very importantly, how we have to also see it from the context of, you know, people doing their own thing, DIY kind of science that happens. And also, small -scale commercial activities which are not fully under the oversight mechanism of the government, right? So, considering all of these points, right, the policy evaluation must expand from model -centric assessment to socio -technical assessment. And this would include, you know, evaluating things like how much capability uplift relative to the government capacity that is there. So, government has certain capacity to manage or do oversight, but these AI tools, how are they changing that? Incentive structures, very, very important, that shape the model deployment. Also, the diffusion of risk across borders.

All of these things don’t respect national borders, right? So, how it’s going to spread. If people using VPN or other things, a number of other things that are there. So integration, lastly, the integration with existing biosafety and resource security systems as I had already mentioned. So briefly, like performance evaluation is necessary, but governance -relevant evaluation must be systemic. And otherwise, we risk auditing algorithms while ignoring the institutions that operationalize them. And that is very, very important, how we focus on that institutional level mechanisms. Thank you. Thank you.

Moderator

Piti, kind of the last structured question before we move into a bit more of an open conversation. AI becomes embedded not just in new capacities, but also existing programs like biosurveillance, public health systems. And so there’s a mix between emerging kind of scientific knowledge with more legacy, let’s call engineering knowledge as well. So. So how do we make sure that safety, evaluation, interoperability, all of that exists in this divide without fragmentation happening across the ecosystem? Because, you know, you can easily imagine everyone’s doing their own AI, you know, safety evaluation and not necessarily talking to each other.

Speaker 2

Thank you, Shyam. I think this is a very important question. And it’s also a topic that I’m really passionate about as well, which is biosurveillance. To your point, I think, you know, countries are already deploying AI -enabled biosurveillance systems that are, you know, either syndromic surveillance or it could be, you know, genomic sequencing pipelines or outbreak modeling. The countries are already doing that, but they are not building on… the unified data standards. So they’re basically building on very incompatible data standards with very different legal regimes across the borders. We’ve seen that in Southeast Asia. We’ve seen that even countries like, for example, Singapore to Malaysia, you see different legal regiments on how they monitor the data and also the biosurveillance.

And so the fragmentation risk is actually not a technical risk, I would argue, because it’s not just a technical risk, because we’ve seen COVID. I feel like if anyone is anybody saying, I think we all were a little bit traumatized by COVID. We’ve seen how data hoarding and incompatible reporting actually cost lives. And I saw that especially happening across the region in the lower resource settings. Like countries like Cambodia, for example. AI systems that are trained on non -representative data obviously are going to perform much worse. And guess what happens? When they perform worse, the region that is most affected is the region that needs the help the most. And because of that, and also that region is also the same region with the least data infrastructure.

And so I guess to sort of like answer your question and what I think we need to do, I think there are three things to be addressed here. The first one is obviously the data standards harmonization. Currently, we don’t have that. I think we would need not like a global overhead standard that enforces on every country, but more of a federated interpretability that applies frameworks that applies to different countries. So I can think of like HL7FHIR, which is the federated… healthcare interpretability resources that are attempting to address these very specific issues on clinical data, but this one would be adapted for public health surveillance. And the second point is the legal safe harbors for basically just kind of cross -border sharing of data for public health emergencies that are negotiated beforehand because, and this is important, beforehand, because if you negotiate during an outbreak, people are going to be freaking out.

People are going to be like, I’m not going to share my data to you. What are you going to do with that data? So this needs to be done beforehand. And the last point, and also the most politically challenging point, is actually to have some kind of shared evaluation criteria across the board between different countries that are embedded into the national surveillance systems. And, for example, like Singapore data infrastructure environment might not apply to countries with like different climate data or like different demographic data. So this needs to be applied into, you know, the national surveillance systems. And what I noticed, I guess like the last message is that what I noticed the AI governance framework often thinks of biosurveillance as like an edge, like a niche edge case.

And then people in biosecurity frameworks, like doing biosecurity frameworks, thinks that AI governance is like a tool. And these people don’t talk to each other. And that gap, that gap right there is where the risk happens. So, yeah, we just need to talk to each other more. That’s easier said. Yeah.

Moderator

So I think I’m just about to close with maybe five minutes or just under that for audience questions. Thank you, Justin. 10 second final thoughts on each of you from the panel. Suresh.

Speaker 1

Just wanted to very quickly, we need to also keep in mind that how AI could help solve some of these AI safety challenges. How agentic AI could be used, let’s say, when people are trying to develop vaccines. CEPI has developed this platform where agentic AI is being used to check if there is someone who is trying to jailbreak or someone who is trying to misuse the tool that is there. Second very quick point, also, with all that what I said, there is still a gap to transfer things from digital to physical, what is called digital to physical barrier. So, even if you have everything, you still can’t just develop, modify viruses without having a proper physical infrastructure and there are still some ways to control that.

Thank you.

Speaker 3

I think we should move on transforming from issues to intelligence like learning from the risk that happens and feeding it back to the model training and other assessment activities to mitigate the risk in real time so that is where we need to move towards bringing in more people into evaluations and then making it safer for people to use

Speaker 2

I guess I’ll make it quick the point that I want to make here is that Shurya should echo his point I think you’re right that we should not shoot ourselves in the foot especially for developing countries, I think it’s really important and so my message for the last message here is just kind of like while we are forging ahead in innovation and while we are innovating ahead in whatever domains of scientific domains that we’re doing we need to be conscious of the impact that we have and I think in the AI Impact Summit is one of the really good places to jumpstart that kind of conversations and break the silos. Thank you.

Moderator

Thank you everyone. I’m just going to take probably one minute to kind of summarize key points Evaluation, I see largely a systemic question, safety measure systemic question. I especially like the point on incident response not being already there. And a couple of points on the cross -border solutions and problems, we already have that. Discussion on open signs, we talked about how managed access, safeguards, and comparing government capacity to manage that versus letting it out for more DIY -oriented signs, which is a good term, I really like that. That’s a key area. And for emerging scientific powers, of course, collaboration is key. Tailored approach, that’s something that I’m again waiting to see from IT Madras as well, their contribution on this.

And some cross -border work on legal safe harbors, data standard harmonization, PT that you mentioned, really land well from this panel. I’m going to… I’m going to stop my… summary right now and you know more of this would be kind of put together in a blog at some point in the uh nearby future uh perhaps uh we can go for questions uh first uh yes please i think i can give you mine

Audience Member 1

Thank you so much for your wonderful insights i really enjoyed this session as a researcher in safety of ai at the university of york so i focus on psychological harms of ai and so what i want to ask particularly gita is um when it comes to the definition of harms and traditional safety engineering they’re catering more to physical harms and now we see the whole spectrum of harms expanding beyond that so i would love to know the work being done by karai and you in this area and and in fact enrich my research with it

Speaker 3

Yeah, sure uh so when we actually assess harms and impacts right we do we have to do it from the different two different perspectives one is on the functional side where we assess all these algorithmic risk and other stuff. From the human centric perspective, like you said, we can keep doing everything from the psychology perspective and other ethics and other stuff. So, here at CIRI, we do work on assessing bias, determining whether the model is stereotypical or not and how do we generate explanations for the high level scientific models and all. So, from the perspective of the psychological things, there is this cognitive science or cognitive capabilities of AI models which will actually enhance or degrade the capabilities of humans.

So, those things are we are trying to do some assessments from the incident perspective. So, if you go to read the incident reporting framework that we have, we have a taxonomy of risk and harms and also the impacts. So, from the kinds of harms that we have defined, we have categorized it as physical, psychological, cyber incident based harm set. And moreover, we have all the generic kinds of harms like algorithmic harms, socio -economic harms, the environmental harms and all. So, we are trying to come up with a taxonomy that will cater to the different hierarchies that will be applied to these kind of harms and impacts which will again be model specific, use case specific and the domain specific.

So, that is where we are trying to work on. And we also have a healthcare based tool, a toolkit which will enable people to actually assess the perceptions of how they treat these models, how they see whether these AI applications are helpful for them or not and then come up with some capacity building programs for different roles in which they are working on. And this has been done with CMC Wellure Hospital and we have been assessing the perceptions of healthcare workers. and then come up with a training module which will enable them to use AI models or tools more confidently rather than, say, being resistant or not relying on them for so much.

Moderator

Last, probably last quick question. Maybe keep it short on the responses as well, please. Sorry.

Audience Member 2

Hi. So my question is about, like, we are discussing all the geographical barriers, right? The modality is geography. When we change the geography, the models tend to perform poorly. Are we concerned about the temporal modality as well? When we go forward in time, the data is going to change eventually, and that is going to affect modeling. And how do we plan on, like, you know, mitigating such a problem if it arises?

Speaker 3

Yeah. So this comes under the model monitoring, the system monitoring approach, where we consider the data drifts out of distribution. So we consider the distribution aspects of the data and models. So definitely this is one of the criteria where you assess safety and evaluate the impacts of it.

Moderator

Yes, I think last question

Audience Member 3

Thank you so much for the insightful discussion, really appreciated the expertise that you’re bringing to the topic and thanks PT for bringing up COVID because my question is about that. As we learn from COVID biosecurity risk can quickly become a cross border existential threat. So what would a successful web of prevention and incident response framework look like and who are you looking up to in this space? Like who’s doing it well in this space?

Speaker 1

I can start maybe PT can add. So I think as I was mentioning, it will have to be more decentralized but at the same time integrated to the leadership. So I think there needs to be more empowerment of people who are like biosafety officers in the lab or who are institutional biosafety committee members, who are people who are working on the ethics and research security side at the institutes. So those are the people who need to be empowered. So there needs to be more capacity building of those people and at the same time there needs to be a mechanism established so they can report those incidents to the very top and there is top leadership sitting in the capitals.

They can in some way get an overview or monitor the situation as it is going on at different institutes level.

Speaker 2

Thanks. I can add a little bit to that. So in Singapore we actually have different agencies responsible for this. So we have the National Environmental Agency and then we have the MOH, obviously the Ministry of Health and then we also have different smaller agencies like Communicable Disease Agency and also like Prepare Agency where they are responsible for different tasks. But I want you to envision this as almost like the way that Singapore is trying to establish itself. I think it’s trying to establish itself almost as a firefighter. So when there’s an incident where there’s a crisis, who is actually doing what is very clear but it’s always not always clear across like different countries. For example, in Laos, Vietnam, might be looking very different, but I think having a very coordinated response across the different agencies on who is doing what.

Like, for example, National Environmental Agency is responsible for wastewater surveillance. So monitoring how the sickness is increasing or spiking or not, those are the people, yeah, that you would look up to. And I think that’s the last word, right? It all comes down to prevention and preparedness, even in this much like anything else with biocontext.

Moderator

Thank you, everyone, for the question, and thank you to my brilliant panelists, Suryesh, Geeta, and P.T. This was a very insightful discussion. On the screen is the work from RAND Europe with CLTR, some of what was referred to by P.T. and other panelists as well, some aspects of what we were discussing about risk typification. You’ll probably get some ideas there as well. And with that, I close. I’m surprised. I’m supposed to hand over these mementos to apparently including me, so let us do that now. Thank you.

Speaker 1

Speech speed

159 words per minute

Speech length

Institutionalizing Independent Evaluation and Red‑Teaming

Topics

Artificial intelligence | Monitoring and measurement

Speaker 2

Speech speed

152 words per minute

Speech length

Integration, Interoperability, and Systemic Safety Measures

Topics

Data governance | Artificial intelligence

Speaker 3

Speech speed

147 words per minute

Speech length

Broad Socio‑Technical Assessment Beyond Model‑Centric Evaluation

Topics

Artificial intelligence | Human rights and the ethical dimensions of the information society

Moderator

Speech speed

125 words per minute

Speech length

969 words

Speech time

462 seconds

Call for independent evaluation and red‑teaming

Explanation

The moderator asks whether systematic independent evaluation and red‑team activities should become a norm for AI systems that generate biological outputs.

Evidence

“Should independent evaluation and red teaming of AI systems from a technical kind of solution perspective for this problem generate biological outputs, especially thinking biosecurity, given your perspective on this, should it become a norm and part of the global scientific specialist infrastructure?” [81].

Major discussion point

Institutionalizing Independent Evaluation and Red‑Teaming

Topics

Artificial intelligence | Monitoring and measurement

Audience Member 1

Speech speed

167 words per minute

Speech length

100 words

Speech time

35 seconds

Expanded harms taxonomy

Explanation

The audience member highlights the need to broaden the harms framework beyond physical risks to include psychological, cyber, socio‑economic, and environmental impacts.

Evidence

“Thank you so much for your wonderful insights i really enjoyed this session as a researcher in safety of ai at the university of york so i focus on psychological harms of ai …” [133]. “And moreover, we have all the generic kinds of harms like algorithmic harms, socio‑economic harms, the environmental harms and all.” [131]. “So, from the kinds of harms that we have defined, we have categorized it as physical, psychological, cyber incident based harm set.” [132].

Major discussion point

Broad Socio‑Technical Assessment Beyond Model‑Centric Evaluation

Topics

Human rights and the ethical dimensions of the information society | Artificial intelligence

Audience Member 2

Speech speed

170 words per minute

Speech length

75 words

Speech time

26 seconds

Temporal data‑drift concern

Explanation

The audience member points out that data will evolve over time, potentially degrading model performance, and asks how to mitigate this issue.

Evidence

“When we go forward in time, the data is going to change eventually, and that is going to affect modeling.” [126]. “And how do we plan on, like, you know, mitigating such a problem if it arises?” [127].

Major discussion point

Integration, Interoperability, and Systemic Safety Measures

Topics

Artificial intelligence | Monitoring and measurement

Audience Member 3

Speech speed

190 words per minute

Speech length

78 words

Speech time

24 seconds

Decentralized incident response

Explanation

The audience member asks what a successful, decentralized web of prevention and incident‑response framework would look like, emphasizing the need for linked biosafety officers and central leadership.

Evidence

“So what would a successful web of prevention and incident response framework look like and who are you looking up to in this space?” [110].

Major discussion point

Governance and Oversight of AI‑Enabled Biosecurity

Topics

Building confidence and security in the use of ICTs | Artificial intelligence

Agreements

Artificial intelligence | Building confidence and security in the use of ICTs | Monitoring and measurement

Recognition of cross-border nature of AI risks requiring international coordination

Speakers

– Speaker 1
– Speaker 2
– Audience Member 3

Arguments

Safety measures should focus on socio-technical assessment rather than just model-centric evaluation

Biosurveillance systems are being deployed with incompatible data standards and different legal regimes across borders

Cross-border incident response frameworks require clear agency coordination and preparedness measures

Summary

All three recognize that AI-related risks, particularly in biosecurity contexts, cross national borders and require coordinated international responses with clear roles and responsibilities

Topics

Artificial intelligence | Building confidence and security in the use of ICTs | Social and economic development

Similar viewpoints

All speakers recognize that traditional governance approaches are inadequate for AI-related risks and that new, proactive approaches focusing on the design and pre-deployment phases are necessary

Speakers

– Speaker 1
– Speaker 2
– Speaker 3

Arguments

Risk landscape is shifting upstream to design side due to AI-enabled biodesign tools, requiring new governance approaches beyond traditional physical containment measures

Pre-deployment assessment with structured rubrics is essential before releasing frontier models and tools

AI readiness varies significantly across regions, with most large language models failing safety benchmarks for Southeast Asian contexts

Topics

Artificial intelligence | Building confidence and security in the use of ICTs | The enabling environment for digital development

Both speakers emphasize the importance of inclusive approaches that consider marginalized communities and the need for interoperable systems that can work across different contexts while protecting vulnerable populations

Speakers

– Speaker 2
– Speaker 3

Arguments

Need for federated interoperability frameworks, legal safe harbors for data sharing, and shared evaluation criteria

Incident reporting mechanisms should capture harms experienced by marginalized communities and provide real-time feedback

Topics

Artificial intelligence | Human rights and the ethical dimensions of the information society | Data governance

Both speakers advocate for developing countries to create their own governance frameworks rather than simply adopting Western models, with successful approaches serving as models for other developing nations

Speakers

– Speaker 1
– Speaker 3

Arguments

Scientific ecosystems have significant heterogeneity in resources and capabilities, requiring tailored rather than imported Western frameworks

Emerging powers like India are creating sandboxes and frameworks that can serve as learning models for other developing countries

Topics

Artificial intelligence | The enabling environment for digital development | Capacity development

Unexpected consensus

Importance of maintaining open science while managing risks

Speakers

– Speaker 1
– Speaker 2
– Speaker 3

Arguments

Risk landscape is shifting upstream to design side due to AI-enabled biodesign tools, requiring new governance approaches beyond traditional physical containment measures

Open source tools remain necessary for innovation, especially in lower resource settings, and should not be conflated with danger

Emerging powers like India are creating sandboxes and frameworks that can serve as learning models for other developing countries

Explanation

Despite coming from different perspectives (biosecurity, technical AI safety, and policy), all speakers agreed on the importance of preserving open scientific collaboration while implementing appropriate safeguards. This consensus is unexpected given the typical tension between security and openness in dual-use technology discussions

Topics

Artificial intelligence | Building confidence and security in the use of ICTs | The enabling environment for digital development

Need for Global South leadership in AI governance rather than passive adoption

Speakers

– Speaker 1
– Speaker 3
– Moderator

Arguments

Scientific ecosystems have significant heterogeneity in resources and capabilities, requiring tailored rather than imported Western frameworks

Emerging powers like India are creating sandboxes and frameworks that can serve as learning models for other developing countries

Emerging scientific powers and Global South countries need to shape AI governance rather than just inherit Western frameworks

Explanation

There was unexpected consensus across speakers from different backgrounds that developing countries should actively shape AI governance frameworks rather than simply adopting Western models. This represents a shift from traditional technology transfer models to more autonomous governance development

Topics

Artificial intelligence | The enabling environment for digital development | Closing all digital divides

Integration of AI tools for solving AI safety challenges

Speakers

– Speaker 1
– Speaker 2

Arguments

Safety measures should focus on socio-technical assessment rather than just model-centric evaluation

Six-monthly monitoring and assessment of risks should become standard practice with AI automation support

Explanation

Both speakers unexpectedly agreed on using AI itself as a tool for managing AI safety challenges, such as using agentic AI for monitoring potential misuse or automating risk assessment processes. This reflexive approach to AI governance was not anticipated but emerged as a shared perspective

Topics

Artificial intelligence | Building confidence and security in the use of ICTs | Monitoring and measurement

Overall assessment

Summary

The speakers demonstrated strong consensus on several key areas: the need for context-specific governance frameworks tailored to developing country contexts, the importance of pre-deployment assessment and proactive safety measures, the value of decentralized governance approaches with empowered local institutions, and the necessity of balancing openness with appropriate safeguards. There was also agreement on the cross-border nature of AI risks requiring international coordination and the importance of capacity building and awareness among scientists and end users.

Consensus level

High level of consensus with significant implications for AI governance policy. The agreement across speakers from different backgrounds (biosecurity, technical AI safety, and policy) suggests these principles could form the foundation for practical AI governance frameworks. The consensus on Global South leadership in developing tailored governance approaches rather than adopting Western models represents a significant shift in thinking about technology governance. The shared emphasis on maintaining scientific openness while managing risks provides a pathway for balanced policy development that doesn’t stifle innovation while addressing legitimate safety concerns.

Differences

Different viewpoints

Centralized vs. decentralized governance approaches

Speakers

– Speaker 1
– Speaker 2

Arguments

Need for decentralized oversight mechanisms with empowered local biosafety officers and institutional review panels

Independent AI safety institutes with formal government relationships and anchoring to biological weapons conventions needed

Summary

Speaker 1 advocates for decentralized checks and balances with empowered local officials, while Speaker 2 proposes centralized independent AI safety institutes with formal government relationships

Topics

Artificial intelligence | Building confidence and security in the use of ICTs | The enabling environment for digital development

Scope of safety evaluation focus

Speakers

– Speaker 1
– Speaker 3

Arguments

Safety measures should focus on socio-technical assessment rather than just model-centric evaluation

Model monitoring must address temporal data drift and out-of-distribution issues

Summary

Speaker 1 emphasizes broad socio-technical assessment including institutional factors, while Speaker 3 focuses more on technical model monitoring aspects like data drift

Topics

Artificial intelligence | Building confidence and security in the use of ICTs | Monitoring and measurement

Unexpected differences

Role of open source in AI safety

Speakers

– Speaker 1
– Speaker 2

Arguments

Scientific ecosystems have significant heterogeneity in resources and capabilities, requiring tailored rather than imported Western frameworks

Open source tools remain necessary for innovation, especially in lower resource settings, and should not be conflated with danger

Explanation

While both speakers support context-appropriate approaches, there’s an implicit tension between Speaker 1’s emphasis on institutional controls and Speaker 2’s strong advocacy for open source access, which could create implementation conflicts

Topics

Artificial intelligence | Building confidence and security in the use of ICTs | The enabling environment for digital development

Overall assessment

Summary

The speakers show broad consensus on the need for context-appropriate AI governance but differ on implementation approaches, particularly regarding centralization vs. decentralization and the balance between technical and socio-technical evaluation methods

Disagreement level

Low to moderate disagreement level – speakers share fundamental goals but propose different pathways, which could lead to complementary rather than conflicting approaches if properly coordinated

Partial agreements

All speakers agree that Western frameworks cannot be directly imported and that context-specific approaches are needed, but they differ on implementation – Speaker 1 focuses on institutional heterogeneity, Speaker 2 on capability-based differentiation, and Speaker 3 on regional sandbox approaches

Speakers

– Speaker 1
– Speaker 2
– Speaker 3

Arguments

Arguments

Limited awareness about AI safety among scientists creates gaps in understanding potential harms

Incident reporting mechanisms should capture harms experienced by marginalized communities and provide real-time feedback

Topics

Artificial intelligence | Capacity development | Human rights and the ethical dimensions of the information society

Similar viewpoints

All speakers recognize that traditional governance approaches are inadequate for AI-related risks and that new, proactive approaches focusing on the design and pre-deployment phases are necessary

Speakers

– Speaker 1
– Speaker 2
– Speaker 3

Arguments

Risk landscape is shifting upstream to design side due to AI-enabled biodesign tools, requiring new governance approaches beyond traditional physical containment measures

Pre-deployment assessment with structured rubrics is essential before releasing frontier models and tools

AI readiness varies significantly across regions, with most large language models failing safety benchmarks for Southeast Asian contexts

Topics

Artificial intelligence | Building confidence and security in the use of ICTs | The enabling environment for digital development

Both speakers emphasize the importance of inclusive approaches that consider marginalized communities and the need for interoperable systems that can work across different contexts while protecting vulnerable populations

Speakers

– Speaker 2
– Speaker 3

Arguments

Need for federated interoperability frameworks, legal safe harbors for data sharing, and shared evaluation criteria

Incident reporting mechanisms should capture harms experienced by marginalized communities and provide real-time feedback

Topics

Artificial intelligence | Human rights and the ethical dimensions of the information society | Data governance

Both speakers advocate for developing countries to create their own governance frameworks rather than simply adopting Western models, with successful approaches serving as models for other developing nations

Speakers

– Speaker 1
– Speaker 3

Arguments

Scientific ecosystems have significant heterogeneity in resources and capabilities, requiring tailored rather than imported Western frameworks

Emerging powers like India are creating sandboxes and frameworks that can serve as learning models for other developing countries

Topics

Artificial intelligence | The enabling environment for digital development | Capacity development

Takeaways

Key takeaways

AI governance in scientific research requires a shift from traditional physical containment measures to upstream design-side risk management, particularly for biodesign tools and life sciences applications

A tiered, capability-based governance approach is more effective than blanket access restrictions, with differentiated safeguards based on risk levels rather than universal prohibitions

Pre-deployment assessment with structured rubrics is essential for frontier AI models and tools, as risks become difficult to mitigate once systems are released

Scientific ecosystems, particularly in the Global South, have significant heterogeneity in resources and capabilities, requiring tailored frameworks rather than direct adoption of Western approaches

Open source AI tools remain crucial for innovation in lower-resource settings and should not be conflated with security risks, but require appropriate safeguards and credentialed access systems

Cross-border biosurveillance systems currently operate with incompatible data standards and legal regimes, creating fragmentation risks that were highlighted during COVID-19

AI safety evaluation must expand beyond model-centric assessment to include socio-technical factors, institutional capacity, and deployment environment considerations

Decentralized oversight mechanisms with empowered local biosafety officers and institutional review panels are needed, rather than centralized top-down approaches

Emerging scientific powers like India are developing frameworks (sandboxes, incident reporting systems, safety commons) that can serve as models for other developing countries

Resolutions and action items

Development of a Global South network for trustworthy AI to enable collaboration and learning between developing countries (Speaker 3)

Launch of an AI safety commons for the Global South as part of the safe and trusted AI pillar within 1-2 years (Speaker 3)

Implementation of six-monthly monitoring and assessment rituals for AI risks with government and multilateral investment (Speaker 2)

Creation of independent AI safety institutes with formal government relationships and anchoring to biological weapons conventions or WHO (Speaker 2)

Integration of AI evaluation modules into grant review processes and creation of cross-trained AI biosafety review panels (Speaker 1)

Development of federated interoperability frameworks for biosurveillance data standards across countries (Speaker 2)

Establishment of legal safe harbors for cross-border data sharing during public health emergencies, negotiated beforehand (Speaker 2)

Capacity building programs for biosafety officers, institutional review committee members, and ethics/research security personnel (Speaker 1)

Development of incident reporting mechanisms tailored to specific regional contexts, particularly capturing harms to marginalized communities (Speaker 3)

Unresolved issues

How to effectively balance innovation incentives with security concerns across diverse scientific ecosystems with varying resource levels

Specific mechanisms for enforcing or incentivizing adoption of safety frameworks across heterogeneous institutional landscapes

Technical details of how federated interoperability frameworks would work in practice across different legal and regulatory regimes

Funding and resource allocation strategies for implementing proposed safety institutes and monitoring systems, particularly in resource-constrained environments

How to address the gap between AI governance frameworks and biosecurity frameworks, where practitioners in each field don’t communicate effectively

Scalability challenges of moving from model-centric to socio-technical assessment approaches across large numbers of AI applications

Methods for addressing temporal data drift and out-of-distribution issues in AI models deployed across different time periods and contexts

Specific criteria and thresholds for risk classification systems that would work across different cultural and regulatory contexts

How to manage the digital-to-physical barrier in biological applications while maintaining appropriate oversight

Suggested compromises

Tiered access systems that provide open source tools for innovation while restricting access to highest-risk capabilities through credentialed researcher networks

Differentiated governance at capability level rather than blanket restrictions, allowing beneficial applications while controlling dangerous ones

Federated interoperability approaches rather than global overhead standards, allowing countries to maintain sovereignty while enabling cross-border coordination

Hybrid oversight models combining decentralized local capacity with centralized monitoring and reporting mechanisms

Pre-deployment assessment requirements balanced with streamlined processes to avoid stifling innovation in beneficial applications

Voluntary self-regulation frameworks with accountability measures rather than rigid top-down regulatory approaches

Shared evaluation criteria embedded in national systems rather than imposed universal standards

This comment identifies a critical structural problem in how different expert communities approach AI safety – the dangerous siloing of expertise that creates blind spots in risk assessment and management.

Impact

This observation provided a meta-commentary on the discussion itself and highlighted why interdisciplinary conversations like this panel are essential. It influenced the moderator’s closing emphasis on breaking silos and reinforced the need for collaborative approaches mentioned by other speakers.

Overall assessment

Explanation

This addresses the need to broaden safety frameworks to encompass the full spectrum of potential harms from AI systems, particularly in human-centric applications.

Disclaimer: This is not an official session record. DiploAI generates these resources from audiovisual recordings, and they are presented as-is, including potential errors. Due to logistical challenges, such as discrepancies in audio/video or transcripts, names may be misspelled. We strive for accuracy to the best of our ability.