WS #283 AI Agents: Ensuring Responsible Deployment

Summary

This discussion at the 17th Internet Governance Forum focused on agentic AI and the need for responsible governance as AI evolves from reactive tools to proactive, autonomous agents capable of independent decision-making. The panel, moderated by Jim Prendergast from Galway Strategy Group, included experts from Google, Fundación Vía Libre, Heritage Partners, and the UK government who explored both opportunities and challenges presented by this emerging technology.

Will Carter from Google defined agentic AI by two key characteristics: the ability to perform complex reasoning and take autonomous actions on behalf of users. He outlined Google’s vision of AI agents handling daily tasks like meal planning, budget management, and scheduling to reduce mental load and allow humans to focus on more meaningful activities. However, the panelists identified significant challenges requiring immediate attention, including privacy protection, security vulnerabilities, and the need for robust safeguards against misuse.

Luciana Benotti from Argentina highlighted critical misalignment issues, citing research showing that workers often don’t want the type of automation currently offered by AI agents. She presented a concerning example from Argentine schools where an AI career guidance agent exhibited bias by recommending lower-cost degree programs to low-income students, despite most universities being tuition-free. Dominique Lazanski emphasized the importance of technical standards, interoperability, and security frameworks, noting that different regulatory approaches across regions could create challenges for global deployment.

The discussion revealed that governments worldwide are still grappling with basic questions about what agentic AI actually is and how to regulate it appropriately. Participants stressed the need for user control, transparency, third-party assessments, and inclusive development processes that involve affected communities as meaningful co-designers rather than just data providers.

Keypoints

## Major Discussion Points:

– **Defining and Understanding Agentic AI**: The panel struggled with the lack of consensus on what constitutes “agentic AI,” identifying key characteristics as complex reasoning capabilities and the ability to perform autonomous actions on behalf of users, representing an evolution from reactive AI tools to proactive, decision-making agents.

– **Safeguards and User Control**: Extensive discussion on protecting privacy, safety, and autonomy through user control mechanisms, including granular data access preferences, human oversight for important decisions, security against prompt injection attacks, and transparency to enable accountability.

– **Misalignment Between AI Capabilities and Human Needs**: Significant concern raised about the disconnect between what AI agents currently offer and what workers actually want automated, citing a Stanford study showing nearly half of current AI agent tasks are undesired for automation, while high-potential tasks receive little investment.

– **Standards, Interoperability, and Governance Challenges**: Discussion of the critical need for technical standards, compatibility frameworks, and coordinated international governance approaches, with different regions (EU, UK, Asia-Pacific) developing varying regulatory frameworks that need alignment.

– **Workforce and Societal Impact**: Concerns about AI agents’ effects on employment, particularly in software engineering, questions about who benefits from automation profits, and the need for educational system changes to prepare for an agent-integrated internet.

## Overall Purpose:

The discussion aimed to explore the emerging field of agentic AI and develop collaborative approaches for responsible governance. The panel sought to understand the implications of AI systems that can act autonomously, identify key challenges and opportunities, and work toward ensuring these technologies serve humanity ethically and responsibly.

## Overall Tone:

The discussion maintained a balanced, thoughtful tone throughout, combining cautious optimism with realistic concern. Panelists demonstrated technical expertise while acknowledging significant unknowns and challenges. The tone was collaborative and educational, with participants from different sectors (industry, academia, government, civil society) sharing perspectives constructively. While there was underlying urgency about addressing governance challenges, the conversation remained measured and solution-oriented rather than alarmist, emphasizing the need for continued dialogue and multi-stakeholder cooperation.

Speakers

– **Jim Prendergast** – Galway Strategy Group, moderator/facilitator of the discussion

– **Will Carter** – Leads AI policy at Google, focuses on advancing AI innovation at Google and smart AI policy around the world, based in New Mexico, United States

– **Luciana Benotti** – Head of research in AI at Fundacion Villibre, innovates in participatory methods for AI evaluation in Latin America, computer science teacher/professor at university, joining from Argentina

– **Jayantha Fernando** – Veteran technology and digital law expert, head of technology media and telecom practice at Heritage Partners in Colombo, Sri Lanka, member of the Sri Lankan AI task force, former government official, joining from Kuala Lumpur

– **Dominique Lazanski** – Expert in standards development, negotiates for the UK government at the ITU, fellow at the University of Pittsburgh working on tech policy, consultant on tech projects globally including in Ukraine, splits time between London and Lillehammer

– **Nico Caballero** – GAG chair at ICANN (Governmental Advisory Committee)

– **Richard Wingfield** – Works for sustainability consultancy called BSR in the tech and human rights team

– **Anne McCormick** – EY, Global Head of Public Policy

**Additional speakers:**

– **Participant** – Carl Fred Kirkland, member of the tech community (speaking on behalf of himself)

# Comprehensive Report: Agentic AI and Responsible Governance

## 17th Internet Governance Forum Discussion Summary

### Introduction and Context

The 17th Internet Governance Forum hosted a discussion on agentic AI and responsible governance frameworks, moderated by Jim Prendergast from Galway Strategy Group. Prendergast noted this was his 17th IGF and reflected on how agentic AI wasn’t around at the first forum, highlighting the rapid pace of technological change. The 60-minute session brought together diverse expertise from industry, academia, government, and civil society to examine the opportunities and challenges presented by AI systems capable of autonomous decision-making.

The panel featured Will Carter, who leads AI policy at Google; Luciana Benotti, Head of research in AI at Fundación Vía Libre; Jayantha Fernando, a technology and digital law expert from Sri Lanka’s AI task force (joining late from Malaysia where he was participating in a cybercrime program); and Dominique Lazanski, an expert in standards development who negotiates for the UK government at the ITU. The session also included contributions from audience members Nico Caballero (GAG chair at ICANN), Richard Wingfield (sustainability consultant with BSR’s tech and human rights team), and Anne McCormick (EY Global Head of Public Policy).

### Defining Agentic AI: Core Characteristics

Carter outlined two key characteristics that distinguish agentic AI from previous generations: the ability to perform complex reasoning and the capacity to take autonomous actions on behalf of users. He explained that agents typically use the largest models trained with unsupervised learning, plus additional supervised training and fine-tuning based on user feedback.

When asked by Caballero about machine learning types, Carter clarified that agents integrate multiple approaches: “They use unsupervised learning from these large models, but then they also have supervised training on top of that, and then they have fine-tuning based on user feedback.” This represents a shift from reactive AI tools to proactive systems that can independently plan and execute tasks.

Carter presented examples of AI agents handling daily tasks such as meal planning, budget management, and scheduling, aimed at reducing mental load and allowing humans to focus on more meaningful activities. However, the definitional uncertainty around what constitutes agency and autonomy creates challenges for policymakers, who Carter noted primarily ask basic questions like “what are agents?” when engaging with the technology sector.

### Safeguards and Technical Challenges

Carter outlined Google’s approach to safeguards, emphasizing user control through granular data access preferences, human oversight for important decisions, security measures against prompt injection attacks, and transparency mechanisms. However, he acknowledged a fundamental paradox: “One of the defining paradigms of modern cybersecurity practices is bots are bad,” which creates challenges for legitimate AI agents that may be blocked by security systems designed to prevent malicious bot activity.

Lazanski warned that the attack surface for agentic AI will be enormous, requiring shared security practices among companies and robust technical standards. She emphasized the need for interoperability frameworks and coordinated security approaches, noting that different regulatory frameworks across regions could create vulnerabilities.

The panel identified the urgent need for authentication standards and protocols to distinguish between beneficial AI agents and harmful bots, including development of agent-to-agent protocols and model context protocols for safer interactions between AI systems.

### Misalignment Between AI Capabilities and Human Needs

Benotti presented research showing a disconnect between what AI agents currently offer and what workers actually want automated. Citing a Stanford study of 1,500 US workers across 100 occupations, she revealed that “workers do want automation, but in a large proportion, not the kind of automation that AI agents currently offer. Almost half of the tasks by the AI agents that were described there that are offered now by the AI agents were seen by workers as non-desired for automation.”

She argued that misaligned agents create unnecessary environmental burden through energy consumption while failing to address genuine human needs, raising the question: “Who is responsible for this misalignment between human needs and AI agents’ current capabilities?” The discussion highlighted examples such as workers’ reluctance to automate agenda setting while showing interest in automating scheduling coordination through natural language conversation.

### Bias and Evaluation Challenges

Benotti presented a case study from Argentine schools demonstrating how AI agents can perpetuate harmful biases. An AI career guidance agent exhibited systematic bias by recommending lower-cost degree programs to low-income students, despite most Argentine universities being tuition-free.

She explained fundamental flaws in current AI evaluation methods, noting that standard metrics like F1 and accuracy “treat all kinds of errors in the same way.” Using government energy subsidies as an example, she illustrated how “a false positive here means giving a subsidy to someone, to a family, that may not need it, so it’s an inefficiency. But a false negative, denying a subsidy to a family that depends on it, can mean kids going without power to study or to store food.”

Benotti mentioned her work on the HESEA methodology for evaluation frameworks that account for different error types and their varying social costs, emphasizing the need for domain experts to be involved early in AI design rather than providing only late-stage oversight.

### Government and Regulatory Approaches

Fernando outlined Sri Lanka’s approach, which focuses on flexible governance tools and soft law approaches designed to balance innovation with appropriate oversight. He emphasized the importance of balancing privacy protection with innovation, particularly in sensitive domains like healthcare, noting that existing data protection laws provide frameworks for handling sensitive information.

Lazanski highlighted the complexity of international coordination, contrasting the EU’s risk-based approach with specific prohibitions against the UK’s focus on innovation and adoption, and various Asia-Pacific approaches that emphasize different priorities. She noted the need for better understanding among policymakers about the technical realities and limitations of agentic AI systems.

### Privacy and Data Protection Concerns

The panel addressed technical limitations in privacy protection for agentic AI systems. Benotti argued from a technical standpoint that “there’s no way to ensure private information won’t be disclosed if it’s used in training data,” explaining that “large language models can inadvertently disclose private training data” and that “complete anonymization of natural language data is technically impossible.”

Fernando noted that healthcare data requires special protection under existing data protection laws, though questions from participants about the security of AI agents handling sensitive healthcare information revealed ongoing uncertainty about whether current technical safeguards are adequate for high-stakes domains.

### Workforce Impact and Human Agency

The panel addressed concerns about agentic AI’s impact on employment, particularly in software engineering. Benotti noted that AI agents are increasingly used by software developers and raised concerns about developers not understanding AI-generated code due to time pressures, which could create long-term risks for software quality and security.

Wingfield challenged Carter’s framing of tasks like financial management as routine, arguing that “things like financial management, thinking about your children’s health and well-being, being able to critically assess options and make a decision are all quite important things that we do as humans.” Carter responded by emphasizing that agents should handle routine execution while preserving human strategic thinking and value-based decisions.

### Third-Party Assessment and Accountability

McCormick highlighted growing market demand for third-party verification and public assessments of AI systems, mentioning that EY would be publishing something on third-party assessment “next week.” She suggested that accountability mechanisms are becoming commercial necessities rather than merely regulatory requirements.

Benotti challenged the panel to consider who should conduct such assessments, asking: “How can we credibly work towards a future where this third party is built by the most affected, the workers, the teachers, the students… When they are meaningful co-designers of the AI agents that affect them and that are involved also in analyzing the environmental impact with metrics that they can understand and not just use them as raw data providers.”

### Technical Infrastructure and Standards

Carter outlined the need for new protocols and standards for what he termed the “agentic web,” noting that agents interact with systems differently than humans do. The technical complexity of agentic AI systems requires sophisticated policy enforcement mechanisms and robust technical standards.

Lazanski stressed the importance of interoperability and technical standards development, noting that the lack of common frameworks creates challenges for both security and functionality. The discussion highlighted the need for international coordination on technical standards to ensure that agentic AI systems can operate safely across different jurisdictions.

### Key Recommendations

The discussion generated several concrete recommendations:

– Development of soft law approaches with policy guidance rather than rigid regulatory frameworks

– Risk-based approaches that apply different levels of oversight based on potential impact

– Continued development of authentication standards for AI agents

– Better evaluation metrics that account for different error types and social costs

– Investment in agent-to-agent protocols for safer interactions between AI systems

– Enhanced education and relationship building between policymakers and stakeholders

### Conclusion

As the session reached its time limit (with Prendergast noting the final 10 minutes), the discussion revealed both the promise and challenges of agentic AI governance. While technical capabilities continue to advance rapidly, the governance frameworks necessary for responsible deployment require sustained collaboration between industry, academia, government, and civil society.

The conversation highlighted the importance of involving affected communities as meaningful participants in AI development rather than merely as sources of data or feedback. Key unresolved issues include the lack of consensus definition for agentic AI, questions about international harmonization of governance frameworks, and the need for better alignment between AI capabilities and genuine human needs.

The panel demonstrated that constructive dialogue between diverse stakeholders can identify shared priorities and potential pathways forward, even as significant technical and policy challenges remain to be addressed in ensuring agentic AI serves humanity responsibly.

Jim Prendergast: You have not signed into the Zoom room, I recommend and ask that you do, just we do have some online participants and we hope that it helps with the facilitation and discussion between in person and online participants as well. And if you’re looking for the audio, it’s on channel one of the headset. My name is Jim Prendergast, I’m with the Galway Strategy Group. And as I was preparing for this, I realized this is my 17th IGF, which is a little frightening but exciting because what we’re going to talk about is one of those issues that wasn’t around at the first one that I attended, but rapidly has appeared on the horizon as the IGFs have progressed. So the topic we’re diving into today is not just timely, but it’s absolutely critical. It’s agentic AI and the need for responsible governance. We’re witnessing a pivotal moment where AI is evolving beyond reactive tools to become proactive, autonomous agents capable of independent decision making and action. This presents both incredible opportunities, but also some challenges that demand immediate and thoughtful attention. And that’s what we want to do today. Our goal is to explore these new frontiers, understand the implications of this powerful technology, and sort of collaboratively work towards a future where agentic AI serves humanity responsibly and ethically. So let’s get started. Where is my clicker? Maybe advance the slide if you can in the back. Could you grab that for me? There we go. Okay. Back to the agenda, if you would. Back one slide. All right. So just real briefly, I’ll cover what we’re going to do today. We only have 60 minutes for what could possibly be a three-hour discussion. And when I say discussion, I mean it. We want this to be as interactive as possible. We want to include everybody’s comments, thoughts, questions, et cetera. So we’re going to leave about half of our time for Q&A. What we’re going to do is we’re going to have some brief scene-setting remarks from our four fantastic panelists touching on both understanding agentic AI and both the challenges and ethical considerations that come with the use of this technology. So let me introduce you to our panelists. On my left here in the room we have Will Carter who leads AI policy at Google where he focuses on advancing AI innovation at Google and smart AI policy around the world. He made his way from his home in New Mexico in the United States. Joining us online is Luciana Benotti who is the head of research in AI at, here goes my Spanish, Fundacion Villibre. She innovates in participatory methods for AI evaluation in Latin America and is joining us from Argentina. Jayantha Fernando Fernando is a veteran technology and digital law expert and the head of technology media and telecom practice at Heritage Partners in Colombo, Sri Lanka. He’s also a member of the Sri Lankan AI task force. He will be joining us in a few minutes I believe from Kuala Lumpur. Tracking him down is difficult to say the least. He’s always on the move. And then finally to my right we have Dominique Lazanski who is an expert in standards development and negotiates for the UK government at the ITU. She’s also a fellow at the University of Pittsburgh working on tech policy and is consultant on a wide variety of tech projects globally including in Ukraine. She splits her time between London and Lillehammer just north of us here in Oslo. So to get us started I’m going to ask Will and Dominique to sort of give us a brief intro into agentic AI including the role that standards could play in responsible deployment. So Will, you want to get us going?

Will Carter: Thanks Jim and thank you everyone for joining us today. Jim gave me the unenviable job of trying to define agentic AI and set the stage for this conversation which is probably one of the most fraught topics in AI policy at the moment. There is no consensus definition of what agentic AI is and we’re only beginning to see the capabilities of this technology emerge, and I think that a lot will transform over the next couple of years as more of these agentic AI solutions are rolled out around the world. And at the same time, we can identify certain attributes of agentic AI systems that are more unique, certain defining characteristics that differentiate agentic AI from previous generations of AI assistants. And really, I would point to two. The first is the ability to do more complex reasoning within the model, and the ability to perform more complex tasks on behalf of the user. And really, those two key characteristics, the ability to reason about more complex topics, and then take action, are what differentiate emerging agentic AI solutions from previous generations of AI capabilities. So what is this about? Why are we even doing this? Why are we pursuing these technologies? I think that, you know, for us at Google, agents are the next step on our mission from our foundation to really organize the world’s information and make it universally accessible and useful to people. You can think of agents as the next step along a journey that started with what we called PageRank, the original Google search. Over time, we introduced machine learning into search in order to better understand context and provide more relevant answers to user queries. We introduced models like BERT and MUM, early large language models that allowed us to process more complex queries and route users more efficiently to relevant results. And now we’ve introduced features like AI overviews, AI mode, the Gemini apps, which leverage really cutting edge large language models in order to answer users’ questions and direct them to actionable results. And agents are that next step that are going to get our users to more relevant responses, more relevant results, in fewer queries and allow them to take action more efficiently. So it’s really about, you know, that next step in simplifying the user journey and really delivering better results for our users. And what does that look like? Ultimately, this is aspirational. It’ll take years for this technology to get to this point. But we’d like to see a world in which AI agents can pick up a lot of the day to day mental load of our day to day tasks. So I’m a parent, I imagine most of the parents in the room would love if they had a virtual assistant that could manage pickups from after school activities, meal plan for your family based on their eating preferences and needs, you know, help to manage the family budget and with your permission, pay your bills. And, you know, identify an SUV that can accommodate three car seats, but also fit into cramped urban parking spaces and find it at a local dealership in your preferred color. All of these little things are things that I think that agentic AI will hopefully be able to take off our plates, allowing us to focus on more interesting and complex tasks, and the things that are most important to us. So that’s the positive vision. And really, that’s where we see this technology going. And I’ll pass it over to Dominique to talk a little bit more about the implications.

Dominique Lazanski: And actually, well, you bring up the point that I think agentic AI started in the 90s with search and human agents, right? It’s something that I was thinking about recently. So thanks for bringing that up. So I’m going to talk a little bit about standards, AI standards, because we’ve heard a lot about AI here, AI in governance and regulation. I’m going to talk about the sort of bedrock for all this work that Will’s doing at Google or indeed that we’re experiencing when we actually are using it. And as Will said, because he keeps stealing my thunder, setting definitions and terms is really, really key, and there’s been a lot of proposals in this area. And again, when I think of standards, I’m talking about technical standards. We also, for agents as well, we have more standards that we’re going to have to look at in the future regarding personal information, privacy, frameworks around deployment. And as you probably know, a lot of like the European Commission and a lot of other organizations globally are starting to think about that, as well as not-for-profits and other think tanks as well, and academic institutions like my own. But so definition is one that I’m really, you know, we’re all kind of struggling with because, you know, when we say AI agents, it could mean different things to different people depending on regulation or depending on where they’re sitting, quite frankly. So from a technical point of view, JTC 21 within ISO actually has set up a group for defining standards, and they’ve started to produce documents, and they’re also talking a lot about agents as well within that. So the thing is, as AI progresses and as it becomes standardized, there’s a few other things that’ll need to happen with respect to agents as well. One is defining the processes, test methodology, which will be particularly important for engagement, metrics, reference points, and risk analysis. But I think the most important thing as we go forward from a standards point of view and from a discussion point of view for all the different companies and the different organizations coming up with agents is going to be how there’s going to be compatibility and interoperability. Without that, the different ideas and the different potential market solutions to things, whether it’s on a travel site or whether it’s food shopping or perhaps even medication, won’t be able to take place. And standards provide stability as well, provide a baseline and provide a foundation for everything going on as this continues to grow. And I just wanted to highlight from a UK perspective where I’m from and where I work, there’s a great organization called the AI Standards Hub that just put on a AI standards summit. And there’s videos from that summit as well that are really good to dip into and gives you an idea of what the technical standards are, what people are talking about with GDPR and et cetera, and all of that. So from that point of view, I think there needs to be some stability and some sort of common framework or multiple common frameworks from which we can jump off of. And I think that’s what’s happening from that point of view. So I’m going to stop there. We can talk more about it.

Jim Prendergast: Sure. Thanks, Dominic. And, you know, sort of we’re setting the stage here. We can certainly come back to some of the topics that were raised. Luciana, I’m going to turn to you next since we’re still waiting on Fernando to join us. Why don’t you go ahead and, you know, talk about some of the research you’ve done and some of the challenges you’ve discovered along the way.

Luciana Benotti: Well, first of all, thank you very much for the invitation, Jim. I’m happy to be here. I hope next time I can show you guys there in person. I haven’t been to the IGF in person myself. I think the slides need to go on. Yeah, that’s right. So those are my slides. In fact, I wanted to share something, and it’s my first reference there, that is very related to one question that was just posted in the chat by Tinuat from Nigeria on human workforce. So I wanted to share this recent study by the Social and Language Technology Lab at Stanford. I wish we had a similar study for Latin America and also for Africa, Tinuat. It surveyed 1,500 US workers from 100 occupations, asking if they want automation with AI agents. The results showed that workers do want automation, but in a large proportion, not the kind of automation that AI agents currently offer. In this survey, almost half of the tasks by the AI agents that were described there that are offered now by the AI agents were seen by workers as non-desired for automation, including, for instance, agenda setting, while other tasks, for example, scheduling meeting times through a natural language conversation, that have a high potential for automation, according to the researchers that were also involved in the survey to see whether the tasks were actually feasible. But these tasks receive little investment. So who is responsible for this misalignment between human needs and AI agents’ current capabilities? Well, not only companies deploying agents, as many of them rely on a few large language models that work well for some tasks and not for others behind AI agents. It’s also important to notice that this misalignment carries an environmental burden. Every hour and every task performed by a misaligned agent, and I’m not talking about training that agent, contributes to unnecessary energy. energy consumption and carbon footprint that is not making most humans’ lives better according to this survey. So, there is another aspect. Once we decide on a task, if that task is aligned with the needs of the humans, how do we choose one AI agent over another? So, there are also ethical considerations there. And in order to choose between one AI agent over another for a particular task, usually we use metrics. So, for measuring, for instance, the energy consumption and also metrics for deciding which agents make less errors. Some of the most used metrics in AI are not transparent. For example, F1 and accuracy, they treat all kinds of errors in the same way. Let me give you a complete example. Suppose a government is using an AI agent to decide which families receive an energy subsidy. A false positive here means giving a subsidy to someone, to a family, that may not need it, so it’s an inefficiency. But a false negative, denying a subsidy to a family that depends on it, can mean kids going without power to study or to store food, for instance. So, F1 and accuracy, that these are these metrics that are very common in machine learning, do not distinguish between these two types of error. In her paper that we see here, Luciana Ferrar argues that these metrics obscure these risks. She argues that to select good models, researchers can’t just hand off this risk consideration of different kinds of errors to policy implementers. These decisions, these metrics, have to be built into model selection from the start, better metrics, metrics that consider different kinds of errors. This means involving domain experts in early stages of AI design to decide what kind of ferros have different kinds of risks and costs, and not in the late stages as is done in human oversight. So I will leave the last reference there for later on to get the discussion going.

Jim Prendergast: Okay, thank you very much. We are still waiting on Fernando to join us. So I guess, you know, as we previewed for you, we want to make this interactive. So what I’m going to do is I’m sort of going to get the discussion going. I’ve got a couple of questions for our panelists, but so everybody knows rules of engagement here in the room. There are microphones on either side. For our remote participants, if you have a question, please type it into the chat. I know that’s not ideal, but from a technological standpoint, that is the most efficient way to make it happen. So I will read your question into the chat. So let me, you know, we touched on, let me just bring up some thoughts. So obviously, safeguards is something I think everybody in this room is concerned about. You know, you hear the potential horror stories of AI agents running amok. What are some of the safeguards that we do need to protect, you know, privacy, safety, and autonomy in a world full of AI agents? Well, I’m sure you’ve given us a lot of thought.

Will Carter: Quite a lot of thought. This has been core to our mission at Google from the beginning, from our earliest days. And I think agents in some ways take some of these risks and some of these questions to a new level, particularly the degree of autonomy with which they can operate. But really the tradeoff between autonomy and human oversight, I think is crucial to this. And it really is at the center of our approach, which focuses on putting the user in control. You think about what are some of the unique challenges that arise with agents, because in many ways, I want to emphasize, we don’t need to reinvent the wheel. AI agents are AI systems that reflect a lot of the long-standing challenges and concerns associated with AI in general but they do have certain unique attributes that raise new challenges One is the fact that AI agents are often made exponentially more effective by greater access to sensitive data You think, for example, of an agent that, to Luciana’s example earlier, wants to schedule meetings in your calendar It needs access to your calendar, perhaps to your email, in order to understand your schedule and what can be moved around, where there is availability for a meeting to be scheduled Making sure that the user is able to express granular preferences and control what data is accessible to their agents in order to perform the tasks that they want that agent to perform Also making sure that they’re able to control what data is retained by the agent, they’re able to delete old sessions These are all core capabilities that we build into our agents that we think are going to be crucial going forward Another form of user control is making sure that we’re passing back from the agent to the user to make important decisions particularly ones that have significant impacts or that are difficult to reverse To give you a concrete example, let’s say that you ask your agent to plan your dream vacation The agent can autonomously go look at possible vacations on the web Identify ones that map to your particular travel preferences, maybe you’re a beach person, maybe you’re a mountain person, and plan a great vacation But we want that agent to pass back to the user to confirm before it goes ahead and makes any bookings, spends any money, makes commitments on behalf of you and your family So that user control, I think, is really central Another big concern with agents is that they’re susceptible to what’s called prompt injection So the idea that third parties might try to manipulate you are here to talk to us about how to manipulate the agent and get the agent to take actions that are not in the interests of the user. Here, part of it is making sure that we have robust security built into our systems and that agents are, that the models themselves are trained to identify third-party attacks and also to prioritize the user’s needs and instructions over those submitted by a third party. The second piece is making sure that you have appropriate policies in place and that you have appropriate tools to enforce those policies so that if the agent does behave in ways that are unexpected or unintended that you’re able to intervene and prevent the agent from taking actions that could be harmful to the user. And then the other piece is making sure that there’s transparency that enables accountability to the user. You want to make sure that the user is able to understand what actions the agent is taking on their behalf and is able to intervene as needed to make sure that they remain aligned with that user’s interest. The last thing I’ll say is a big part of this, as Dominique alluded to earlier, is going to be about how we build the protocols and standards for the agentic web. And I think, you know, it’s really important to remember that agents interact with data and systems in a very different way than humans. But right now we’re building agents that are using user interfaces and online capabilities that were designed to interact with people. I think that’s going to change as more and more agents are taking more actions on behalf of users on the web. And there’s a very real possibility that we’ll end up with what is in many ways a new layer of the Internet. We’re already starting to see this develop. Google developed the agent-to-agent protocol to help agents to interact with each other in a vendor agnostic and application agnostic way. Anthropic developed the model context protocol to help agents understand and interact with data in safer and more intuitive ways. where additional legislation that’s specific to agents is needed and make sure that we’re putting good policies in place. Thanks.

Jim Prendergast: Well, Luciana, you want to add to this?

Luciana Benotti: Sure. So apart from privacy that we already mentioned, I also wanted to talk about biases. And here I wanted to share a real case that happened that also I had the reference before in the slides on a paper that we wrote with my team. This is a real case that I want to share with you, but like this, there are many similar ones because we did a teacher training on AI agents for 500 teachers in secondary school in Argentina from all different subjects at high school. And so for this, we created a methodology that is called HESEA, that is an evaluation methodology, a critical evaluation methodology for agents constructed with language models. And so at the particular public school in Argentina, high school teachers developed an agent for career guidance for students for choosing a degree to pursue at the university using a well-known LLM API that we provided an interface for them to use it. more easily. And they discovered that this agent tended to recommend degrees that usually have lower tuition fees. So this was something recurrent. So it was recommending careers with lower tuition fees to low-income students. However, in Argentina, for the moment, most universities do not charge tuition fees. And so this kind of, so it didn’t make sense for the agent to recommend careers that do not charge, that do have a lower tuition fee. So this emergent bias can have long-lasting effects on the students’ lives. Unlike this one, we found many other cases. This is one of the points I wanted to raise. And the other one is a question more to the audience, right, to ask ourselves. So when we have AI agents to automate a task, who is going to gain the profits from that task being automated? So I think that’s a very important discussion when we are talking about automation. Thank you.

Jim Prendergast: Great, thanks. And, you know, thanks for posing a question to the audience. And I’m looking at the audience because the microphones do not have anybody standing at them. So now is your chance. Dominique, do you want to add something to this? And I’ll just let everybody know that Jayantha Fernando

Dominique Lazanski: is dialing in now. Fantastic. That’s great. Good. I just wanted to sort of touch upon things which both Will and Luciana mentioned. One is security and one is actually just ensuring a secure environment, a secure development environment from a technical point of view, a secure deployment environment, iteration, testing, and all of that is something that’s are going to be really, really key, because the attack surface is going to be quite large. And I think it’ll be ever more important to make sure that there’s shared practices among even among different companies about how that they’re doing that to the best that they could share that. I think the other thing that I just wanted to mention that, that that’s off the back of Will’s point about creating another layer of the of the Internet, so to speak. I think this AI and agents in particular are going to challenge all of us to think about education differently. So to think about how we are educating our kids or how the schools or any kind of environment that we’re in changes how they describe how to use the Internet or how to get online or how to interact with different things. So there’s definitely an education piece. That’s a long term thing, but it’s something that’s going to to really be needed in light of all of this, in light of this huge step change as well in the economy.

Jim Prendergast: Okay, great. Thanks, Dominique. So we do have Jayantha Fernando Fernando, who has joined us, I believe from Kuala Lumpur. It’s kind of tough coming in 20 minutes into a discussion. But let me catch you up real briefly. I think a lot of what we touched on is talking about sort of misalignment, you know, some of the unpredictable behavior, some of the challenges that come with AI agents, as well as, you know, some of the benefits that come. But Jayantha Fernando, you know, you were a former government official in Sri Lanka. Now you’re working with governments all over the world talking about these issues. You’re on the AI task force there in Sri Lanka. Maybe from a, you know, government perspective, if you can, you know, how are you, how are they thinking about looking at AI agents? And you know, what role do they want to play in the rollout of those?

Jayantha Fernando: Thank you, Jim. Can you guys hear me? We can. Yes. Thanks. I’m sorry to come in late, just in the middle of a cybercrime and electronic evidence program ongoing here with the government authorities here in Malaysia. Just to home directly to your question, I just want to emphasize that at a very early stage of adoption of AI-based systems in private sector, as well as some parts of government, in some of our countries, mostly the strategy is centered around building blocks for basic AI governance kind of rules. So the way I see regulatory intervention in my country, in Sri Lanka, evolving is with the objective of developing a regulatory roadmap. The AI taskforce has set up a legal working group entrusted with that task. It’s expected that we deliver the roadmap or a draft of public comment and stakeholder feedback in around mid-July. So we are at the very tail end of constructing something, but the objectives are to promote responsible innovation through flexible governance tools and address critical AI risks, including discrimination, transparency, privacy, safety, cybersecurity, as well as accountability. Then to look at existing legal frameworks to see how best AI-related challenges or facilitation is provided, if the existing legal frameworks are a barrier. There is an expected timeline that’s currently being developed to modify, amend, or develop new legal frameworks. And in doing new potential legal frameworks, the AI legal working group of the task force has identified both the risk-based hard law approaches to soft law approaches. And the tendency in our part of the world is to look at soft law primarily as an approach being what is perhaps best for Sri Lanka. Given the landscape evolution that is expected, you mentioned about AI agents, exact scope and perimeter and the boundary, and the risk challenges associated with AI agents are being examined. And there is not much visibility on how the challenges may evolve before we even think of potential legal regulatory steps. In Sri Lanka, we don’t want legal regulatory steps to be an impediment towards innovation. And finally, I would like to simply say that the roadmap is focused on fostering public trust in AI system and to align with international development of standards. And in that context, a lot of expectation is in relation to global harmonization. We know that Japan has adopted a very proactive, forward-thinking legal regulatory regime that seems to be… more aligned to our thinking. However, the policymakers back home are even considering looking at NIST and other standard-setting bodies for guidance in this area. Thank you.

Jim Prendergast: Great, thank you. So, one of the things I picked up from what he was saying was, you know, the consultations that governments are undertaking. Will, I’m sure you participated in plenty of those, both in the U.S. and around the world, and I’m sure Luciana as well, and Dominique. I mean, what are some of the common themes that you’re seeing as you’re talking to governments about, you know, what questions do they have, what resources do they need, and what role can both the private sector, civil society, and the technical community play to educate policymakers about the best way to approach this? Dominique, you want to maybe jump? Go ahead. Will, sorry. Sure.

Will Carter: What are governments asking about agents? I mean, frankly, the biggest question that I get about agents from government officials right now is what are agents? And I think that, as I mentioned earlier, that is, you know, the hot topic of the day. I also get a lot of questions about, you know, what really is different about agents and what do we need to think about differently, which I addressed earlier. But I do think that another kind of family of questions that we’re getting from governments is what are the implications of this agentic evolution of AI systems for the broader ecosystem? You think about, for example, the implications for the broad world of cybersecurity outside of AI cybersecurity or agentic cybersecurity itself. You know, one of the defining paradigms of modern cybersecurity practices is bots are bad. That’s generally the assumption, and we prevent a lot of malicious activity online by blocking identifiable bot activity. as agents, which act like bots online and look like bots to a lot of cybersecurity systems. How do we actually enable them to do their jobs while also making sure that bad bots continue to be prevented from causing harm? This is a going to be a much broader question. There’s a lot of really interesting academic research actually going on right now to understand how we differentiate between bad bot behavior online and good agent behavior online. How do we actually identify the two? But then there’s also a lot of work being done on how we authenticate agents, how they identify themselves to third parties, developing those standards that Dominique talked about earlier. The other piece that I’ll flag that we are hearing a lot from governments is they want to understand what is the vision for agents? What is the positive goal? What is it that we’re trying to accomplish by rolling out this technology? And how does it actually work under the hood? And I am really encouraged to hear this. I think it’s a lesson learned from the last few years of AI policymaking that really having a robust understanding of the technology and how it works and what the problem is we’re trying to solve for users is important to making good policy. So to me, it’s really encouraging to see governments increasingly starting from that place and building that into their thinking about regulation.

Jim Prendergast: Great. Thanks. I see Luciana’s got her hand raised.

Luciana Benotti: Yes, I just wanted to add that since I’m also a computer science teacher, a professor at the university, one question I get a lot is how is this going to impact software engineering? Because AI agents are being used more and more by software developers to automate a large part of the software development chain. So I think I think this is a very important discussion also to have, because it has the potential to impact in a strong way the workforce of software engineering, also for those countries that have a lot of workers in this domain. So I think that’s another topic that I wanted to raise.

Jim Prendergast: Thank you for raising that as the parent of a rising third-year computer science major, because it’s definitely front and center in our house. And these are the discussions you need to have with a 20-year-old these days. It’s really important to focus on. Dominique?

Dominique Lazanski: One of the things that I want to point out also is that the regulatory frameworks are still emerging, but they’re also very different. So in the UK, there’s a push, and the AI strategy there, more generally, is to focus on adoption and use and innovation, whereas the AI Act in Europe has really specific requirements in terms of risk analysis, practices that are prohibited. And in particular, one of the prohibitive practices that’s named in the AI Act in the EU is behavior manipulation. So that’s going to be something that’s going to actually be quite challenging, and how to align that, for example, with what’s going on in North and South America, what’s going on in other countries, other regions. We’ve heard from what’s going on in Asia Pacific, more generally. So I think there’s going to still have to be some alignment and agreement about how agents are going to act internationally. And again, best practices and frameworks are going to be key here. But just keep in mind that different countries and different regions are probably going to react differently to agents, also from a cultural point of view as well. So that’s something I wanted to bring up.

Jim Prendergast: Great. Nico, thank you for being the first. first to come up and introduce yourself and fire away.

Nico Caballero: Can you hear me? Yep. Oh, well, so thank you to our fantastic panelists. My name is Nico Caballero. I’m the GAG chair at ICANN, Governmental Advisory Committee. And I have a question for the experts here regarding the influence or the effect that nowadays, sorry, I’m having trouble with my headphones. You can hear me, right? We can hear you. Maybe take it off, and it’s a little easier for you. All right, OK, that’s better. So my question is regarding machine learning. Let’s say machine learning in general. We have supervised machine learning, unsupervised, and then reinforced. Supervised being classification, for example, the way banks and credit cards and everything use classification systems, clustering for unsupervised learning. And then we have reinforced learning, LLMs, and ChatGPT, and DeepSeek, and Gemini, and many others. So my specific question is, how are agents having, or what kind of influence do agents have as regarding supervised and unsupervised learning? For example, classification, just to give an example, or clustering as regarding unsupervised learning. Sorry, I think it’s a very simple question. I’m not over overly complicating things. But if you can give us what the latest news you have in that regard, I would greatly appreciate it.

Will Carter: So it’s an interesting question. I don’t think that, as far as I’m aware, agents are not used in overseeing. are trained using training, at least that I’ve seen up to this point. Generally, agents are trained, are derived from the largest and most complex models because those models are able to conduct the multi step reasoning and take more complex actions as I mentioned earlier. So that means that they’re often trained using unsupervised learning. That being said, there’s a lot more that goes into training those models than just the really large pre-training runs that people think of. And often there is additional supervised training and fine tuning that’s done on those models. They’re often retrained using user feedback or based on actual user journeys, sometimes with what people tend to think of as reinforcement learning, which is more real time, but often in batches and using curated data sets. And so I think that you see a mix of all of these things. An additional piece is when we use agents, we’re also using a variety of other systems, many of which are ML based around those systems, for example, classifiers in order to oversee and manage agent behavior and enforce our policies. And so all of these systems are working together. All these methods are working together to make sure that as we’ve been talking about, agents remain aligned with the user’s goals and kind of with broader societal values, and that they’re doing what the user needs.

Dominique Lazanski: So that’s really interesting because you have to think of it as a real complex system, right? It’s not, I mean, so as humans, we’re used to thinking of things as very linear, very sort of one to one, but I think it’s actually more complex than that. So it’s really interesting to hear from you about that.

Jim Prendergast: Luciana?

Luciana Benotti: Yeah, I just wanted to add that, well, all machine learning algorithms and and agents and supervised learning, everything is developed also by people, right? And that these people are programmers, the machine learning engineers and so on. And these people are using AI agents, as I mentioned before, to help them program. So in a way, it’s influencing also what they do. And something that is important in this, if these people that are doing this job are pressured in their work to maybe hand in their work before they have actually time to read it, that’s also a risk, right? Because there could be code inside what they are building that is not well, that maybe they don’t even understand. So we don’t know what the implications of that are going to be when that software gets deployed.

Jim Prendergast: Great, thank you. I notice we have a question to the left. Just introduce yourself and let us know.

Richard Wingfield: Great, thank you. So my name is Richard Wingfield. I work for a sustainability consultancy called BSR in the tech and human rights team. I wanted to ask a question which touches upon a couple of points that Dominique and Will made, which I think is around how agentic AI will change the way we use the internet and technology. And I thought the examples you gave at the start, Will, of the use cases, things like managing your budget, doing your shopping, buying a car, you sort of said we can do less of that and do more important things. And actually I think you could reframe those as saying things like financial management, thinking about your children’s health and well-being, being able to critically assess options and make a decision are all quite important things that we do as humans. And so I guess my question primarily for Will is if we are thinking about the way that agentic AI may affect our ability as humans to analyse, make decisions. and Dominique, what do you think should be the technical or policy interventions that might be needed to complement companies’ own considerations on that aspect of responsible

Will Carter: AI? That is a great question and first off I’ll say I completely agree that thinking about your family’s finances, about your children’s health is absolutely critical. I mean like I said I’m a parent, I think about this stuff all the time. I think the idea here is not that you don’t think about those things but that I can spend more time thinking about, for example, the kinds of foods that I want my child to be exposed to early in their life, understanding, you know, the many competing opinions about exposure to allergens for example and all of those things. I can spend my time really understanding those issues and making informed decisions about what I want my child to eat as opposed to spending my time writing out the actual grocery list itself. Now that doesn’t mean that I don’t want to be making sure that the groceries that my hypothetical future agent is buying for the family and making sure that they actually reflect my values and my preferences but it’s just a lot of that kind of very menial basic day-to-day load that is embedded in those activities is stuff that I think can be taken off of our plates. The human doesn’t go away, the human needs to be at the center of these processes but it allows us to be a little bit more strategic and hopefully take some of the pure kind of mental weight that we feel out of the day. I will say that for us at Google, making sure that the user has the opportunity to express their preferences not just in the user journey but in how they set up their agents and how they set up their accounts is really important. are all important. That’s why in your Google account, you have the ability to determine what types of data are available to Google AI systems and to Google agents that you’re interacting with, which ones that you want to choose to use, the types of use cases that you want to choose. So really empowering the user and putting them in the driver’s seat is the most important piece of this for us.

Dominique Lazanski: Just also a quick, I’m just going to make a quick point. And that is, thanks, Richard. These great questions. And, you know, we could probably talk about this for hours to be honest, but we have 10 minutes left. Yeah. 10 minutes and 25 seconds. I think there is going to have to be a bigger piece in terms of education for policy makers as well in government. Like, I mean, they don’t even know how the internet runs. Now, I’m not saying that about Nico because clearly no, but on a more serious level, I think that they there needs to be more engagement and interaction because these systems as well as really eloquently said are so complex. Right. And I think that’s going to have to happen. There’s just going to have to be more relationship building in that area with all stakeholders, not just, you know, the private sector engineering, but civil society has a part to play with that too. And I think that’s going to be key in order to develop the right kind of interventions and see what, see what happens.

Jim Prendergast: Luciana, did you want to come in?

Luciana Benotti: I think it’s Lab who has the hand up. Okay. It must be for me. Yeah.

Jim Prendergast: Yeah. I’m not sure. Do we, it keeps coming and disappearing. So I’m not sure. There was a hand raised and now it’s gone. But, you know, there is a sort of related, you know, there is a question in here and I’m going to paraphrase it. Essentially, you know, when dealing with things like healthcare, really sensitive personal information needs to be super secure. How, how secure are these AI agents that are dealing in this? in this particular space, and can they be trusted not to compromise your personal information in one way or another?

Dominique Lazanski: I mean, that’s a great question. And that’s something, it’s not just about the technical aspect and about the risks and about those sort of all of the potential DDoS attacks and the attack surface and everything like that. It’s also about ensuring, and the AI Act in Europe does this, they have a risk analysis and a framework and risk reporting that actually has to describe the risk that is available or potential risk that’s available, but also any issues, any breaches, any attacks. And I’m afraid we’ll probably see quite a few of them, but so far, I think with the baked in security of internet protocols and other evolving security, you know, security develops on different layers of the internet so far that things are moving in the right direction. But I guarantee there is going to be some issues.

Jim Prendergast: Luciana?

Luciana Benotti: Well, I completely agree there that we will see some issues. So as an NLP researcher, we know that large language models, when they use private information for training, they can repeat that information at any point, even if there is no particular attack to extract that information. So there is no way to ensure from the technological point of view that that information is not disclosed to someone that it shouldn’t be disclosed to, unless that information is not used for training. The problem is that since there’s a lack of enough data for training systems, especially in domains that are sensitive, like healthcare, is… tempting, let’s say, to use information that is not properly anonymized for training purpose. And the problem is that if the information is in natural language, it’s in fact impossible to completely anonymize it because we can identify names, we can identify addresses, personal identification numbers, but there can be private information that is not expressed in any of these formats or that combined with other information makes a person identifiable. So this is, from a technical point of view, this is a problem that at the moment has no solution.

Participant: Jim, can I come in on that? Sure, please. Yeah, so Jim, I think on that particular area that the panelists touched on, I just want to give some perspective while I tend to agree with both of them. My view is that generally most of the data protection laws tend to cover the use of personal data and processing them in an agentic, AI agent environment may constitute or trigger perhaps the various regulatory regimes governing or regulating processing personal data. And in the context of maybe health-related information, which would definitely come within the category of special categories of personal data, a high degree of protection is required in most countries. The same is true when it comes to Sri Lanka, for example, we have a personal data protection law. founded on the OECD privacy guidelines as well as based on some of the principles contained in GDPR but the enforcement regime is not based on GDPR. Enforcement framework is primarily based on some of the APAC privacy framework principles with options to do force corrections before moving towards a regulatory fining kind of regime. So I think both the in terms of regulatory intervention one has to be careful not to stifle innovation while we need agent AI growth that should not be hindered by regulatory frameworks such as privacy laws. Privacy laws should be should provide the platform for course correction to address the grievances of those individuals whose rights are affected and more so in the area of personal data that might related to health related information. So the clear boundaries and striking the right balance is an approach that we are looking at and that’s where I believe a kind of a light touch regulatory a soft law approach with some kind of policy guidance in this realm may be more appropriate but we still do not have a standardized international response in this area. Thank you. Thank you and this always happens we

Jim Prendergast: we have three minutes and 20 seconds left and we have two people at the microphone so what I’m going to ask you to do is both ask your question we’ll try and answer it but they have told me they will cut us off at the 12 12 30. So I’m not sure who was first but go ahead and we’ll try and answer them real quick in a lightning round and finish up as well.

Participant: Thanks my name is Carl Fred Kirkland. I’m here on behalf of myself as a member of the tech community. We’ll touch on the topic of keeping the human in the loop on impactful decisions. made by the agents and with the introduction of agentic AI the role and responsibilities of the user and service providers change and as companies start offering agentic AI solutions how do you think the dynamic between the user and the service provider will change and how do we establish a conscious and informed approach to the responsibilities and accountability in this new dynamic?

Jim Prendergast: We’ll take a question to the left.

Anne McCormick: Thank you, Anne McCormick, EY, Global Head of Public Policy. I’m interested in this context of policy not just being regulation. One of the key things are market forces but market forces by themselves are slow. We’re seeing on in the space of AI something which is demand by board members as well as actually developers which is third-party verification, third-party assessments, third-party assurance, third-party audits to be made public and meaningful. I think this is something to look at as a tool, a voluntary tool, but a tool with specific parameters to be applied to agentic. It supports adoption, it supports trust and it forces disclosure and learning throughout the ecosystem. We’re about to publish something on this next week online looking at what’s happening across the world. I’m happy to share look it up on EY’s website but these are existing tools that are flexible. Thank you very much. Great, thank you. So 15 seconds per panelist.

Jim Prendergast: Respond and wrap up and give us a final parting thought. Great, so I’ll go just

Dominique Lazanski: to say that like the kite marking or assurance assessment has a long history in telecoms and physical devices so there’s a lot to, as you were saying, there’s a lot to base like a sort of kite marker and assurance mark on. So that’s great actually that we have like a history of that. I don’t know, I think I’m gonna go off the grid. Or pass.

Will Carter: I completely agree there is a long history there and I do think that building assurance and validation frameworks and having third parties helping us to build trust in these AI systems and agentic systems is going to be critically important. Coming up with good metrics and evals to support that is a continuing area of research and I think we’re really investing in that on the Google side and are really excited to build partnerships to do more of that. The other question, I would just say, as I mentioned before, being really thoughtful about passbacks and how we loop the user back in in ways that are useful and beneficial to the user is really the critical piece. But also as we learn over time from user feedback what they like and don’t like and what does enable them to trust the system, that’s going to be, that’s going to allow us to continue to improve.

Jim Prendergast: Luciana, final comment.

Luciana Benotti: Yes, so I completely agree with this last question. So third party assessment is crucial and I leave the audience with a question. So how can we credibly work towards a future where this third party is built by the most, is made up by the most affected, the workers, the teachers, the students, these are the most affected at times that we are seeing in Latin America. When they are meaningful co-designers of the AI agents that affect them and that are involved also in analysing the environmental impact with metrics that they can understand and not just use them as raw data providers. So that’s my question for you. And finally, if you are there in person, please reach out to Beatriz Busaniche that can tell you more about our work in Fundación Vía Libre that is there in the room.

Jim Prendergast: Jayanta, I’m sorry, but our wonderful, polite Norwegian tech support team is giving me that look. We’re unfortunately going to have to wrap up, but Will and Dominique are still going to be here. Any questions? I’m sure they’d be happy to talk to you afterwards. Thanks for Luciana and Jayantha Fernando for joining us remotely, and thanks for everybody for participating in person. Thank you.

Agreement points

Need for clear definitions and standards in agentic AI development

Speakers

– Will Carter
– Dominique Lazanski

Arguments

Lack of consensus definition creates challenges for policy development

Standards development is crucial for defining processes, test methodology, and ensuring interoperability

Summary

Both speakers emphasize that the lack of clear definitions for agentic AI creates fundamental challenges for both policy development and technical implementation. They agree that establishing standards and common frameworks is essential for the field to progress effectively.

Topics

Digital standards | Data governance

Importance of security measures and robust safeguards

Speakers

– Will Carter
– Dominique Lazanski

Arguments

Security measures against prompt injection attacks and robust policy enforcement

Attack surface will be large, requiring shared security practices among companies

Summary

Both speakers recognize that agentic AI presents significant security challenges requiring comprehensive safeguards. They agree on the need for robust security measures, shared practices among companies, and strong policy enforcement mechanisms.

Topics

Network security | Cybercrime

Need for enhanced education and stakeholder engagement with policymakers

Speakers

– Will Carter
– Dominique Lazanski

Arguments

Governments primarily ask ‘what are agents?’ showing need for basic education

Need for enhanced education and relationship building between policymakers and stakeholders

Summary

Both speakers identify a critical gap in policymaker understanding of agentic AI, with governments asking basic definitional questions. They agree that enhanced education and better relationship building between all stakeholders is essential for effective governance.

Topics

Capacity development | Data governance

Support for third-party assessment and verification mechanisms

Speakers

– Will Carter
– Anne McCormick
– Dominique Lazanski

Arguments

Importance of user feedback loops and passback mechanisms for maintaining trust

Market demand for third-party verification and public assessments of AI systems

Different regulatory frameworks across regions create alignment challenges

Summary

Multiple speakers agree that third-party assessment, verification, and accountability mechanisms are crucial for building trust in agentic AI systems. They see this as both a market demand and a technical necessity for responsible deployment.

Topics

Consumer protection | Digital standards

Privacy and data protection concerns require special attention

Speakers

– Luciana Benotti
– Jayantha Fernando
– Jim Prendergast

Arguments

Large language models can inadvertently disclose private training data

Healthcare data requires special protection under existing data protection laws

Security concerns about AI agents handling sensitive healthcare information require careful consideration

Summary

All three speakers acknowledge that privacy and data protection present unique challenges in agentic AI, particularly for sensitive data like healthcare information. They agree that existing technical limitations make complete data protection difficult to guarantee.

Topics

Privacy and data protection | Network security

Similar viewpoints

Both speakers recognize that agentic AI requires fundamentally new technical infrastructure and standards, as agents interact with systems differently than humans do. They see this as requiring new protocols and frameworks for the emerging ‘agentic web’.

Speakers

– Will Carter
– Dominique Lazanski

Arguments

Need for new protocols and standards for the ‘agentic web’ as agents interact differently than humans

Standards development is crucial for defining processes, test methodology, and ensuring interoperability

Topics

Digital standards | Critical internet resources

Both speakers express concern about the misalignment between what humans actually want from AI automation and what is currently being developed. They worry about the impact on human agency and decision-making capabilities.

Speakers

– Luciana Benotti
– Richard Wingfield

Arguments

Workers want automation but not the kind currently offered by AI agents

Concern that agentic AI may reduce human ability to analyze and make important decisions

Topics

Future of work | Human rights principles

Both speakers advocate for inclusive approaches to AI governance that involve affected communities and stakeholders in meaningful ways, while maintaining flexibility to avoid stifling beneficial innovation.

Speakers

– Luciana Benotti
– Jayantha Fernando

Arguments

Need for meaningful involvement of affected communities in AI agent design and assessment

Sri Lanka focuses on flexible governance tools and soft law approaches to avoid stifling innovation

Topics

Human rights principles | Capacity development

Unexpected consensus

Complex systems approach to AI agent development

Speakers

– Will Carter
– Dominique Lazanski
– Nico Caballero

Arguments

Complex systems integrate multiple ML approaches including classifiers for policy enforcement

Different regulatory frameworks across regions create alignment challenges

Question about how agentic AI influences supervised and unsupervised machine learning approaches

Explanation

There was unexpected consensus that agentic AI should be understood as complex systems rather than simple linear technologies. Even the technical question about machine learning approaches revealed agreement that agents integrate multiple ML methods working together, representing a systems-level approach.

Topics

Digital standards | Network security

Balance between innovation and protection

Speakers

– Will Carter
– Jayantha Fernando
– Dominique Lazanski

Arguments

User control through granular data access preferences and human oversight for important decisions

Balance needed between privacy protection and innovation

Different regulatory frameworks across regions create alignment challenges

Explanation

Despite representing different sectors (industry, government, standards), there was unexpected consensus on the need to balance innovation with protection rather than choosing one over the other. All speakers advocated for nuanced approaches that enable beneficial uses while managing risks.

Topics

Data governance | Privacy and data protection

Overall assessment

Summary

The speakers demonstrated strong consensus on fundamental challenges including the need for clear definitions, robust security measures, enhanced education for policymakers, third-party assessment mechanisms, and special attention to privacy concerns. There was also notable agreement on the complexity of agentic AI systems and the need to balance innovation with protection.

Consensus level

High level of consensus on core challenges and approaches, with implications that the agentic AI community has identified shared priorities for responsible development. The consensus suggests that collaborative approaches involving multiple stakeholders are both feasible and necessary for addressing the governance challenges of agentic AI.

Different viewpoints

Human agency in decision-making vs. AI automation

Speakers

– Will Carter
– Richard Wingfield

Arguments

Agents should handle menial tasks while preserving human strategic thinking and value-based decisions

Concern that agentic AI may reduce human ability to analyze and make important decisions

Summary

Carter argues that AI agents should take over routine tasks to free humans for more strategic thinking, while Wingfield contends that activities like financial management and health decisions are themselves important human capabilities that shouldn’t be delegated to AI.

Topics

Human rights principles | Future of work

Regulatory approach – hard law vs. soft law

Speakers

– Dominique Lazanski
– Jayantha Fernando

Arguments

Different regulatory frameworks across regions create alignment challenges

Sri Lanka focuses on flexible governance tools and soft law approaches to avoid stifling innovation

Summary

Lazanski highlights the challenges created by different regulatory approaches across regions (noting EU’s specific prohibitions vs UK’s innovation focus), while Fernando advocates specifically for soft law approaches as optimal for avoiding innovation barriers.

Topics

Data governance | Jurisdiction

Privacy protection feasibility

Speakers

– Will Carter
– Luciana Benotti

Arguments

User control through granular data access preferences and human oversight for important decisions

Large language models can inadvertently disclose private training data

Summary

Carter emphasizes user control mechanisms and granular preferences as solutions for privacy protection, while Benotti argues that from a technical standpoint, there’s no way to ensure private information won’t be disclosed if it’s used in training data.

Topics

Privacy and data protection | Network security

Unexpected differences

Fundamental technical feasibility of privacy protection

Speakers

– Will Carter
– Luciana Benotti

Arguments

User control through granular data access preferences and human oversight for important decisions

Complete anonymization of natural language data is technically impossible

Explanation

This disagreement is unexpected because it reveals a fundamental divide between industry and academic perspectives on whether privacy can be technically guaranteed in AI systems. Carter’s confidence in user control mechanisms contrasts sharply with Benotti’s assertion that technical limitations make complete privacy protection impossible.

Topics

Privacy and data protection | Digital standards

Worker preferences vs. current AI development priorities

Speakers

– Will Carter
– Luciana Benotti

Arguments

Complex reasoning and autonomous action capabilities distinguish agentic AI from previous generations

Workers want automation but not the kind currently offered by AI agents

Explanation

This represents an unexpected disagreement about market alignment, where Carter presents AI agent development as a natural technological progression serving user needs, while Benotti provides evidence that workers actually don’t want the types of automation currently being developed, suggesting a fundamental misalignment between industry development and user preferences.

Topics

Future of work | Sustainable development

Overall assessment

Summary

The discussion revealed moderate levels of disagreement across several key dimensions: the appropriate balance between human agency and AI automation, regulatory approaches (hard vs. soft law), technical feasibility of privacy protection, and alignment between AI development priorities and actual user needs.

Disagreement level

The disagreements are significant but not fundamental – speakers generally agree on the importance of responsible AI development but differ on methods and feasibility. The most concerning disagreement involves technical feasibility of privacy protection, which has direct implications for user safety. The misalignment between worker preferences and current development priorities suggests potential market failures that could undermine adoption and social acceptance of agentic AI.

Partial agreements

Similar viewpoints

Both speakers recognize that agentic AI requires fundamentally new technical infrastructure and standards, as agents interact with systems differently than humans do. They see this as requiring new protocols and frameworks for the emerging ‘agentic web’.

Speakers

– Will Carter
– Dominique Lazanski

Arguments

Need for new protocols and standards for the ‘agentic web’ as agents interact differently than humans

Standards development is crucial for defining processes, test methodology, and ensuring interoperability

Topics

Digital standards | Critical internet resources

Both speakers express concern about the misalignment between what humans actually want from AI automation and what is currently being developed. They worry about the impact on human agency and decision-making capabilities.

Speakers

– Luciana Benotti
– Richard Wingfield

Arguments

Workers want automation but not the kind currently offered by AI agents

Concern that agentic AI may reduce human ability to analyze and make important decisions

Topics

Future of work | Human rights principles

Both speakers advocate for inclusive approaches to AI governance that involve affected communities and stakeholders in meaningful ways, while maintaining flexibility to avoid stifling beneficial innovation.

Speakers

– Luciana Benotti
– Jayantha Fernando

Arguments

Need for meaningful involvement of affected communities in AI agent design and assessment

Sri Lanka focuses on flexible governance tools and soft law approaches to avoid stifling innovation

Topics

Human rights principles | Capacity development

Key takeaways

Agentic AI is distinguished by complex reasoning capabilities and autonomous action, but lacks consensus definition creating policy challenges

User control and human oversight are essential safeguards, particularly for high-impact decisions that are difficult to reverse

Significant misalignment exists between what workers want automated and what AI agents currently offer, creating unnecessary environmental burden

AI agents can perpetuate harmful biases, as demonstrated by systems recommending lower-cost degrees to low-income students despite free university education

Standard evaluation metrics (F1, accuracy) are inadequate as they don’t distinguish between different types of errors with varying social costs

Technical standards and interoperability protocols are crucial for responsible deployment and the emerging ‘agentic web’

Different regulatory approaches across regions (UK focus on adoption vs EU risk-based prohibitions) create international alignment challenges

Privacy protection is technically challenging as large language models can inadvertently disclose private training data, and complete anonymization of natural language is impossible

Third-party assessment and verification are increasingly demanded by markets as tools for building trust and ensuring accountability

Enhanced education and relationship building between policymakers, private sector, and civil society stakeholders is essential for effective governance

Resolutions and action items

Sri Lanka’s AI task force legal working group to deliver regulatory roadmap draft for public comment by mid-July

Continued development of agent-to-agent protocols and model context protocols for safer AI agent interactions

Investment in better evaluation metrics and frameworks that account for different error types and social costs

Development of authentication standards for AI agents to distinguish them from malicious bots

EY to publish research on third-party verification and assessment tools for AI systems

Unresolved issues

No consensus definition of agentic AI exists, hampering policy development

How to achieve international harmonization of AI agent governance across different regulatory frameworks

Who will benefit from the profits generated by AI agent automation of human tasks

How to ensure meaningful participation of affected communities (workers, teachers, students) in AI agent design and assessment

Technical challenge of preventing AI agents from disclosing private information used in training data

How to distinguish between legitimate AI agents and malicious bots in cybersecurity systems

Impact of AI agents on software engineering workforce and the risks of developers not understanding AI-generated code

How to balance privacy protection with innovation in AI agent development

Establishing clear boundaries for accountability and responsibility between users and service providers in agentic AI systems

Suggested compromises

Soft law approaches with policy guidance rather than hard regulatory frameworks to balance innovation with protection

Light-touch regulatory approach with options for course correction before moving to fining regimes

Flexible governance tools that can adapt as the technology evolves rather than rigid regulatory frameworks

Agents handle menial tasks while preserving human strategic thinking and value-based decision-making

Passback mechanisms that loop users into important decisions while allowing automation of routine tasks

Risk-based approaches that apply different levels of oversight based on the potential impact of AI agent decisions

We surveyed 1,500 US workers from 100 occupations, asking if they want automation with AI agents. The results showed that workers do want automation, but in a large proportion, not the kind of automation that AI agents currently offer. Almost half of the tasks by the AI agents that were described there that are offered now by the AI agents were seen by workers as non-desired for automation, including, for instance, agenda setting, while other tasks, for example, scheduling meeting times through a natural language conversation, that have a high potential for automation… So who is responsible for this misalignment between human needs and AI agents’ current capabilities?

Speaker

Luciana Benotti

Reason

This comment is deeply insightful because it challenges the fundamental assumption that AI automation is inherently beneficial or desired. It introduces empirical evidence showing a critical disconnect between what companies are building and what workers actually want automated, raising important questions about who drives AI development priorities.

Impact

This comment fundamentally shifted the discussion from technical capabilities and governance frameworks to human-centered concerns about automation’s real-world impact. It introduced the concept of ‘misalignment’ not just as a technical problem, but as a socioeconomic issue, and prompted later questions about workforce impact and who benefits from automation.

F1 and accuracy, they treat all kinds of errors in the same way. Let me give you a complete example. Suppose a government is using an AI agent to decide which families receive an energy subsidy. A false positive here means giving a subsidy to someone, to a family, that may not need it, so it’s an inefficiency. But a false negative, denying a subsidy to a family that depends on it, can mean kids going without power to study or to store food… F1 and accuracy, that these are these metrics that are very common in machine learning, do not distinguish between these two types of error.

Speaker

Luciana Benotti

Reason

This comment is exceptionally thought-provoking because it exposes a fundamental flaw in how AI systems are evaluated – that technical metrics obscure real-world ethical implications. The concrete example of energy subsidies makes abstract technical concepts viscerally understandable and highlights how seemingly neutral metrics can perpetuate harm.

Impact

This comment elevated the technical discussion to include ethical considerations and real-world consequences. It demonstrated how technical decisions about metrics are actually policy decisions with human impact, influencing later discussions about the need for domain experts in early AI design rather than just oversight.

One of the defining paradigms of modern cybersecurity practices is bots are bad. That’s generally the assumption, and we prevent a lot of malicious activity online by blocking identifiable bot activity. as agents, which act like bots online and look like bots to a lot of cybersecurity systems. How do we actually enable them to do their jobs while also making sure that bad bots continue to be prevented from causing harm?

Speaker

Will Carter

Reason

This comment is insightful because it identifies a fundamental paradox in the current internet infrastructure – the systems designed to protect us may inadvertently block the AI agents meant to help us. It reveals how agentic AI challenges existing security paradigms in unexpected ways.

Impact

This comment introduced a new dimension to the governance discussion by highlighting how agentic AI will require rethinking fundamental internet security principles. It led to discussions about authentication, standards development, and the need for a new ‘layer of the internet’ specifically designed for agent interactions.

I think the examples you gave at the start, Will, of the use cases, things like managing your budget, doing your shopping, buying a car, you sort of said we can do less of that and do more important things. And actually I think you could reframe those as saying things like financial management, thinking about your children’s health and well-being, being able to critically assess options and make a decision are all quite important things that we do as humans… if we are thinking about the way that agentic AI may affect our ability as humans to analyse, make decisions.

Speaker

Richard Wingfield

Reason

This comment is profoundly thought-provoking because it challenges the core value proposition of agentic AI by questioning whether the tasks being automated are actually ‘menial’ or are fundamental to human agency and skill development. It reframes convenience as potentially problematic skill atrophy.

Impact

This comment forced a critical examination of the underlying assumptions about what constitutes valuable human activity. It shifted the conversation from efficiency benefits to concerns about human capability preservation, prompting Will to clarify the distinction between strategic thinking and menial execution, and influencing discussions about user control and education.

How can we credibly work towards a future where this third party is built by the most, is made up by the most affected, the workers, the teachers, the students, these are the most affected at times that we are seeing in Latin America. When they are meaningful co-designers of the AI agents that affect them and that are involved also in analysing the environmental impact with metrics that they can understand and not just use them as raw data providers.

Speaker

Luciana Benotti

Reason

This closing comment is deeply insightful because it challenges the entire governance framework being discussed by questioning who gets to be at the decision-making table. It highlights the power dynamics inherent in AI development and calls for genuine participatory design rather than top-down solutions.

Impact

As a closing statement, this comment reframed the entire discussion by emphasizing that technical and policy solutions are insufficient without meaningful inclusion of affected communities. It challenged participants to consider whether their governance approaches were truly inclusive or merely consultative.

Overall assessment

These key comments fundamentally transformed what could have been a technical discussion about AI governance into a nuanced examination of power, values, and human agency. Luciana Benotti’s contributions were particularly impactful, consistently challenging assumptions about automation benefits and technical neutrality while introducing empirical evidence and concrete examples. Richard Wingfield’s intervention forced a critical examination of core value propositions, while Will Carter’s cybersecurity paradox highlighted unexpected technical challenges. Together, these comments elevated the discussion from ‘how do we govern agentic AI’ to ‘who decides what problems agentic AI should solve, and for whom.’ The conversation evolved from technical implementation details to fundamental questions about human agency, democratic participation in technology design, and the alignment between technological capabilities and human needs. This progression demonstrates how thoughtful, evidence-based challenges to prevailing assumptions can transform policy discussions from reactive governance to proactive consideration of technology’s role in society.

How do we establish a conscious and informed approach to the responsibilities and accountability in the new dynamic between users and service providers with agentic AI?

Speaker

Carl Fred Kirkland

Explanation

This addresses the changing relationship dynamics and accountability frameworks needed as AI agents take on more autonomous roles

How can we credibly work towards a future where third party assessment is built by the most affected – the workers, teachers, students – as meaningful co-designers of AI agents that affect them?

Speaker

Luciana Benotti

Explanation

This focuses on ensuring those most impacted by AI agents have meaningful participation in their design and evaluation rather than just being data providers

How is agentic AI going to impact software engineering and the workforce in this domain?

Speaker

Luciana Benotti

Explanation

This addresses the significant potential impact on software development careers and practices as AI agents automate parts of the development chain

Who is going to gain the profits from tasks being automated by AI agents?

Speaker

Luciana Benotti

Explanation

This raises important questions about economic distribution and benefits of AI automation

How do we differentiate between bad bot behavior online and good agent behavior online?

Speaker

Will Carter

Explanation

This is critical for cybersecurity as traditional ‘bots are bad’ paradigms need to accommodate legitimate AI agents

How do we develop authentication standards for AI agents to identify themselves to third parties?

Speaker

Will Carter

Explanation

This is necessary for creating trust and security in agent-to-agent and agent-to-system interactions

How will different cultural perspectives and regulatory frameworks be aligned internationally for AI agents?

Speaker

Dominique Lazanski

Explanation

This addresses the challenge of harmonizing different regional approaches to AI governance and cultural acceptance

What kind of influence do agents have on supervised and unsupervised machine learning, particularly in classification and clustering?

Speaker

Nico Caballero

Explanation

This seeks to understand the technical relationships between different ML approaches and agentic AI systems

How secure are AI agents dealing with sensitive healthcare information and can they be trusted not to compromise personal data?

Speaker

Anonymous participant (via chat)

Explanation

This addresses critical privacy and security concerns in high-stakes domains like healthcare

Need for similar studies to the Stanford worker survey in Latin America and Africa regarding worker preferences for AI automation

Speaker

Implied by Luciana Benotti in response to Tinuat from Nigeria

Explanation

This would provide crucial regional data on worker attitudes toward AI automation to inform policy and development

How do we develop better metrics that consider different types of errors and their varying impacts, especially in sensitive domains?

Speaker

Luciana Benotti

Explanation

This addresses the inadequacy of current ML metrics like F1 and accuracy that treat all errors equally when some have much more serious consequences

How do we ensure domain experts are involved in early stages of AI design rather than just late-stage human oversight?

Speaker

Luciana Benotti

Explanation

This is important for building appropriate risk considerations into model selection from the beginning rather than as an afterthought