WEF Business Engagement Session: Safety in Innovation – Building Digital Trust and Resilience

Summary

This World Economic Forum session at the Internet Governance Forum focused on implementing safety by design principles in digital innovation, featuring representatives from major tech companies including TikTok, Meta, AWS, EAND, and OpenAI. The discussion centered on how companies can embed safety considerations from the earliest stages of product development rather than treating safety as an afterthought or compliance checkbox.

Each panelist shared their organization’s approach to safety by design, revealing both common themes and unique challenges. TikTok emphasized building different safety features for various user groups, particularly implementing stronger protections for teenage users such as automatic privacy settings and messaging restrictions. Meta highlighted their stakeholder engagement process, consulting with over 120 experts across 34 countries when developing AI content labeling policies. AWS focused on providing safety tools and resources to their diverse customer base across the entire technology stack, including automated detection systems for harmful content.

The panelists identified several key challenges in implementing safety by design, including the need to balance innovation speed with comprehensive safeguards, the difficulty of anticipating all potential misuses by bad actors, and the lack of technical expertise in safety principles among development teams. They emphasized the importance of breaking down silos between safety teams and product development teams, with several speakers noting the need for better collaboration between engineers, policy makers, and safety experts.

A recurring theme was the necessity of multi-stakeholder engagement, including partnerships with civil society organizations, academic institutions, and government regulators. The discussion also addressed the rapid evolution of AI technology and the unique safety challenges it presents, with OpenAI describing their approach of external red teaming and transparent documentation of safety measures. The session concluded with calls for greater inclusion of engineers and product managers in policy discussions and continued collaboration across the industry to develop effective safety standards.

Keypoints

## Major Discussion Points:

– **Defining “Safety by Design”**: Each panelist provided their interpretation of what safety by design means in practice, revealing different approaches across companies – from TikTok’s multi-layer moderation strategies to OpenAI’s Model Spec framework to AWS’s infrastructure-level safety tools. The discussion highlighted the need for common language and understanding in the digital safety space.

– **Balancing Safety and Innovation**: A central theme was dispelling the notion that safety and innovation are incompatible. Panelists emphasized that safety should be viewed as an enabler of innovation rather than a constraint, with proper governance frameworks allowing companies to be both safe and innovative.

– **Cross-Industry Collaboration and Stakeholder Engagement**: The conversation extensively covered the importance of breaking down silos – both within companies (between engineering, product, and safety teams) and across the broader ecosystem (with civil society, academia, governments, and other companies). Multiple panelists stressed the need for multi-stakeholder approaches to address complex safety challenges.

– **AI-Specific Safety Challenges**: Given the rapid evolution of AI technology, panelists discussed unique challenges in implementing safety by design for AI systems, including the need for red teaming, external expert evaluation, transparency through system cards, and maintaining human oversight as AI capabilities advance.

– **Regulatory Frameworks as Innovation Drivers**: The discussion explored how emerging regulations (like the EU’s AI Act and UK’s Online Safety Act) can actually foster innovation by providing clear frameworks and pushing companies toward more creative safety solutions, rather than simply constraining development.

## Overall Purpose:

The discussion aimed to explore how major technology companies implement “safety by design” principles in their products and services, sharing best practices and challenges while building common understanding among industry stakeholders. The session was designed to demonstrate that safety and innovation can coexist and to encourage greater collaboration across the digital safety ecosystem.

## Overall Tone:

The tone was collaborative and constructive throughout, with panelists openly sharing both successes and challenges in their safety implementations. There was a notable spirit of industry cooperation, with speakers acknowledging each other’s work and building on shared themes. The discussion maintained a professional yet accessible tone, with panelists demonstrating genuine commitment to safety principles while being realistic about ongoing challenges. The audience engagement toward the end added energy and urgency to calls for greater inclusion of engineers and product managers in policy discussions, reinforcing the collaborative atmosphere.

Speakers

– **Agustina Callegari** – Digital Safety Lead at the World Economic Forum, session moderator

– **Rafaela Nicolazzi** – Data Privacy and Consumer Protection Lead at OpenAI

– **Vinicius Fortuna** – Software engineer at Google, leads the Internet Freedom team at Jigsaw

– **Luis Adrian** – Professor, former Minister of Science, Technology and Telecommunication of Costa Rica

– **Andrea Vega** – Works in tech policy and news integrity

– **Audience** – Unidentified audience member (Beatrice – lawyer and assistant professor in law)

– **Jeff Collins** – Head of Global Trust and Safety at Amazon Web Services

– **Peter Stern** – Stakeholder Engagement Director at META, works on content policy team

– **Valiant Richey** – Global Head of Outreach and Partnership at TikTok

– **Renato Leite** – VP of Legal, AI, Privacy and Innovation at EAND

**Additional speakers:**

– **Dr. Nermeen Saleem** – Secretary General of Creators Union of Arab, CASA Consultative Statist with the UN

# Implementing Safety by Design in Digital Innovation: A World Economic Forum Session Report

## Executive Summary

This World Economic Forum session at the Internet Governance Forum in Norway brought together leading representatives from major technology companies including TikTok, Meta, Amazon Web Services, OpenAI, and EAND to discuss the implementation of safety by design principles in digital innovation. The discussion, moderated by Agustina Callegari, Digital Safety Lead at the World Economic Forum and the Global Coalition for Digital Safety, revealed both consensus on fundamental safety principles and varied approaches to implementation across the industry.

The one-hour session with five speakers demonstrated that safety and innovation are increasingly viewed as complementary rather than competing priorities. Panellists shared concrete examples of how their organisations embed safety considerations from the earliest stages of product development, moving beyond reactive content moderation to proactive, systemic approaches. The conversation highlighted the evolution of digital safety to encompass the entire technology ecosystem, from infrastructure to user-facing applications.

## Key Participants and Their Perspectives

### Platform Safety Approaches

**Valiant Richey**, Global Head of Outreach and Partnership at TikTok, outlined a multi-layered approach to safety by design encompassing automated safeguards, human moderation, and targeted protections for vulnerable users. Drawing from his previous experience at “After School,” a teen-focused app, Richey emphasised that safety must be embedded from product development inception through cross-team collaboration.

TikTok implements differentiated safety features for various user groups, with automatic privacy settings and messaging restrictions for users under 16, and additional screen time limits for those under 18. The platform has developed family pairing features and created platform-agnostic Digital Safety Partnerships for Families resources that extend beyond TikTok-specific guidance. Richey noted the ongoing challenge of balancing safety with fundamental rights like freedom of expression whilst addressing motivated bad actors, particularly with nuanced content like hate speech that can be “highly contextualised.”

**Peter Stern**, Stakeholder Engagement Director at Meta, highlighted the company’s extensive consultation processes, including engaging over 120 experts across 34 countries when developing AI content labelling policies. Despite being at Meta for 11 years, Stern noted this was his first Internet Governance Forum, reflecting the company’s increased engagement with multi-stakeholder processes.

Meta treats safety as a core principle alongside voice, dignity, and privacy. Stern revealed Meta’s significant shift in content moderation strategy in the United States, transitioning from third-party fact-checking to a crowdsourced community notes model whilst maintaining traditional fact-checking partnerships globally. This represents a fundamental departure from industry norms and demonstrates ongoing experimentation with different approaches to misinformation.

### Infrastructure and AI Safety

**Jeff Collins**, Head of Global Trust and Safety at Amazon Web Services, brought a unique infrastructure perspective, emphasising that safety thinking must span the entire technology stack. His background includes previous roles at TikTok and “After School,” providing him with experience across different layers of the technology ecosystem.

AWS provides safety tools across their extensive service portfolio, including automated detection systems and built-in safety controls such as input/output filtering and CSAM (Child Sexual Abuse Material) detection through partnerships with NECMEC (National Center for Missing and Exploited Children) using hash matching technology. Collins stressed the importance of integrating trust and safety considerations into AWS Bedrock, their generative AI service, recognising the need to break down silos between teams during rapid AI development cycles.

**Rafaela Nicolazzi**, Data Privacy and Consumer Protection Lead at OpenAI, provided insights into AI-specific safety challenges, emphasising proactive commitment from inception rather than waiting for regulation. OpenAI’s approach includes model specification documents, system cards, and preparedness frameworks for risk assessment, alongside rigorous testing and external red teaming before model release.

Nicolazzi discussed OpenAI’s development of “Reasoners” and the O1 model, noting the profound implications of AI systems potentially exceeding human intelligence: “AI is already smarter than us, right? And it’s just getting smarter and smarter… it will be very hard for us to leave a room in the near future and think that we are smarter than AI around there.” This raises critical questions about maintaining human oversight and governance as AI capabilities advance.

### Regional Implementation Challenges

**Renato Leite**, VP of Legal, AI, Privacy and Innovation at EAND, offered insights into implementing safety by design in regions with developing regulatory frameworks. Working primarily in the Middle East, Leite highlighted challenges of low regulatory maturity and expertise, contrasting this with Brazil’s more advanced discussions around data protection that began in 2015.

EAND’s approach emphasises governance from design, requiring automation and AI for policy enforcement across all stages. Leite noted the importance of brand reputation in driving safety adoption in markets where regulatory enforcement may be limited, and stressed the need for upskilling teams on responsible AI principles whilst breaking down language barriers between legal/policy and engineering teams.

## Areas of Consensus

### Safety as a Design Principle

All panellists agreed that safety must be embedded from the beginning of product development rather than treated as an afterthought. This consensus represents significant industry maturation, moving beyond reactive approaches to proactive safety integration requiring cross-team collaboration and breaking down traditional silos.

### Multi-Stakeholder Engagement

There was strong agreement on the importance of transparency and stakeholder engagement. Both Meta and OpenAI described extensive consultation processes, with Meta’s 120-stakeholder engagement for AI labelling policies and OpenAI’s public consultation processes demonstrating industry commitment to external input and transparent documentation.

### AI-Specific Considerations

The discussion revealed consensus that AI development requires special safety considerations beyond traditional content moderation. Both AWS and OpenAI emphasised rigorous testing, external validation, and maintaining human oversight as AI capabilities advance, recognising that AI safety requires fundamentally different approaches from traditional digital safety measures.

## Key Disagreements and Tensions

### Content Moderation Strategies

The most significant disagreement emerged around Meta’s transition to crowdsourced community notes in the United States whilst maintaining third-party fact-checking globally. This represents a fundamental departure from approaches taken by other platforms like TikTok, which continues with traditional fact-checking partnerships and media literacy programmes.

### Scope of Safety Responsibilities

Collins’s emphasis on infrastructure-level safety considerations represents a broader conceptual approach than other speakers who focused primarily on user-facing platform safety. This highlights ongoing debates about where safety responsibilities lie within the technology ecosystem.

### Implementation Methodologies

Whilst speakers agreed on fundamental principles, they revealed different operational approaches: TikTok emphasises cross-team collaboration and automated safeguards, Meta prioritises stakeholder engagement, AWS focuses on infrastructure-level controls, and OpenAI emphasises external validation and transparency documentation.

## Critical Challenges Identified

### Technical-Policy Integration Gap

**Vinicius Fortuna**, Software Engineer at Google’s Jigsaw team, made a critical observation highlighting a fundamental implementation gap: “One thing I noticed is that I don’t actually see lots of engineers here. I also don’t see product managers but those are the builders and we can’t have multi-stakeholder governance without having the builders in the room.”

This comment exposed the disconnect between policy discussions and technical implementation, with engineers and product managers often absent from governance conversations despite being responsible for building the technology. Multiple panellists acknowledged this gap and shared experiences of trying to bridge technical and policy teams within their organisations.

### Balancing Innovation Speed with Safety Depth

A recurring theme was the challenge of balancing rapid technological development, particularly in AI, with comprehensive safety safeguards. Nicolazzi articulated this tension, noting the difficulty of balancing “speed of technology evolution with depth of required safeguards.” This challenge is particularly acute in AI development where companies face pressure for rapid product release whilst ensuring comprehensive safety measures.

### Regional Implementation Variations

The discussion revealed significant challenges in implementing universal safety standards across regions with varying regulatory maturity. Leite’s experience highlighted how regions with developing regulatory frameworks require fundamentally different approaches, including basic education and capacity building, raising questions about whether universal safety frameworks can work globally.

## Audience Contributions and Additional Perspectives

**Beatrice**, a lawyer and assistant professor, raised important questions about regulation as an enabler rather than constraint for innovation, suggesting that new regulatory frameworks like the EU’s Online Safety Act might push companies toward different forms of innovation.

**Dr. Nermeen Saleem**, Secretary General of Creators Union of Arab and CASA Consultative Statist with the UN, raised concerns about AI systems that appear to act beyond human control, highlighting international concerns about AI governance.

**Luis Adrian**, Professor and former Minister of Science, Technology and Telecommunication of Costa Rica, emphasised the importance of involving government and policymakers in safety innovation discussions to avoid reactive over-regulation.

**Andrea Vega**, working in tech policy and news integrity, highlighted the importance of partnerships with journalists and news organisations for maintaining information quality, adding another dimension to multi-stakeholder approaches.

## Emerging Solutions and Best Practices

### Cross-Industry Collaboration

The discussion highlighted successful cross-industry collaboration examples, including metadata frameworks and watermarking standards development. TikTok’s platform-agnostic family safety resources demonstrate how companies can work beyond immediate commercial interests to address broader societal challenges.

### Hybrid Automated-Human Systems

Multiple speakers emphasised hybrid approaches combining automated safety systems for scale with human oversight for complex decisions. This approach balances efficiency with nuanced judgement for contextualised content decisions.

### Transparency and External Validation

The trend towards transparency through system cards, public consultation processes, and external red teaming represents a shift towards opening safety processes to external scrutiny, building public trust whilst leveraging external expertise.

## Conclusions and Key Takeaways

The session demonstrated significant industry maturation in safety by design thinking, with consensus on fundamental principles despite implementation differences. The most critical insight was recognition that effective safety by design requires systemic changes extending beyond individual companies to encompass industry-wide collaboration, regulatory coordination, and multi-stakeholder engagement.

The absence of engineers and product managers from policy discussions emerged as a critical implementation gap that undermines effective translation of safety principles into practice. The conversation also highlighted unique challenges posed by AI development, where traditional safety frameworks may be inadequate for systems potentially exceeding human intelligence.

Moving forward, the industry faces the challenge of translating broad consensus on principles into consistent implementation practices whilst addressing regional variations in regulatory maturity and cultural contexts. Success will depend on bridging technical and policy expertise, fostering meaningful multi-stakeholder collaboration, and developing governance frameworks adequate for rapidly advancing AI capabilities.

The session concluded with clear recognition of the need for greater inclusion of technical professionals in policy discussions and continued cross-industry collaboration to develop effective safety standards that can keep pace with technological advancement whilst protecting users and society.

Agustina Callegari: Good morning everyone, welcome to this session on safety and innovation organized by the World Economic Forum at the Internet Governance Forum. My name is Agustina Callegari, I’m Digital Safety Lead at the World Economic Forum. I’m really pleased to have you today and I’m going to be moderating this session. I have to say that as part of the Global Coalition for Digital Safety, we have been reflecting on the importance of safety by design, because building safety into the design from the very start isn’t just a good practice, it’s essential for maintaining user trust and fostering responsible innovation. That’s why we are here today, to hear from a diverse group of BIMTEX experts. We don’t have one, we don’t have two speakers, we have five great speakers. So this is going to be a very interesting session. Let me start by introducing our panelists. We have Jeff Collins, Head of Global Trust and Safety at Amazon Web Services. I have Renato Leite, VP of Legal, AI, Privacy and Innovation at EAND. We have Peter Stern, Stakeholder Engagement Director at META. Val Ricci, Global Head of Outreach and Partnership at TikTok. And Rafaela Nicolazzi, Data Privacy and Consumer Protection Lead at OpenAI. I know we have an hour only and we have many speakers, so I want to start by asking each of our speakers a general question, because something that we focus when we work around safety or when we work around the Global Coalition on Digital Safety is on building common ground, building common language, because sometimes we talk about certain terms or certain challenges, but there is no agreement on what we mean by the thing. that we are trying to solve. So I want to ask the panelists what does build safety from their start mean to each of you and what is the biggest challenge that you face when you want to turn this principle into practice? So I think that I can start with TikTok there and then move in here.

Valiant Richey: It sounds great, thank you. All right, well, first of all, thanks to all of you for coming and thanks a lot to the government of Norway for hosting the IGF this year and for this panel and the invitation to participate. I think the topic of this is safety and innovation and I think this concept of safety from the start is one that we’re very committed to at TikTok and it really means ensuring that safety and security are built into our products from the beginning by design. And in other words, what that means is that the product teams building things and the safety teams need to work hand in hand from development to launch. Now, for those who might be less familiar with sort of how things happen in the world of tech, you know, the product teams are building TikTok or building the search function in TikTok. The safety teams are writing the rules and enforcing those rules across and sometimes the product teams will do things without trust and safety involved and that can cause problems and so what we try to do is bring them hand in hand and work together. And the goal is to build responsibly products that help people express themselves and discover and learn and build communities and we need to focus on minimizing risks when we do that. So, when we’re talking about building safely and safety and innovation, what does that look like in practice? Well, what it means is that TNS teams are working with our product and engineering teams to safeguard new product features and innovations as they are developed. So, let me give you an example. We recently started testing a feature called AI Alive, which lets you turn your TikTok stories photos into videos. Now, as part of the development of AI Alive, our TNS team built a multi-layer moderation strategy to make sure that safety checks happen at each step of the way. And we added AI generated content labels and metadata into the features so that people actually are aware that the content was created with AI. So, another example of how we do this at TikTok is by really focusing on building tools and resources into the product that our community can use. That might include tools like content management or screen time management tools, reporting options, account controls, like being able to set your… to private and so on. Now, along that journey, it’s hard, right? There’s a lot of challenges. Unfortunately, you’re facing motivated bad actors who are, they’re really intentionally trying to misuse the product at every step. And so you have to kind of take that into account and anticipating all those misuses of products is not easy. But for a product like TikTok, which features user generated content, one of the biggest challenges we face is balancing safety and fundamental rights like freedom of expression. We’re firmly committed to finding that right balance, but it’s not easy. Something like online hate, for example, can be highly contextualized and nuanced. It’s really difficult to know where to draw the line. So sometimes efforts have to be a bit more reactive in those contests. So that’s when we look at our community principles, our community guidelines, and outside experts to really help us build the system that works the best for our community. So we want to do safety by design, but we also have to recognize that it’s going to be difficult as we go.

Agustina Callegari: Thank you, Val. Peter, going to you, what does safety by design mean to Meta?

Peter Stern: Thanks very much. It’s great to be here. This is actually my first IGF, even though I’ve been at Meta for 11 years now. So it’s really exciting and great to see everybody. I’m on the content policy team at Meta. I run a stakeholder engagement function. My role is to develop relationships with academics, civil society groups, other thought leaders who can help us inform our policies, including our product policies, to build better policies and get better results for our worldwide user base. I want to talk about how we think about safety, both in terms of policies and products. But first, I would want to question any suggestion that safety and innovation don’t go together or are incompatible. Safety is really at the heart of everything that we do, and it’s always been that way. I mean, safety is one of of the core principles that underlie our community standards along with voice, dignity, and privacy. And we’re always thinking about safety as we build our policies. This goes back a long time. I was involved in the rollout of Facebook Live, gosh, seven, eight years ago now. And at that point, we faced a lot of novel questions about how we were going to moderate content, looking in on live streams that reached a certain level of virality or viewership. Safety was obviously incredibly important then and continues to be now. So, one way that we embed safety in our processes is by doing robust stakeholder engagement whenever we revise our community standards. My team maps out a strategy for who we should be talking to around the world, who’s written or voiced an authoritative opinion about a particular policy area, and then we’ll seek to engage with them to get their input and feed that into the policy development process. And by talking to the right people and fleshing out these views, I think we help to identify the impacts that we will have on our users, which I think is one core element of what goes into making up an approach to safety and something that’s very important to us. The policy development process then concludes through a kind of a company-wide convening that we call the Policy Forum and is then rolled out in the form of a public change to our community standards. That transparency, I think, is also an important element of safety that we’ll probably be talking about more today. Shifting gears here, just because I know that time is limited for everybody, in the world of, for example, the development of our policies around AI systems, safety is also fundamental. You know, META deals with issues of safety and AI at a number of different levels. As I think people know, we develop our own foundational models that we open source and make available to others to use. We also create our own services that are fine-tuned for our users using AI, and then we host AI-generated content on our services. So, at each of these elements, we have to take into account issues of safety. Are we using diverse and representative data to train our models? Are we providing information to developers who are using our models to create their own products so that they can build in safety and they have a sense of the guardrails that we think are important? And then how are we testing and red teaming and seeking to safeguard, build in safeguards when we create our own products? And how do we identify? AI-generated content that other people might want to put on our services in a way that people feel is giving them sufficient information but not unduly intrusive. So lots more to be said, but that’s a kind of an overview of how we think about safety in this setting.

Agustina Callegari: Thank you, Pete. Jeff, we want to address a question.

Jeff Collins: Sure. First, thanks to the World Economic Forum for putting this together, and thanks for all of you for showing up. As someone who has been to many of these, and Peter’s an OG in the trust and safety space, so I’m surprised you haven’t been to IGF, but this is a pretty good turnout for late in the week, so thanks for coming. So I’ve been working in the trust and safety space for a decade, and this topic is really near and dear to me, because for a trust and safety practitioner, really at the core of our work is to embed a safety mindset early in the design process. I have worked at a teen app called After School at TikTok, where people like Val are picking up on the work we started there, and now at AWS, cloud infrastructure provider. And for me, an important component and something I want to introduce to you here today is that we need to think about safety across the entire tech stack. A lot of times, people think of trust and safety with respect to user-generated content companies like Facebook or Twitter, but it’s really important that we understand across the entire stack how abuse can occur and how we need to think about safety often and early. So being at AWS has really widened my aperture in how I think about safety. As I said, we’re a cloud infrastructure company. What that means is we have a large, diverse customer base. We provide the underlying technology for our customers to power their services and applications. So you might think of a cloud infrastructure provider as providing storage, but we provide compute, data analysis, AI, many other facets, and really, we help power companies’ businesses. Now our customers from AWS, they are ultimately responsible for what they do in their AWS environment. So they’re responsible for complying with laws, complying with our acceptable use policy. And what we try to do is to help empower them to build safe, secure, and trustworthy services and applications. So how do we do that? Our trust and safety team at AWS works to minimize and mitigate harms and mitigate abuse across the AWS network. And this could be things from security threats, something like a DDoS attack, to content abuse. On using AWS to put out illegal content like terrorist content, for example. Now if we face true bad actors, a bad actor who compromises a customer’s account and launches a DDoS attack, we will take action to mitigate. But if we have a good, trustworthy customer who has a problem, we really work to try to help them address the problem. We try to help them diagnose it at the root cause and then figure out how they can prevent it in the future. So as an example, we might dig into a case where there’s ongoing internet abuse, find out that a customer didn’t patch a security vulnerability. Their account was compromised and then used to illegally mine cryptocurrency. In a case like that, we would work with them to understand what happened. And then there are a number of ways we try to help them prevent the problem in the future, really design safety into their product. So we can do that through guiding them to use AWS services. We have over 200 of our own services, and many of them are really useful in the trust and safety space. You may have heard of Amazon Recognition, which is image and video detection. We have other tools like Amazon Transcribe that can help detect and block language. And so we will guide customers to use those solutions. We also have AWS Marketplace, which is a way that customers can use independent software vendors’ products in their business. So you might be familiar with products like Thorn’s Safer Tool, which is used to detect and prevent child sexual abuse material, or CSAM, entities like Cohere, WebPurify, CheckStep. Those are all in AWS Marketplace. And then we also have a partner network where we have consulting partners with AWS like Accenture, Deloitte, Cognizant, and they help our customers integrate Amazon services like recognition into their businesses. And then finally, we launched something called the Trust and Safety Center, which is a hub to provide information about how to report abuse to our AWS Trust and Safety team, how to troubleshoot certain types of abuse and how to integrate best practices into your work. So I think I’ll save AI for later, but that’s just an overview of how we think about safety at our layer of the tech stack where we are not directly putting out content, but we have a whole range of customers across government, private sector, civil society that are using AWS and we try to work with them to help them integrate safety into their products.

Agustina Callegari: Thank you, Jeff. We are going to be talking a bit more about AI later, but now I want to give the floor to Renato.

Renato Leite: So first and foremost, thanks for the World Economic Forum for organizing the panel, my fellow panelists. It’s an honor to be here at the IGF. Just before, since we have such a crowded room, can everybody hear us okay? Everybody in the back over there? Okay, I know it’s a little bit packed. I know, so it’s okay. Jokes apart over here. So just a little bit of an introduction. I work for ENs and I know that it’s not a company as famous as some of these companies over here in the Western world. I am based in the Middle East, I’m based in Dubai. So ENs is the holding company of a number of organizations that operate greatly in the region, North Africa, Asia, and Europe and expanding to other parts. So basically, it used to be a telco that, such as many telcos in the world, is starting to attack and it does everything. It does from infrastructure, internet services, satellite, cables, robotics, health, finance, payments, entertainment, even hotels. Every time that I talk to a team, I find something new that we do. And as many companies, it’s also decided to turn into an AI-first company and comes with the challenge that we’re coming over here. So very, very specific to the question, the first question over here, how do we see safety from the start and what are some of the challenges that we see to turn into those principles, those that can be unsafe, into practice over here? Something that is very, that ENs saw from the beginning is that guaranteeing safety, what we call here responsible AI in the domain, does not mean that you cannot drive innovation. Even if you see my title, I don’t like titles of those who know me, but it’s about AI, privacy, and innovation. The same time that you can promote AI and AI-first, that you can guarantee privacy, privacy is my main background over here, you can also foster and drive innovation. And that it goes on having a very well-structured and loyal governance processes and policies inside of the company. We can talk a little bit more about that in specific questions, but one of the main challenges, we see a lot of companies doing that. We do, like, we have a good governance and AI, we have a bunch of committees inside of the companies, everybody has chime in, everybody has a say. In practice, it’s very, very hard. And so one of the main challenges that we see first is the lack of expertise. And I’m not talking about engineers, computer engineers that know how to do large language model, multi-modal reasoning and other things. All of the engineers that work at AWS and others doing this marvelous project that we all of us use, is basically saying, what is responsible AI? What is safety from the beginning? What is the things that you have to know in order to be able to build these safe products? So some of the, The first, and also the habit of teaching, is how to upskill people to know what safety means from the beginning. That when they are doing their day-to-day activities, when they are developing products, services, solutions, regardless of the sector, regardless of the area, if it is internet or not, they know what they’re doing. They’re not being say, I’m just doing this because there is a policy that needs to be enforced. This is something that we are working on from the beginning. And that it means also alignment and communication among all of the teams. We need to have a really, really strong buy-in from the top down in order for the team to say, okay, we’re going to take the time in order to learn, in order to develop this, in order to follow this principles and charter that was discussed. And that is also really hard. This is why we, going from challenging some of the solutions, we rely a lot on automation. And this is something that I’m really keen of is how we rely on automation and even AI for policy enforcements. So you discuss all the policies, you provide the awareness, education, and upskill units, but how to understand that there will not be some type of bypass, most of the time because there is no knowledge of all of the processes that need to be followed, how to have AI even to help on that. And so this is what we call, just to finalize, not only talking about safety by design, but governance from design. So every single stage from deployment, from ideation, from development, from risk assessment, from deployment, from monitoring, it have a governance as it can be as automated and also can be embedded into the expertise of all of our employees. We can talk a little bit about what we have been doing internally, but just from the challenges and how to turn into practice. Thank you.

Agustina Callegari: Thank you, Renato. Rafaela?

Rafaela Nicolazzi: Okay, last one to answer this question. I think my fellow panelists already addressed most of the things companies do to address safety by design, but I want to perhaps to start with a little bit of my background. I work based in Brussels, so one of the things that I do a lot is to engage with regulators all over Europe and beyond. So I wanted to answer this question through those lenses, and to me building safety from the start means not waiting for regulation or crisis to come up. It has to be a proactive commitment, so it’s taking responsibility from inception in every product, as my fellow colleagues here already said, and not treating safety as an after-the-commitment or compliance checkbox, which happens sometimes when we see, like, for example, small organizations. I could speak about safety for many, many hours on this. We are almost like 20 minutes away, or no, 35, but I want to highlight four things that we’re doing at OpenAI to bring this safety by design into practice. And number one, it starts with what does it mean, safety by design. One of the key documents that we publish, it’s called Model Spec. I wonder how many of you may have seen that already. So it’s this public commitment that sets the expectations of how we want the model to behave, right, and grounded into societal values. So it’s very inspirational. It’s how we want our AI tool or models to really engage with society. Number two is doing risk mitigation. For that, we have something called the system cards, and the system cards is how we’ve identified, tested, and mitigated potential harms through bias, hallucinations, misuse, and so it goes. So it is basically what we do after we publish or release either a big model or a new feature into the things that we do. Number three is what I call threat monitoring, and that’s translated into preparedness framework. another document that OpenAI released. And it really evaluates how, it outlines actually how we track, evaluate, and stress test what we call catastrophic risks, like misuse in biosecurity. And last but not least, to me, I think one of the most important ones that really speaks to my heart, which is being transparent about all of this. Right, so everything we do, all of these policies, safety measures, how we define what is risk, what are the risks that we think are most important that should be assessed by our safety teams, is shared openly. And the reason is to really invite critique, because we really believe that building safety is a collective effort, right? One example of this is model spec. We allow this for public consultation, for feedback for the general public for two weeks. And then we publish the model spec, the first version of it. And then goes to your last point of the question, which is, what is the challenge? And to me, the challenge is balancing the speed of how this technology is evolving so, so fast with the depth of the safeguards that we have to provide. Right, so we know that sometimes regulation and enforcement can go as fast as how those paradigm shifts are happening in the AI era. So it’s definitely how to balance those two together.

Agustina Callegari: Thank you, Rafaela. Well, we have covered a lot already, but we have also here different practices and different perspectives from a term that could seem very straightforward. When we think about safety by design, we think that it needs to be in bed from the beginning, but we have here different perspectives. And to pick up on something that we’re mentioning here, firstly, we need to recognize that trust and safety is a space that is continually changing and it’s an evolving practice and an evolving discipline as well. And there were a couple of things that came up. from the conversation that were related to building trust and safety not only for users but also within the companies. There was a need of working together with all the teams inside the company but also with other stakeholders in terms of building interventions that embed safety from the beginning. We also think the importance of not differentiating safety and innovation because they come together and also picking on some of the points that Rafaela made about the importance of transparency and balancing the speed. So that’s where we are right now but now I want to go deeper into some of the things that you have mentioned so I will do a follow-up question to each of you. We still have half an hour but we also want to allow the audience to make questions so please be prepared for that as well. So I will start with you Val because I will follow the same order. We know of course that TikTok reached a massive youth audience so what unique safety by design approaches do you employ for children or other vulnerable groups?

Valiant Richey: Well thanks a lot and you know it’s funny because I think TikTok has this long-standing narrative and reputation as being a place that teens love but I think a lot of people would be surprised to learn just how broad the audience is on TikTok. In fact for example in the US our average age is over 30 and my own feed is chock full of parenting advice and cooking recipes and content posted by grandparents. So you know I think that it’s really important to think about the holistic audience as we think about safety not just our youngest users but yes teens around the world love TikTok and we know that and so for years we have approached safety by design relating to teen accounts differently. We build the strongest safeguards into teen accounts so that they can have a positive experience in a safe environment. So for example accounts for teens under 16 are automatically set to private along with their content so their content can’t be downloaded. It’s not recommended in the For You feed and so on. Direct messaging is not available to anyone under 16. I’ll tell you that’s really annoying to my 15 year old who wants to direct message. Every teen wants to direct message but the fact is we made a decision that direct messaging is one of the riskier areas in social media and so for under 16 we don’t allow it at all. Every teen under 18 has a screen time limit automatically set to 60 minutes and only people over 18 are allowed to live stream. So these are some of the things that we have built into the backend to really ensure those safeguards are robust for our youngest users. What we want to do then is that you look at so those are like I said backend sort of product features but we also want to empower our community and particularly parents and guardians. So we’re looking at a holistic response here, right? We give opportunities to give agency to our community and so safety by design really doesn’t just mean those backend features but also those other things that people can use themselves and we recognize that every teen is different, every family is different. So we’ve continued to enhance our family pairing feature which is a feature that allows me to link my account to my teen’s account and set time limits and set times when he can’t use TikTok if I want to. And that I think really allows parents some control over what their teens are doing on TikTok, allows them to block their teens from using the app at certain times of the day for example or setting screen time limits, really helping families customize their experience beyond those robust guardrails that I already described. Finally on this, we really want families to talk, okay? I think we spend a tremendous amount of time, and we should, on thinking about what we can do in this space. We also talk with governments a lot about what they need to be doing and to support this environment. But we also see families and parents and guardians as critical stakeholders in this conversation. And so we wanna make sure that they’re supported in having conversations about digital safety. We also know it’s really hard to have those conversations. I have three boys, super hard to talk to them. They don’t wanna talk, or they wanna talk at midnight when I wanna go to sleep. You know, those things, it’s difficult to start that. So we developed this Digital Safety Partnerships for Families, which is a conversation starter. And we built it through consultations with NGOs, through consultations with teens, with families, and got advice about what helps. Things like, come with an open mind. Be available when they want to talk. When things go bad, don’t freak out, right? So those types of advice that we put together, and you can download it on our website, and it’s not TikTok-specific. It’s for social media generally, because we wanna support families in those conversations. So that’s a little bit about how we approach it. Thanks.

Agustina Callegari: Thank you, Val. Yeah, we’ve been discussing with the coalition the importance of designing targeted interventions, because there is no one-size-fits-all interventions that we need to specifically look at targeted group children and other stakeholders. And building also on your point on partnership, and also, Peter, on the point you made about stakeholder engagement, I would like to ask you to expand a bit more about how to engage with other companies or governments to align on evolving safety practices without undermining innovation, for example.

Peter Stern: Sure, so, okay. So let me give you an example here that I think is illustrative of the practices that we’re talking about. The issue of transparency and how we reveal that something may have been generated by AI to people is an extremely important one. At the moment, we take a scaled approach to this issue. And I wanna kind of tell you briefly what our approach is, and then back up and tell you how we got there, which I think sheds light on how we operate. So currently, we put watermarks, invisible watermarks on content. that is generated using our own AI tools. We also will attach certain types of labels to content that people post on our services that have been generated using AI in part or in whole. And we also have in our policies the option to apply a more prominent label if we determine that content that has been generated by AI could be highly misleading and would be a matter of public interest, potentially putting people at risk. So that’s sort of the bumper sticker version of what the policies are. There’s a lot of nuance, but tracing back how we got there, we’ve had a policy on what we called manipulated misleading video. Yeah, manipulated misleading video as going back to 2020. And a case went up to our oversight board, which is an independent oversight mechanism that we created a number of years ago. And the oversight board told us some important things that we really took to heart in this space. One is that we needed to update the policy because it didn’t cover enough different types of content. We thought that made sense. The previous policy was focused purely on video. They also said, interestingly, from the perspective of freedom of expression, that they wanted us to focus on labeling and transparency rather than taking a heavy-handed approach that would have involved removing a lot of video that was created or other types of content that was created by AI. We thought both of these inputs were very important. And that kicked off a policy development process for us. We went out and talked to 120 stakeholders in 34 countries to get their input on how we should be achieving these. balances and drawing these lines. We also did extensive opinion survey research with 23,000 respondents in 13 countries, and we learned some interesting things. There was support for labeling. There was particular support for labeling in high-risk settings, and there was also a lot of support for self-disclosure, people putting labels on themselves, which is something that we also thought was important. So, we took this all into our policy development process and came up with the approach that I’ve described. So, already we’re talking about multiple, multiple touch points with experts and civil society. There’s also an important industry component, because in order to be able to identify content that has been generated using AI off of our services, we need to be able to interpret certain types of metadata so that we can then apply labels. So, we worked closely with a consortium of companies under the aegis of Partnership for AI so that we could develop a common framework in that area, and we also used Partnership for AI’s best practices in approaching our own solution on watermarking. So, it’s a work in progress. These are not perfect solutions. There are ways around many of these tools, and we need to continue to be vigilant about that, but the approach that we’ve come up with is broad-based, and I think is helpful in understanding how we’re going to tackle these types of problems in the future.

Agustina Callegari: Thank you, Peter. And following up on the AI aspect that you briefly mentioned, but also Jeff mentioned before, Jeff, could you expand a bit on how you operationalize safety by design, given the scale and the speed of AI?

Jeff Collins: Sure. I want to start by picking up on something you mentioned, Augustina. You said that trust and safety is continuously evolving. That’s very true. We need to stay attuned to what’s happening, but it’s also very important to recognize we have learned a lot of lessons over the last decade as trust and safety practitioners, and we need to use those lessons in our work. With the really rapid ascent of Gen AI, I think what we have seen is across the tech industry, companies trying to figure out where they place trust and safety, how it fits in with responsible AI. comes from, its background is academia and there’s a lot of overlap between the two, but there’s not a ton of clarity. So one thing I’ve tried to do at AWS is really make sure that we take this lesson over the last decade that Val touched on in the beginning, we need to break down silos and we need to integrate some of our lessons learned from trust and safety into our work. So I’ll touch on one thing that we do in the AI front. So AWS Bedrock is our service that allows customers to leverage foundational models from many different companies in their own work and we try to make it simple for customers to integrate safety into all cycles of the AI process from design and development to deployment to operations. So in Bedrock we built in the ability for our customers to use a number of safety controls. These are things like input filtering to make sure that if you don’t want to allow a user to be able to input something, certain language, say around child sexual abuse material, you can prevent that. Output filtering, PII detection and redaction. There’s a host of different safety things or steps that that our customers can take and in doing that they can kind of tailor their products, their services, and apps for to align with their own values. Something else we did I’m really proud of is we built in in Bedrock CSAM detection. So we use hash matching technology to be able to detect potential CSAM in any inputs and then when we do detect that and we do it with a 99.9% accuracy, we block the content, we automatically report to NECMEC, the National Center for Missing and Exploited Children, which is the clearinghouse for CSAM in the United States, and then we advise the customer. And the reason I mention all of those tools is because we developed those by having our trust and safety team work closely with our Gen AI teams. And I think this is similar, I don’t work at other companies, but just having talked to people in the industry, I think that I face similar situations where AI teams were working fast and furiously. Engineers who have been working on machine learning and AI for a long time to get products out and we really wanted to make sure that we did learn this lesson from the past that we need to be careful and not rushing things out too fast, but rather build safety into the products. With that said, I will say that this is still very new territory. As OpenAI well knows, the pace at which new models are coming out is just a blistering pace, and we’re still figuring out across the industry what exactly safety means. For me, what I would like, I would like us to get to a point where, really going back to your primary initial question, where safety is considered a utility, where users and customers of AWS and others really expect safety to be built into products and they’re essentially demanding it. So I’ll stop there.

Agustina Callegari: Thank you Jeff, and I think we can expand on that if there is time. But now I want to go to Renato to ask you a bit more about the regional perspective, and especially in a region with rising digital access and more regulations. So if you can expand a bit on that.

Renato Leite: So in the region, so mainly, even though we operate here in almost 40 countries, our main market is still the Middle East, so the Gulf region, and most of the countries, they either are just enacting some regulation or discussing the regulation, or the regulation is there but it’s not being enforced. So there’s not a lot of a stick to push people to do things. And it’s, and that also brings out what is something that I said in the beginning, the level of maturity in the discussions and the expertise and how to do things is very low. I kind of compare, so I’m originally from Brazil, as Rafael is also from Brazil, and we had, and I see some Brazilian friends over here, you may remember when it was in Brazil like in 2015 when we were discussing the general data protection law, and it was very going to the basic concepts, the basic ideas. and it’s kind of hard when you are discussing with the teams in order that we have to follow x, y and z and guarantee, we say here, legal assurance when there is not a level of material. So my team, the team that have opportunity to manage overseas legal assurance for the entire group, for AI, for privacy and for innovation, including intellectual property over here, and within this field of safety and mostly talking about safety, not long ago we developed a responsible AI framework. As a lot of companies do, we went there, we published and I must say we failed. And we failed because it was very beautiful, it was very colorful, it shined a lot of there, everybody could go, you could do marketing, but the principles that were there did not really reflect how the maturity of the region, it did not really reflect how the Intel works of the organizations. So we said we need to come back. So we went back, we went to talk with all of the areas of the company, all of the verticals, we went to a lot of some of our major jurisdictions, we tried to internalize some not only global standards, some regulatory concepts into this framework such as explainability, which is not the same as transparency, and it might be something that we take it for granted in some jurisdictions such as in Europe when you’re talking about that, but how to explain what is the difference between transparency and explainability and how to build that into the AI solutions that we have internally and also provide a service, it’s kind of hard. But when you go to this understanding, so these are the key differences, it’s not because it is in the regulation, because as Agostino said, it is a nascent regulation in the area, but not only because it is in the policy, but how that can not only drive innovation, how can only that achieve the goals of the organization in the region, we are very much driven about reputation, about brand reputation, brand value, that has a big impact and also helps to get the buy-in from the different areas, and having an interdisciplinary review and assessment of all of the solutions that we have over there. So, we also created an AI governance committee, which I’m a part of, but we even changed the mandate of the AI governance committee to the point to say, to discuss it from the ideation phase, are these ideas that are worthwhile pursuing from a business perspective, but what are the safety measures that we not only want, but we need to embed from the design before we put that detail from paper and we start developing. So, that’s what I’ve been doing over the last years over there.

Agustina Callegari: Thank you, Renato. You didn’t only provide examples on MENA, but also in Brazil, so thank you for that. So, Rafaela, over to you. You already mentioned some principles that you are focusing on OpenAI. Now, I would like you to ask you for maybe a concrete example on how OpenAI brings safety air force into real practice.

Rafaela Nicolazzi: Absolutely. So, first, perhaps I can start bringing a little bit of context, so giving one step back about how do we see this fast-paced evolution of AI technologies, and many times nowadays when you think about AI, I think most of us have in our minds the chatbots, right, which are this AI technology that allows you to engage in network conversations, it gives you fast responses with prompts that you give to the to the modern sound source. But to us this is just the first level of AI. The second one that we were already talking about for a few years, we were calling that Reasoners. And Reasoners is basically AI that can think for longer. It’s AI that would also have a chain of thoughts and will be able to allow you to challenge how it reached that response. And because you were thinking for longer, it got much much better on addressing questions on the fields of science or mathematics. And when you saw that that was coming up, we saw that that would be a big leap in the capability, right? It would be a huge paradigm shift. And when we first introduced this to society, it was end of last year, around September. And we called this model O1. We know it might be a bit tricky to follow all the names. O1. And because we knew it would be a very capable model, we also wanted to ensure it would be the safest model we would have in the market. So we did five things to make sure that they were addressing those safety concerns to society, to regulators, to policy stakeholders, to go in the right direction. Number one was the very rigorous testing. So we conducted public internal evaluations to assess disallowed content, demographic fairness, hallucination tendencies, and dangerous capabilities. The second thing we do, it was to do external expert red teaming. I heard my colleagues already talking about it here. But one thing that we have very clear at OpenAI, especially because we’re rooted in a research lab that started with a bunch of PhD together, understanding, trying to understand what the future of AI would look like, is that inside your own lab, you can’t understand and assess all the risks. You need people from the external community to do that with you. And for O1, we had partnered with more than 10 external organizations all over the world to really challenge the model to identify risks that perhaps we wouldn’t be able to see by ourselves. So that was number two. Number three, it was to bring more advanced safety features. So we implemented new safety mechanisms, like blacklist and safety classifiers, to really guide the AI behavior to prevent any type of inappropriate content generation. Number four, I already said in the principles, but how we did is in practice. So it’s showing the work. We’re very committed to that. committed to be open about the model capability, how is the risks, how we’re mitigating the risks, and then, of course, the publication of the safety cards that I invite you all to check it out if you haven’t read them yet. And last but not least, what we’ve done as well was partnerships with governments. At that time, we had the U.S. and the AEI Safety Institute. We gave them early access to our models, so they would give us feedback, and that really allowed us to set the precedent before we released this to the public.

Agustina Callegari: Thank you, Rafaela. Now we still have ten minutes, so I’m going to open the floor for questions or comments. There is a mic this side, so if there are questions, I will ask you to stand up. Good. And we also have remote participation, so my colleague Judith, also from the World Economic Forum, is monitoring the chat in case there are any questions or comments from online participants. Great.

Andrea Vega: Hello. Thank you for this panel. It’s been really informative. I think it’s nice to see all the industry folks in the conference in one panel. My name is Andrea Vega. I work in tech policy and news integrity. So my question is, you know, being here at a multi-stakeholder conference, some of you talked about the kind of engagements and outreach you do with civil society and academia, but a key tenet of digital trust, as named in the panel, and to create a safe experience is to ensure that the information you host is factual, true, and thus not harmful. And often promoting that quality information comes from working with journalists. So can you each talk about your approach to, one, hosting news online, and two, the kind of partnerships that you have with news organizations and media, if any, that inform your work? Thank you.

Agustina Callegari: Thank you. I think we can collect a couple of questions and then address everything at the end. We cannot hear very well from here, right? So I hope we can address all your questions, but we are checking on the screen. That’s why you will see us looking at the screen to make sure we are… understanding your question so please go ahead.

Vinicius Fortuna: Hi my name is Vinicius Fortuna I’m a software engineer and I lead at Google and I lead the Internet Freedom team at Jigsaw and this is my first IGF. One thing I noticed is that I don’t actually see lots of engineers here. I also don’t see product managers but those are the builders and we can’t have multi-stakeholder governance without having the builders in the room so I would like to like what we see is like legal and policy so I’d like to ask like all our companies to like can we please bring the builders to this conversation and I know that like big tech engineers live like a well-paid and live in a bubble quite often and they’re not even thinking about these issues but so we our companies need to actively proactively put the incentive in place to bring those people into the conversation and now in the same time like remove the hurdles. I know that at least one company here that banned their engineers and product people from going to a lot of these events in the past. We used to have very good and productive conversations and I would like I would love to see that more. I think this will help having more productive conversation with everyone and also build trust on your companies and our companies. I think it will help everyone.

Agustina Callegari: Thank you very much. I think Jeff can quickly address that.

Jeff Collins: I love that question. I’m just gonna give you a quick response. It brought a smile to my face because I come from a policy background but now I’m surrounded by engineers at AWS. Last year it was two IGFs ago but it was really a year and a half ago in Kyoto. I made sure to bring the senior engineer on my team to IGF because she kept asking like what is our policy team doing and I brought her there and the first day or so she was so confused because she went to these panels and she wanted to have ABC leads to solution DEF and she didn’t understand. and, you know, like how this worked. But by the end of the time, she met a bunch of people and she really could understand better this is part of the policymaking process, but I couldn’t agree with you more. And I brought her because I wanted to help close that gap. And I think that’s important at conferences like this, but also in companies. So when I was at TikTok, we worked very hard to not just hire people from Meta, we needed to hire people, we needed to hire people who had really good experience. No, we had a lot of people there and they were great because they knew the history of trust and safety operations but we wanted to, so they were like high caliber, everyone wanted them. But we also hired people from academia, from media, and of course we needed engineers. But you’re absolutely right, there needs to be more of that. Now, if you interact with the ICANN world here, you do start to come across a lot more technical people, but I couldn’t agree with you more. So let’s just think about ways that we can do that at this conference and elsewhere.

Vinicius Fortuna: Yeah, that’s a good point. Like removing the barrier inside the company is also important because like, as you all know, like engineers and policy people are very separated.

Agustina Callegari: Very true. Thank you. Thank you. Being very mindful of time, we only have four minutes and there are three people waiting to ask questions. So I kindly ask you to be very brief with the first question and also your question. Thank you very much.

Audience: Yes, can you hear me now? My name is Beatrice, I am a lawyer by training, but I’m also an assistant professor in law. So that’s kind of the perspective I’m coming to the conversation. One of the things that I think Peter mentioned at the beginning, but I also kind of referring to what Rafaela was saying, is kind of this discussion of perhaps a conflict between safety and innovation. But a lot of what we hear in terms of kind of the conversation is perhaps a tension between innovation and regulation. So kind of, oh, regulation is going to hamper innovation or prevent innovation. I also believe, as a lawyer, that regulation can foster innovation and we are seeing now kind of a new wave of regulatory efforts from countries like the EU, but also the UK, where I’m based, trying to kind of foster online safety through regulation. So I’m interested in hearing from you in terms of whether to what extent these new regulatory frameworks, these laws, have been pushing you towards different forms of innovation, and the Online Safety Act is one of them, but I think we have other examples. Is it kind of an enabler, a constrainer, is it selling the floor, a ceiling? How do you see regulation in this space of kind of internal policy? Thank you. Hi. Good afternoon. I’m Dr. Nermeen Saleem, Secretary General of Creators Union of Arab, a CASA Consultative Statist with the UN. First of all, I would like to express my sincerity to the distinguished speakers for their insightful presentation. Allow me to raise a question stemming from a news article I recently came across regarding an AI application. While I do not specialize in technical dimensions of AI, I will do the best to convey my concern. The article discussed a machine learning-based system that in its practical application appears capable of developing data independently without human input. In some instances, the system reportedly acted beyond human control, no longer limited to the data it was trained on, even surpassing intended human directives. My question is, doesn’t this present a significant risk that all professionals and stakeholders of AI field must seriously consider? And the other one, in your expert opinion, what measures can be taken to address such concerns? to share their concerns and ensure the responsible AI development. Thank you.

Luis Adrian: Hello, good morning. My name is Luis Adrián. I come from Costa Rica and I am a professor right now. However, I am a former Minister of Science, Technology and Telecommunication of Costa Rica. And my concern is simple. When we have a digital trust, we can have a safe innovation. However, my concern is how can we involve more the public policy makers and the government. Because if you don’t involve all of them, the response to make more safety is to generate regulation. And the over-regulation can produce that we stop the innovation. So I would like to have your point of view. Thank you very much.

Agustina Callegari: Okay. I know we are running out of time and we have questions about partnerships and involving more the public policy makers. We have questions about regulations and also about responsible AI development. So I ask to my panelists if you want to pick up on any of these points because we need to wrap up the session now as soon as possible.

Peter Stern: Why don’t I take the first question because I took that to be primarily directed to misinformation on the platform, which is something that is a concern to journalists but also to all of our users who have told us they want to see more context around certain types of content. We have a policy directed to misinformation and harm. And under this policy, we will remove content that is false when we determine that it could present real-world safety risks to journalists. people. And this is an area that started off as a relatively small subset of content on the platform. During COVID, it became much larger. That continues to be a policy that is on the books and that we inform regularly with input from local people on the ground. We also have policies directed to providing the type of context that I mentioned. Traditionally, we have worked with third party fact checkers. We continue to work with three PFCs outside the United States. In the United States, we announced earlier this year that we’re going to be moving to a different crowdsource model, which has a lot of virtues to it in terms of the way that observations about the truth or falsity of content are reported. And we’re hoping that this system will be more broad based, more legitimate, and also more scalable. There’s a lot more to be said about that. I know there’s a very robust conversation about it, but I’ll leave it at that. Community notes in the United States and third party fact checking remains in place around the rest of the world. Thank you.

Agustina Callegari: We can have one or two more minutes.

Valiant Richey: Great. I’ll be real fast. Like Meta, TikTok has a robust misinformation policy framework. We also do a lot of media literacy to make sure that our community is receiving information with a discerning eye. And we collaborate with fact checkers in order to create that content. So, it’s a pretty robust system. And then we also obviously have a lot of entities presenting news from citizen journalists, so-called to establish news organizations. And we work closely with them to make sure that that content is in line with our guidelines. Regarding the point around building engineers and product to these meetings, we actually have a network of 10 safety advisory councils around the world and a youth council. And we always bring our product people to those meetings because they are amazing opportunities for them to hear from outside experts and integrate that knowledge directly into their work every day. So, full support for that idea. It’s something we practice regularly. On regulations, the thing that we have found to be, I think, helpful in how we think about innovation is the risk assessments. They’re challenging. They can be annoying in some respects. But they are also good opportunities to rethink about how you approach that work and that safety work. And I think they’ve pushed us. And finally, on AI and responsible development, we have published our responsible AI principles, which guide our work on AI development. You can find them on our website. And I think they guide our work every day there. So, something that we take very seriously and will continue to do so. Thank you.

Agustina Callegari: Thank you very much. Renato, final words?

Renato Leite: Very quick. Yes, over here. I’ve always wanted to work with engineers. So even in my previous organization the privacy team which I managed was very siloed. Privacy legal, privacy policy, privacy engineering and so on. We put them all together. So at a certain point I’m a lawyer and I had 10 privacy engineers reporting to me and working together. I had to find a translator. In my new organization we started to do the same. So it didn’t work like we were not speaking the same language at a point. When we started to really work together, bridging that gap, like we started to understand the language of the others and build a better product because of that.

Rafaela Nicolazzi: Very quick, and I’m not sure that’s the best closing, but AI is already smarter than us, right? And it’s just getting smarter and smarter. We already know this, like the whole AI community is already trying to address the problem of going to this breakthrough. And three things I wanted to say about this is the things that we have already using to deal with this internally is number one, governance. Number two, what we call model alignment. And number three, which is the most important one to address the question from the article, is always having human in the loop. I’m happy to go deeper into this, we don’t have more time. But yeah, just to say that it will be very hard for us to leave a room in the near future and think that we are smarter than AI around there.

Agustina Callegari: Thank you, Rafaela. Thank you, everyone. We need to follow this conversation. So I think we have seen how important continuing the conversations about safety is, and I’m hearing very loudly that we need to break silos, not only within the companies, within teams, and within the different communities from AI, from digital safety, and also build more partnership with civil society organizations, with other organizations, and also including users’ perspective as much as possible. So with that, I will close the session. I really want to thank all the panelists. I also want to thank the ICF and the business track for offering us this space. And yeah, thank you very much for being here as well. Thanks for watching and don’t forget to subscribe !

Agreement points

Safety must be embedded from the beginning of product development with cross-team collaboration

Speakers

– Valiant Richey
– Peter Stern
– Jeff Collins
– Renato Leite
– Rafaela Nicolazzi

Arguments

Safety must be embedded from product development start with cross-team collaboration

Safety is a core principle alongside voice, dignity, and privacy, requiring stakeholder engagement

Safety thinking must span the entire tech stack, not just user-facing applications

Governance from design requires automation and AI for policy enforcement across all stages

Safety by design means proactive commitment from inception, not waiting for regulation or crisis

Summary

All speakers agree that safety cannot be an afterthought but must be built into products from the very beginning of the design process, requiring collaboration between different teams within organizations

Topics

Legal and regulatory | Cybersecurity | Human rights

Breaking down silos between technical and policy teams is essential

Speakers

– Jeff Collins
– Renato Leite
– Vinicius Fortuna

Arguments

Need to bring engineers and product managers into multi-stakeholder governance conversations

Breaking down barriers between engineering and policy teams within companies

Need to bring engineers and product managers into multi-stakeholder governance conversations

Summary

There is strong consensus that engineers, product managers, and policy teams must work together more closely, with technical builders participating in governance conversations

Topics

Legal and regulatory | Development | Infrastructure

Transparency and stakeholder engagement are crucial for effective safety implementation

Speakers

– Peter Stern
– Rafaela Nicolazzi
– Agustina Callegari

Arguments

Need for diverse stakeholder input and transparency in policy development processes

Public consultation processes for policy development and open publication of safety measures

Breaking down silos within companies and with external stakeholders is crucial for effective safety implementation

Summary

Speakers agree that transparency in safety processes and broad stakeholder engagement, including public consultation, are essential for building effective and legitimate safety measures

Topics

Legal and regulatory | Human rights | Sociocultural

AI safety requires special attention with human oversight and rigorous testing

Speakers

– Jeff Collins
– Rafaela Nicolazzi

Arguments

Integration of trust and safety lessons into generative AI product development

Rigorous testing, external red teaming, and government partnerships before model release

Human-in-the-loop systems, governance, and model alignment to address AI capabilities exceeding human intelligence

Summary

Both speakers emphasize that AI development requires special safety considerations, including rigorous testing, external validation, and maintaining human oversight as AI capabilities advance

Topics

Cybersecurity | Legal and regulatory | Human rights

Similar viewpoints

Both companies implement comprehensive approaches to misinformation that combine content removal policies with fact-checking partnerships and user education, focusing on content that poses real-world safety risks

Speakers

– Valiant Richey
– Peter Stern

Arguments

Robust misinformation policies with fact-checking partnerships and media literacy programs

Removal of false content presenting real-world safety risks and provision of context through fact-checking

Topics

Sociocultural | Human rights | Legal and regulatory

Both organizations conduct extensive stakeholder consultation processes, including public input periods, to inform their policy development and ensure broad input into safety measures

Speakers

– Peter Stern
– Rafaela Nicolazzi

Arguments

Extensive consultation with 120 stakeholders across 34 countries for AI labeling policies

Public consultation processes for policy development and open publication of safety measures

Topics

Legal and regulatory | Human rights | Development

Both speakers acknowledge the challenge of implementing safety measures in environments where regulatory frameworks and expertise are still developing, requiring educational approaches and careful balancing of innovation speed with safety depth

Speakers

– Renato Leite
– Rafaela Nicolazzi

Arguments

Challenges of low regulatory maturity and expertise in Middle East region requiring basic education

Balancing speed of technology evolution with depth of required safeguards

Topics

Development | Legal and regulatory | Economic

Unexpected consensus

Infrastructure-level safety considerations are as important as user-facing safety

Speakers

– Jeff Collins
– Renato Leite

Arguments

Safety thinking must span the entire tech stack, not just user-facing applications

Governance from design requires automation and AI for policy enforcement across all stages

Explanation

It’s somewhat unexpected to see such strong agreement on the importance of infrastructure-level safety considerations, as discussions often focus on user-facing applications. Both speakers emphasize that safety must be considered at all levels of the technology stack

Topics

Infrastructure | Cybersecurity | Legal and regulatory

Regulation can be an enabler rather than constraint for innovation

Speakers

– Valiant Richey
– Audience

Arguments

Risk assessments as opportunities to rethink safety approaches despite being challenging

Regulation can foster innovation rather than constrain it when properly implemented

Explanation

There’s unexpected consensus that regulation, particularly risk assessments, can actually drive innovation and improvement rather than simply constraining it, challenging the common narrative of regulation versus innovation

Topics

Legal and regulatory | Development | Human rights

Brand reputation as a driver for safety implementation in developing markets

Speakers

– Renato Leite

Arguments

Importance of brand reputation and interdisciplinary review in driving safety adoption

Explanation

It’s noteworthy that in regions with less regulatory enforcement, brand reputation emerges as a significant driver for safety implementation, suggesting market-based incentives can work where regulatory frameworks are still developing

Topics

Economic | Legal and regulatory | Development

Overall assessment

Summary

There is remarkably strong consensus among speakers on fundamental principles of safety by design, the need for cross-team collaboration, transparency, and stakeholder engagement. Agreement extends across different types of organizations (social media platforms, cloud infrastructure, AI companies) and regions.

Consensus level

High level of consensus with practical alignment on implementation approaches. This suggests the safety by design principles are becoming well-established industry standards, though challenges remain in execution, particularly around balancing innovation speed with safety depth and addressing regional differences in regulatory maturity.

Different viewpoints

Content moderation approach for misinformation

Speakers

– Peter Stern

Arguments

Transition to crowdsourced community notes model in US while maintaining third-party fact-checking globally

Summary

Meta is moving away from third-party fact-checking to a crowdsourced community notes model in the US, while other platforms like TikTok continue with traditional fact-checking partnerships. This represents a fundamental disagreement on the most effective approach to content moderation.

Topics

Sociocultural | Human rights | Legal and regulatory

Scope of safety considerations across technology stack

Speakers

– Jeff Collins

Arguments

Safety thinking must span the entire tech stack, not just user-facing applications

Summary

While other speakers focus on user-facing platform safety, Jeff Collins argues that safety must be considered across the entire technology infrastructure stack, representing a broader conceptual approach to safety implementation.

Topics

Cybersecurity | Infrastructure | Legal and regulatory

Unexpected differences

Regional approach to safety implementation

Speakers

– Renato Leite

Arguments

Challenges of low regulatory maturity and expertise in Middle East region requiring basic education

Explanation

While other speakers discussed universal safety principles, Renato highlighted that regional differences in regulatory maturity require fundamentally different approaches, suggesting that one-size-fits-all safety frameworks may not work globally. This was unexpected as it challenges the assumption of universal safety standards.

Topics

Development | Legal and regulatory | Economic

Role of automation in safety governance

Speakers

– Renato Leite

Arguments

Governance from design requires automation and AI for policy enforcement across all stages

Explanation

While other speakers focused on human oversight and stakeholder engagement, Renato emphasized heavy reliance on automation and AI for policy enforcement. This represents an unexpected disagreement on the role of human versus automated decision-making in safety governance.

Topics

Legal and regulatory | Infrastructure | Cybersecurity

Overall assessment

Summary

The discussion revealed surprisingly few fundamental disagreements among speakers, with most conflicts centered on implementation methods rather than core principles. Key areas of disagreement included content moderation approaches, the scope of safety considerations, and the role of automation versus human oversight.

Disagreement level

Low to moderate disagreement level. Speakers largely agreed on fundamental safety principles but differed on tactical approaches. The implications suggest that while there is industry consensus on the importance of safety by design, there remains significant variation in how companies operationalize these principles, potentially leading to inconsistent user experiences and regulatory challenges across platforms and regions.

Partial agreements

Similar viewpoints

Both companies implement comprehensive approaches to misinformation that combine content removal policies with fact-checking partnerships and user education, focusing on content that poses real-world safety risks

Speakers

– Valiant Richey
– Peter Stern

Arguments

Robust misinformation policies with fact-checking partnerships and media literacy programs

Removal of false content presenting real-world safety risks and provision of context through fact-checking

Topics

Sociocultural | Human rights | Legal and regulatory

Both organizations conduct extensive stakeholder consultation processes, including public input periods, to inform their policy development and ensure broad input into safety measures

Speakers

– Peter Stern
– Rafaela Nicolazzi

Arguments

Extensive consultation with 120 stakeholders across 34 countries for AI labeling policies

Public consultation processes for policy development and open publication of safety measures

Topics

Legal and regulatory | Human rights | Development

Both speakers acknowledge the challenge of implementing safety measures in environments where regulatory frameworks and expertise are still developing, requiring educational approaches and careful balancing of innovation speed with safety depth

Speakers

– Renato Leite
– Rafaela Nicolazzi

Arguments

Challenges of low regulatory maturity and expertise in Middle East region requiring basic education

Balancing speed of technology evolution with depth of required safeguards

Topics

Development | Legal and regulatory | Economic

Key takeaways

Safety by design requires embedding safety considerations from the very beginning of product development, not as an afterthought or compliance checkbox

Cross-team collaboration is essential – breaking down silos between engineering, product, policy, and safety teams within organizations

There is no inherent conflict between safety and innovation; they should work together rather than be viewed as competing priorities

Different user groups require targeted safety interventions, particularly vulnerable populations like children and teens

Multi-stakeholder engagement including civil society, academia, governments, and external experts is crucial for effective safety implementation

Transparency in safety processes and policies builds trust and enables collective improvement through external critique

The rapid pace of AI development creates unique challenges in balancing speed of innovation with depth of safety safeguards

Regional differences in regulatory maturity and expertise require tailored approaches to safety implementation

Trust and safety is a continuously evolving discipline that must adapt to new technologies while applying lessons learned from past experiences

Resolutions and action items

Companies should actively bring engineers and product managers to multi-stakeholder governance conversations and remove internal barriers preventing their participation

Organizations need to invest in upskilling and education programs to build safety expertise across all teams, not just specialized safety departments

Industry should continue developing common frameworks for AI safety, including metadata standards and watermarking technologies through consortiums like Partnership for AI

Companies should implement automated governance systems and AI-assisted policy enforcement to scale safety practices

Organizations should establish safety advisory councils and youth councils with direct product team participation to integrate external feedback

Unresolved issues

How to effectively balance the speed of AI technological evolution with the depth of safety safeguards required

Addressing AI systems that may develop capabilities beyond human control or act independently without human input

Determining optimal approaches for involving government and policymakers without triggering over-regulation that stifles innovation

Establishing universal definitions and common language around safety concepts across different organizations and regions

Scaling safety practices effectively across diverse regional contexts with varying levels of regulatory maturity

Managing the challenge of motivated bad actors who intentionally try to misuse products at every development stage

Suggested compromises

Using risk assessments as opportunities to rethink safety approaches, viewing regulatory requirements as innovation drivers rather than constraints

Implementing scaled approaches to content labeling that balance transparency with user experience (e.g., different label prominence based on risk level)

Adopting hybrid models that combine automated safety systems with human oversight to maintain control while enabling innovation

Balancing global safety standards with local customization to address regional differences in regulatory maturity and cultural contexts

Creating family-centered approaches that combine platform-level protections with parental controls and educational resources

We need to think about safety across the entire tech stack. A lot of times, people think of trust and safety with respect to user-generated content companies like Facebook or Twitter, but it’s really important that we understand across the entire stack how abuse can occur and how we need to think about safety often and early.

Speaker

Jeff Collins

Reason

This comment reframes the safety discussion beyond just social media platforms to include infrastructure providers, introducing a systems-thinking approach that considers the entire technology ecosystem. It challenges the narrow view of where safety responsibilities lie.

Impact

This expanded the conversation’s scope significantly, moving beyond platform-specific safety measures to consider infrastructure-level interventions. It influenced subsequent discussions about different layers of responsibility and introduced the concept that safety isn’t just about end-user interfaces but about foundational systems.

Something like online hate, for example, can be highly contextualized and nuanced. It’s really difficult to know where to draw the line. So sometimes efforts have to be a bit more reactive in those contests.

Speaker

Valiant Richey

Reason

This comment honestly acknowledges the fundamental tension between proactive safety design and the contextual nature of harmful content. It challenges the idealistic notion that all safety can be built-in from the start and introduces the reality of reactive measures.

Impact

This comment introduced nuance to the ‘safety by design’ discussion, acknowledging its limitations and setting a more realistic tone for the conversation. It helped other panelists discuss both proactive and reactive approaches more honestly.

One thing I noticed is that I don’t actually see lots of engineers here. I also don’t see product managers but those are the builders and we can’t have multi-stakeholder governance without having the builders in the room.

Speaker

Vinicius Fortuna

Reason

This comment exposed a critical gap in multi-stakeholder discussions – the absence of the actual people who build the technology. It challenges the effectiveness of policy discussions that exclude implementers and highlights structural barriers within companies.

Impact

This comment was a turning point that shifted the conversation from theoretical policy discussions to practical implementation challenges. It prompted immediate responses from multiple panelists who acknowledged this gap and shared their own experiences trying to bridge it, leading to concrete suggestions for improvement.

We developed this Digital Safety Partnerships for Families, which is a conversation starter… it’s not TikTok-specific. It’s for social media generally, because we wanna support families in those conversations.

Speaker

Valiant Richey

Reason

This comment demonstrates a shift from competitive, platform-specific solutions to collaborative, industry-wide approaches to safety. It shows how companies can work beyond their own interests for broader societal benefit.

Impact

This comment influenced the discussion toward collaborative approaches and shared responsibility models, moving away from individual company solutions to industry-wide cooperation. It reinforced themes about partnership and stakeholder engagement that other panelists built upon.

AI is already smarter than us, right? And it’s just getting smarter and smarter… it will be very hard for us to leave a room in the near future and think that we are smarter than AI around there.

Speaker

Rafaela Nicolazzi

Reason

This stark acknowledgment of AI’s rapidly advancing capabilities introduces existential questions about human control and oversight in AI systems. It challenges assumptions about human-in-the-loop approaches and raises fundamental questions about governance.

Impact

This comment brought urgency and gravity to the AI safety discussion, moving it from technical implementation details to fundamental questions about human agency and control. It served as a sobering conclusion that reframed earlier discussions about AI governance and safety measures.

We went back, we went to talk with all of the areas of the company, all of the verticals… we tried to internalize some not only global standards, some regulatory concepts into this framework such as explainability, which is not the same as transparency.

Speaker

Renato Leite

Reason

This comment reveals the complexity of implementing safety frameworks across diverse organizational structures and introduces important technical distinctions (explainability vs. transparency) that are often conflated in policy discussions.

Impact

This comment added depth to the discussion about implementation challenges and introduced technical nuance that helped distinguish between different aspects of AI safety and transparency. It influenced the conversation toward more precise terminology and practical implementation considerations.

Overall assessment

These key comments fundamentally shaped the discussion by expanding its scope beyond individual platform policies to systemic, industry-wide challenges. The conversation evolved from theoretical principles to practical implementation realities, with critical interventions highlighting gaps in current approaches (missing engineers in governance, reactive vs. proactive measures) and introducing urgency around AI advancement. The comments collectively moved the discussion from a company-centric view to a more holistic, collaborative approach that acknowledges both the complexity of safety implementation and the need for broader stakeholder engagement. The progression from technical solutions to existential questions about AI capabilities created a narrative arc that grounded abstract policy discussions in concrete realities and future challenges.

How can companies bring more engineers and product managers into multi-stakeholder governance conversations?

Speaker

Vinicius Fortuna

Explanation

This is critical for effective multi-stakeholder governance as the actual builders of technology need to be part of policy discussions, but they are often absent from these conversations due to company barriers or lack of incentives

How can regulation serve as an enabler rather than a constraint for innovation in the safety space?

Speaker

Beatrice (assistant professor)

Explanation

Understanding whether new regulatory frameworks like the EU’s Online Safety Act are pushing companies toward different forms of innovation is important for balancing regulatory compliance with technological advancement

What measures can be taken to address AI systems that appear to act beyond human control and develop data independently?

Speaker

Dr. Nermeen Saleem

Explanation

This addresses a fundamental safety concern about AI systems potentially operating outside intended parameters and human oversight, which is critical for responsible AI development

How can public policy makers and governments be better involved in digital trust and safety innovation to avoid over-regulation?

Speaker

Luis Adrian

Explanation

This is important because lack of government involvement in safety innovation discussions can lead to reactive over-regulation that stifles innovation rather than enabling safe development

What specific partnerships do companies have with news organizations and journalists to ensure factual information hosting?

Speaker

Andrea Vega

Explanation

This is crucial for understanding how platforms maintain information integrity and work with media professionals to combat misinformation and promote quality journalism

How can companies better bridge the language and communication gap between legal/policy teams and engineering teams?

Speaker

Renato Leite

Explanation

This emerged as a practical challenge for implementing safety by design, as different teams often don’t speak the same language, hindering effective collaboration on safety measures

How can the AI community better address the challenge of AI systems becoming smarter than humans while maintaining human oversight?

Speaker

Rafaela Nicolazzi

Explanation

This addresses the fundamental challenge of maintaining human control and oversight as AI capabilities rapidly advance beyond human intelligence levels