Building Trustworthy AI Foundations and Practical Pathways

I give this example because I’m fairly confident that when you look it up and when you try it yourself it will work. And I know it will work, by the way. That is, it will fail rather. On the current versions of ChatGPT, it will not fail, by the way. In the next generation, I do some stuff with Google for example, it won’t fail in the next generation of Gemini anymore. Because they’re putting a lot of effort into fixing this one error. They haven’t fixed the underlying problem. They saw some presentations of people like me pointing this stuff out so they’ve just put a band -aid on top. Now we can’t run life on band -aids.

Band -aids is what? Band -aids is students mugging up one answer before the exam so they get the marks for it. That’s not real learning, by definition. The problem is that we’ve built this system which is our attempt to have general software. And we don’t quite know how to do it. We don’t quite know how to handle it. So let me clarify what I… I’m going to say something incredibly stupid and then I’ll bring it into place. We were talking about this not too long ago. A long time ago you had machines that could do one thing. A hammer is a hammer, a car is a car, a door is a door. You can’t use one as the other.

I’m saying something that sounds incredibly stupid, but think about it. Why don’t you need two separate computers, one to run Excel and one to run PowerPoint? How come both run on the same machine? This is not obvious at all. We’re just used to it, so it seems obvious, but it wasn’t obvious. In fact, the first few computation machines that were made, if you go back, look at all of this Vannevar Bush and even before that Charles Babbage, all of these names one reads in history books or whatever, you’ll see they had differential analyzers and this, that and the other. Oh, this machine, it can add. That machine, it can solve differential equations. This other machine.

It can fit curves. This other machine. It can. do this mapping task. This idea that you could have one machine which could do everything was completely ridiculous because there’s only one thing in the universe that we know of that can do that and it’s the human brain. The human brain is a singular object that can retrain itself to play billiards, to arrange chairs in a room, to present, to drive a car, it can do all of these things. So due to a bunch of very clever people like Alan Turing and co, we figured out that wait a minute, we can have one computer, we can build this one machine. I mean think of it just from a manufacturing point of view, like jackpot.

We can build one machine and it can do all the things. All we need to do is we need to have different software, one for each task. So we’ll have one software for Excel, one software for PowerPoint and the same physical machine will be able to run both. So we built general hardware. And that worked for decades and the fact that we had general hardware to the computation and information revolution. Now for the first time, instead of just having general hardware, that is one machine that can run all software, we have general software, which is you don’t need PowerPoint and Excel separately. You can have one software which you tell it what to do and it will do the job of PowerPoint and you tell it something else and it will do the job of Excel also.

That’s what we are trying to build with AI at the end of the day. Going from general software to general hardware. And as we know, this edit, this ability that we got when we built a general purpose machine, before you needed to spend all this money and build separate machines for every task, and the moment you had a single machine that could do all things, that led to an absolutely massive change. It was a massive revolution. Now that you have general software coming in, right? Once we learn how to do that, think of how the world is going to change. Software companies which used to be, there’s a very interesting graph that I really should have put here, which is if you’re manufacturing something, there’s a burn rate.

So you have an increase in the amount of money you have to invest in your company initially. And then if you manufacture 10 cars, you have a certain amount of money you need to invest. If you want to increase the number of cars you manufacture, I’m talking toy cars, I’m not rich enough to manufacture real cars. But as you increase the number of toy cars you’re manufacturing, your costs go up and sort of linear. And there are bumps every time you do a new round of R &D or something. Software companies don’t do that, right? Software companies, you have this huge expense at the beginning to build everything up. And then once you have that, your burn rate is relatively low.

Selling 50 ,000 units of a software and selling it for $1 ,000. Selling 2 ,000 ,000 units of a software. isn’t going to make a material change in the amount of money you’re investing every day. That entire economy is now going to be gone because you don’t need that kind of investment in software anymore. And this has led to multiple real economies collapsing. So I’ll give you two examples just off the top of my head. Web design companies. There were thousands and thousands of them all over India. You know, a group of college students get together, they say, look, we’ll build websites for people. And these were all micro and medium industries, maybe employing anywhere from 10 to 50 people. That economics is just gone.

We all learned when we were small, right? What is the definition of economics? Economics is the study of the allocation of resources under conditions of scarcity. What if it’s not scarce? There’s no economics affair, despite the fact, again, that we’re in Delhi. There’s no econ affair. But similarly now, econ of maybe writing novels is gone. on, right? You saw what happened with C dance recently, just 24 hours ago. The movie industry is worried. Who’s why should people invest in making movies? If I can write, you know what? I want a movie like Sherlock Holmes, but I want Salman Khan to be the main character and I want me to be the side character and I want this to be the story maker to our movie.

I press enter movies done, right? If that comes to pass, then that, that entire economics is just gone, right? We have seen, these are me talking about the future. Let’s talk about right now, right now at this very moment, a large portion of the internet is collapsing because what used to happen is a large portion of the internet used to run on ads, right? So if I have a recipe website, what do I do? I put some ads on it. You visit my website to read my recipe for blueberry cupcakes or whatever, and you get that ad displayed to you. Okay. And you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get that ad displayed to you and you get I get some money from like the Google, you’ve seen that at the bottom of pages and so on, right?

So I get some money off of that. The problem is that now who’s going to come to my stupid website? They’ll just ask ChatGPT or Gemini for that and they’ll get it and nobody’s going to come to my website. Generally speaking, if you got your search engine optimization correct and you were on the first page of Google, your click rate was one in six, okay? This was the official statistic. That is, let’s say I am a top blueberry cupcake chef. I don’t, that’s definitely not a thing, but let’s say I am. I’m very proud of my blueberry cupcakes. And I’ve made my website and everyone agrees it’s a great site, so when you search for blueberry cupcake recipes, let’s say I’m one of the top 10 and so I would normally have, because people don’t just click one link, they usually go to two or three, I would have a one in six chance of getting clicked and I would make some money off of it.

That number in the past year has gone from one in six to one in seven. in 1500. This means that this is multiple orders of magnitude. So all of these websites that ChatGPT and Gemini and DeepSeek and all of these people, they got the data from these websites only. But now no one will go to these websites and they’re all dying. This is even true of open source tools. So Tailwind, which is a major CSS platform, had to let go of a lot of its engineers because what’s happening is these tools have eaten all the open source code and then people are no longer going to the open source libraries to get it. They’re just saying make me this thing that does that and do it.

Of course there are positive sides. There are non -technical people who can now just say things to the system and it will build them a nice little app, which is great. But simultaneously we are destroying much of the infrastructure and much of the information landscape that made this possible. In the first place. So we’re going to do this. So we have to be exceedingly careful about that. Let me sort of poke on that last sentence that I said. And I think that’s a really important thing when we talk about correctness, trustworthiness, and all of this, right? Which is, in many ways, you know, we had machine learning before 2020 also, right? We were doing classification. We were doing all sorts of clever things.

What really changed with ChatGPT was that anyone could use it. It was the genius of the interface. You had the simple chatbot. You didn’t need to program anymore. You could just say things and it would do them, right? And it is this ease of that interface which changed everything about how we interact with these powerful AI systems. But there is an inherent danger in that. What is the danger? Well, we didn’t build, you know, computers. Languages, all their brackets and, you know, weird expressions. We didn’t do that for fun. Okay? We could have had computer, if we could have written computer programs in English and have them run, we would have stuck with that only.

Why create all of these complicated looking languages where if I miss a semicolon, my computer is going to turn into a peacock, right? We did it that way because our normal language is too ambiguous. There are too many ways in which we say things where we assume you already know what I’m talking about. It’s too easy to miscommunicate, right? The teacher told the student that he was going to the fair. Who’s going to the fair, the teacher or the student? This is obviously a very stupid example, but we have thousands and thousands of ambiguities in our language which make it exceedingly difficult to understand what the other person even wants. That’s why we had computer languages in the first place, to disambiguate.

Now we are saying, no need. I will just give the problem description. This general purpose. Software is just good. going to basically custom solve it. Think about how useful your instructions are. This is deadly, right? We have literally got stories about this, right? About how easy or hard our instructions are. We have cautionary tales about the genies and monkeys for storylines, right? Yeah, yeah, yeah. We’ll switch at 15, don’t worry. Yeah, so when we get to those storylines, we hear that someone says, I want to be the richest person in the world or I want to be the most beautiful person in the world. And what happens immediately after that is it kills everyone else. And it says, I have technically, correctly satisfied your query.

Everything you said, I have done. And so when we give a query, we want the machine to basically align with my expectations. That’s

what alignment is. That’s what alignment, that term means, right? That we wanted to align with my expectations of how this stupid thing is going to act. That leads us to the following conundrum. That I have the system, it’s going to do certain things. I worry that it may do certain bad things. How do I define what is the risk of it getting into this bad thing and doing this bad thing? Do we have a clear way to define risk in our context? And for that, I’ll hand over to Anirban. Alright, I can take the clicker. So, I will keep it slightly brief and I’m going to skip over some slides in the interest of time.

We have looked at different aspects. Three of us are at Ashoka, we work together on different aspects of risk. Safety, risks, harm reduction, risk management, safety, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk management, risk and trying to quantify them and understand them.

This is just the map of India part. I’ll get back to India as a question. India is a big nation, as we all know. But there’s a lot of technology, and we have a tendency to solve our questions of scale using a lot of technology. That naturally introduces many challenges. You’re on the fifth day of the summit. I don’t need to tell this to you. All of you have seen different examples of how empowering this technology could be and why it’s important to be a bit skeptical about its deployment, because that could introduce new kinds of risks. But what is a risk? It’s hard to quantify that and define it. Risks and harms would mean different things in different contexts.

Our goal as a team was to understand and try to make sense of these risks. So hard to define. One definition that we’ve chosen is that the probability of an undesirable outcome characterized by two things. The two things are its likelihood and its severity. I think the airplane example is a good example of that. The two things are its likelihood and its severity. This example is just soon up. Okay, it’s coming back. But basically, airplanes are unsafe, all of you know that. Most of you also take airplanes. It’s because the probability of something happening is lower, that’s what likelihood comes in. But airplanes are dangerous, that’s why we like watching aircraft investigations, because of severity.

Those two are just oversimplifying what I mean here. These definitions also need to be grounded in context, context such as where you’re deploying these systems, so education, healthcare, some of the many areas that have been discussed in many of these panels and discussions across different halls here. I’m going to keep it brief. But these risks go beyond hype. There are real, real challenges and the real costs that everyone has to pay when such systems are deployed at scale, without taking risks into account. Some cut off, but one example is from Air Canada. There are many such examples. These are examples of real people suffering. loss of life, loss of liberty, loss of money and property because of AI safety risks.

So we have taken a life cycle view of AI safety risks and tried to create a taxonomy. It’s a comprehensive taxonomy of 37 different kinds of risks. We have launched it earlier today and it’s now available online. I’m just going to give you a brief overview of the kind of work we have done towards that. Again, here are some examples of what is a risk in our definition or not. So what is not a risk is physical destruction of infrastructure. It is an AI related risk but we are not talking about that. Our scope is very limited. There are many global frameworks that talk about these kind of things. You have some coming from Singapore in Asia, we have Europe, we have the US.

But they do not take into account the main challenges that we see in India. India has scale, India has linguistic diversity, but India also has a lot of different things. India also has certain problems like low network connectivity. If you, for example, are deploying AI in a space which is safety critical, but you lose network and someone’s life depends on it, then it could be another kind of challenge that has to be uniquely defined in India. We see that many of these challenges are not covered in international repositories and risk databases like these. So what they have is what we call contextual blindness, where they are not realizing the social challenges and the socio -technological challenges.

India, again, as you know, deploys large amounts of technology. We have larger technology

systems than any country in the world. UPI, EVM, Sadahar are just simple examples of that. The safety risk database that we have launched, it’s in partnership with AICSTEP Foundation, and it’s called ASTRA. It’s AI Safety, Trust, and Risk Assessments. We’ve tried to create a fun acronym that is easy to remember. ASTRA is now formally launched. Some of us worked on it. Ananya, who’s in the audience, is also one of the contributors. AI. It is a seven -step process. And maybe, Anurban, you could just quickly walk… through this process and how Astra was built. Yeah, hi everyone. So both Devayan and Alok did a good job summarizing the overall work. So these are a bit of technical details.

I’ll probably skip most of it. Basically what is there to understand is that this, if you think about it simply, it’s basically a risk, it’s a database of risks, right? But they are contextualized in the Indian context heavily, right? So one formula fits all kind of a narrative does not work in AI safety. This is what our claim is and this is in line with many researchers, right? Many prominent researchers. So what we started with was resource identification and here’s what our work differs from many of the global frameworks that people have built. So when it comes to resource identification, we had to actually do bottom -up research of how and where exactly these risks occur in the Indian context, right?

We have primarily education and financial as of now. but we started an exhaustive study of how exactly these risks manifest across sectors, right? And the final step of this is a comprehensive risk taxonomy and ontology, right? So taxonomy is basically categories and subcategories of risks which you will find probably in many global frameworks but what is there in our database is an illustrative set of use cases, right? Where you have a use case, a risk use case which you can go and click if you are in the financial lending sector, right? You can go click and see what kind of risk has happened in the Indian context exactly related to our language, our caste, our whatever kind of variables we care about, right?

So these are some of the basic steps through which we have worked on building Astra. So there are two parts to it very briefly. So one is the causal taxonomy. So one is we also tell you through this database at which stage the risk has occurred. So it can occur during development. for example bias in AI we all know about it it happens because of probably bias training data that is one of the sources right so it happens during development deployment let’s say you take an AI system which was built in the US and you implement that or deploy that in an Indian solution setup where most of the people speak in Marathi right this is a deployment problem right so it manifests in deployment and usage I take the AI system it was never meant to disseminate disinformation but I did that as the user I actually manipulated it so that’s in usage and then there are stakeholders is the AI system primarily responsible for the error or risk or is it that it happened because of a deliberate end -user kind of an action it also tells you about whether this risk is intentional or unintentional again in no way do we that this database is in any way exhaustive or foolproof right it is currently you know advancing it more and more expanding it to other sectors but But the target is to also tell you about these granularities around risk.

Because risk is not just one term like Alok explained, Debaian explained, right? You also have to look at what is the intent behind it. So there are two main categories of risk. And this is the part that we struggled the most about. By the way, this Astra, it’s currently available on archive. And you can probably go and read this paper. And you can also take a look at the database whose link is present in that paper. But this work took us almost six months. And again, Ananya, if you could wave. So Ananya is a primary contributor of this risk database that we formed. And so we categorized after looking at the type of risk. There are social risks which are easily quantifiable, which you can easily observe.

For example, linguistic bias. An AI system trained in English does not answer Hindi queries that well. So this is a typical risk which comes under social, right? Frontier risks are risks which are very, very difficult to observe, right? There are risks that we know. Could occur. tomorrow AI could replace jobs we all know about it but how do you quantify it I mean in many of these risks have haven’t even occurred in the Indian context you know about it because from some remote Western translation you could translate it we know there’s a gut feeling that it might go wrong but we don’t we can’t quantify them very easily these are the kinds of risks which come under frontier so there are some examples here I’m not going to the details in the interest of time but there is bias and exclusion toxicity risk categories right and then in frontier you have mostly around power seeking an AI system going rogue I’ll just quickly cite an example right I’m not naming the firm but there’s this news on a trading firm which applied an AI system to go do quick trading according to market variables right the AI system performed very well initially and then without the consent of the firm and because they were not monitoring it properly it went rogue it started doing transactions which were extremely lossy you know it was a risk category and then in frontier you have mostly around power seeking and AI system going rogue I’ll just quickly cite an example right I’m not naming the firm but there’s this this news on a trading firm which applied an AI system to go do quick trading according to market variables right the AI system performed very well initially and then without the consent of the firm and because they were not monitoring it properly it went rogue it started doing transactions which were extremely lossy and not just that in a very high volume it started doing that right so this is the example typical example of power seeking now in India Well, there might be some examples abound, but then do you really know whether this kind of risk can be easily quantified?

We don’t know what will happen. We’ll probably deploy and we’ll have to watch. So those are the kind of risks that we have listed in frontier risk. One quick example is also human -computer interaction, right? So we all know, I mean, sorry, there’s a student sitting here, but I’m going to say this, but in most universities, okay, students are using AI and we know that that leads to cognitive decline and lack of critical thinking. But again, how do you quantify it, right? It’s very difficult. So these are frontier risks, right? I’m not going to the details of this. You all know about caste bias, linguistic bias of AI systems, hallucination we all know about, right?

Incorrect outputs by AI and then infrastructure exclusion. So this is one critical example and this came up from a discussion with the XTREP team that let’s say there’s an AI system that you deploy and a farmer is trying to use it. In many regions of India, there are connectivity issues, right? There is an internet connectivity issue and the entire app starts loading, loading and buffering. It doesn’t work, right? Now this is a typical example of infrastructure exclusion. So again, remember the stage of error manifestation is the deployment. Chat GPT or any open AI for that matter will not care about this. It’s not their job, it’s our job. When we are deploying it in context, it’s our job to take into consideration that our connectivity might be poor.

So this is a typical example of some examples of social risks. So this is one reason why these social risks manifest at this level. As you go higher and higher models, they have more persuasive power so they can manipulate you. Frontier risks I already spoke about. I’ll quickly move on to mitigation. The one quick point I want to make about mitigation is it’s an extremely challenging task. So while the database is the first step, as per our AI safety risk framework of Astra, mitigation as they buy and adequately pointed out, is the hardest task. That we have at hand. So these mitigation measures are often not effective. They are very context specific. and there are certain kinds of mitigation measures that also lead to loss of utility.

So we have to be super careful about that, right? You put a very strong mitigation measure but then that leads to lack of utility on the user’s front. That is not a very good mitigation measure contextually speaking. So according to this work, what we want to carry forward is we want to empirically ground these risks going forward. What is the probability of risks really? And finally, we are also trying to include more and more domains. Currently it’s on education and financial lending. We want to expand it very soon to agriculture and many more