HETEROGENEOUS COMPUTE FOR DEMOCRATIZING ACCESS TO AI

20 Feb 2026 13:00h - 14:00h

Session at a glance

Summary

This panel discussion focused on heterogeneous computing and AI infrastructure challenges in India, featuring experts from Qualcomm, Cisco, IIT Madras, and Intel, along with a government minister. The central theme revolved around distributing AI compute across different layers – from edge devices to data centers – to create more efficient and resilient AI systems.

Durga Malladi from Qualcomm emphasized the importance of running AI inference directly on devices, noting that smartphones can now handle 10 billion parameter models while smart glasses can run sub-1 billion parameter models. He advocated for “hybrid AI” that seamlessly distributes computing between devices, edge cloud, and data centers based on connectivity and requirements. The discussion highlighted voice interfaces in native languages as a key application area, with support for 14 languages mentioned.

Arun Shetty from Cisco identified three major impediments to AI adoption: infrastructure constraints (power, compute, and networking), security and safety concerns, and data gaps. He stressed that enterprises and governments possess the best datasets but need secure, fit-for-purpose solutions. The security aspect was particularly emphasized, noting challenges like model hallucination, toxicity injection, and the need for comprehensive visibility across AI systems.

Professor Kamakoti discussed the critical importance of trust in AI systems, explaining that mathematical definitions of trust are complex and context-dependent. He emphasized the need for sovereign AI models and robust cybersecurity measures, particularly for critical infrastructure and public systems. Energy efficiency emerged as a crucial concern, with discussions about power usage effectiveness (PUE) and the need for hybrid energy solutions. The panelists concluded that India’s AI future depends on collaborative efforts to address infrastructure, security, and energy challenges while leveraging the country’s strengths in application development and diverse datasets.

Keypoints

Major Discussion Points:

– Heterogeneous Computing and Distributed AI Infrastructure: The panel extensively discussed the need for distributed computing across devices, edge cloud, and data centers rather than concentrating all compute in single locations. This includes running inference on smartphones (up to 10 billion parameter models) and smart glasses to reduce dependency on network connectivity and data centers.

– Infrastructure Constraints and Resource Management: Significant focus on three critical bottlenecks – power consumption (with projections of 63 gigawatts needed), compute availability, and networking challenges. The discussion emphasized energy efficiency, with data centers requiring 40% power for cooling, 40% for computing, and 20% for connectivity, highlighting the need for better power usage efficiency (PUE).

– Security and Safety in AI Systems: Comprehensive discussion on AI security challenges including model vulnerabilities, adversarial AI, data poisoning, and the need for “shadow AI” detection in enterprises. The panel distinguished between safety issues (models not working as intended) and security threats (external actors changing model behavior).

– Data Quality and Sovereign AI Models: Emphasis on the importance of high-quality, accessible datasets for AI development, with particular focus on India’s need for sovereign large language models using local data rather than relying solely on public datasets used by global models.

– Practical Applications and India’s AI Ecosystem: Discussion of India’s growing AI landscape with 300+ Gen AI startups, focus on application layer development, and the need for localized solutions including voice interfaces in 14 Indian languages and domain-specific models for various verticals.

Overall Purpose:

The discussion aimed to explore India’s path toward building robust, secure, and efficient AI infrastructure through heterogeneous computing approaches, addressing both technical challenges and policy considerations for scaling AI adoption across enterprises and public systems.

Overall Tone:

The discussion maintained a professional, collaborative, and optimistic tone throughout. Panelists demonstrated mutual respect and built upon each other’s points constructively. The tone was forward-looking and solution-oriented, with participants sharing practical insights from their respective domains while acknowledging shared challenges. The minister’s closing remarks reinforced the positive, collaborative atmosphere by emphasizing the partnership between policymakers and technologists for societal welfare.

Speakers

Speakers from the provided list:

– Kazim Rizvi – Moderator/Host of the panel discussion

– Prof. V. Kamakoti – Professor and Director of a premium educational institution in India, involved in India’s AI policies, expertise in cybersecurity and trust in AI systems

– Arun Shetty – Representative from Cisco, expertise in networking, connectivity, AI infrastructure, and AI safety/security

– Gokul Subramaniam – Expertise in edge computing, AI deployment models, vertical-specific AI applications, and infrastructure optimization

– Durga Malladi – Representative from Qualcomm, expertise in processors, heterogeneous computing, AI inference on devices, and hybrid AI solutions

– Sridhar Babu – Honorable Minister, policymaker focused on providing infrastructure support (power, electricity, water, land) for AI development

Additional speakers:

– Sarah – Representative from Intel (mentioned only briefly at the end for gift presentation)

Full session report

This panel discussion on heterogeneous computing and AI infrastructure in India brought together leading experts from industry, academia, and government to address critical challenges and opportunities in the country’s AI development. Moderated by Kazim Rizvi, the panel featured Durga Malladi from Qualcomm, Arun Shetty from Cisco, Professor V. Kamakoti from IIT Madras, Gokul Subramaniam from Intel, and Minister Sridhar Babu, creating a convergence of technical expertise and policy perspectives.

The Shift Towards Distributed AI Infrastructure

Durga Malladi from Qualcomm opened with a compelling vision for distributed computing that challenges conventional AI infrastructure thinking. His central principle—that AI user experience should remain consistent regardless of network connectivity—established the framework for reimagining AI deployment. This necessitates running inference directly on devices rather than relying solely on centralized cloud processing.

Malladi demonstrated the feasibility of this approach with impressive technical achievements: modern smartphones can handle up to 10 billion parameter multimodal models, while smart glasses can efficiently run sub-1 billion parameter models with 24-hour battery life. These capabilities represent a significant leap in edge computing power, enabling sophisticated AI applications to function independently of network connectivity.

The concept of “hybrid AI” emerged as Qualcomm’s strategic approach, distributing computing across devices, edge cloud infrastructure, and traditional data centers based on specific workload requirements. This optimization across the computing continuum moves away from forcing all AI processing through centralized bottlenecks.

Voice interfaces exemplified this distributed approach’s practical applications. Malladi emphasized voice as “the most natural user interface,” particularly important for native language interaction. Supporting 14 languages requires heterogeneous processors capable of handling diverse linguistic and cultural contexts, benefiting from localized processing that understands specific user environments.

Infrastructure Constraints and Energy Challenges

Arun Shetty from Cisco identified three critical impediments to AI adoption in India: infrastructure constraints encompassing power, compute, and networking; security and safety concerns; and significant data gaps. The power challenge emerged as particularly acute, with projections that AI infrastructure will require substantial energy scaling in coming years.

Gokul Subramaniam from Intel highlighted three physical constraints India cannot circumvent: land, water, and power. His analysis revealed that in data centers, 40% of energy goes to cooling, 40% to computing, and 20% to connectivity. This breakdown emphasizes the importance of achieving optimal Power Usage Efficiency ratios, where maximum energy goes to actual computing rather than supporting infrastructure.

The cooling challenge becomes complex as compute requirements scale, with different cooling solutions needed for varying power densities. For India, with its diverse climate conditions, this requires region-specific solutions accounting for local environmental factors.

Subramaniam emphasized the leapfrogging opportunity this presents for India, noting that edge computing can reach areas without traditional connectivity infrastructure, potentially democratizing access to AI capabilities across the country’s diverse geographic and economic landscape.

Security and Safety: Understanding the Distinction

Arun Shetty made a crucial distinction between safety and security concerns in AI systems. Safety issues involve models not working as intended—including hallucination, toxicity, and unpredictable behavior. Security concerns involve external actors deliberately changing model behavior through adversarial attacks or data poisoning.

This distinction has profound implications for risk mitigation strategies. Safety requires internal controls and model validation, while security demands external threat detection and defensive mechanisms. The non-deterministic nature of AI models complicates both challenges, as consistent input-output relationships cannot be guaranteed.

Professor Kamakoti provided a mathematical framework for understanding trust in AI systems, referencing the TV show “Yes Prime Minister” to illustrate that trust is neither reflexive, symmetric, nor transitive. Trust is context-dependent and temporal, varying based on circumstances and changing over time. This complexity necessitates new approaches to AI security that account for trust’s nuanced, contextual nature.

Shetty briefly mentioned the challenge of “shadow AI” in enterprises, where organizations lack visibility into AI applications their employees use, creating potential security vulnerabilities and compliance risks.

Data Sovereignty and Quality

The discussion revealed significant opportunities for India to leverage its unique datasets while addressing quality and accessibility challenges. Shetty observed that while most global AI models train on publicly available data, enterprises and governments possess superior datasets that could enable more effective AI applications.

Kazim Rizvi noted that India has approximately 300 GenAI startups building on large language models while simultaneously developing sovereign models. This dual strategy leverages global AI advances while building indigenous capabilities, balancing innovation speed with strategic autonomy.

Professor Kamakoti suggested incorporating “need to know” principles into AI models, similar to security clearance systems, enabling appropriate responses based on user authorization levels while maintaining functionality for authorized users.

Practical Applications and Strategic Opportunities

Gokul Subramaniam highlighted specific AI applications in education, including real-time translation and transcription services that could transform learning experiences. These domain-specific models optimized for educational content could provide personalized learning and adaptive content delivery, functioning effectively even in areas with limited connectivity.

The education sector represents a particularly promising area for distributed AI deployment, potentially democratizing access to high-quality educational resources across India’s diverse geographic regions.

Small and medium businesses also represent significant opportunities for edge AI deployment, making advanced AI capabilities accessible to organizations that previously couldn’t afford sophisticated cloud-based solutions.

Policy Support and Collaborative Framework

Minister Sridhar Babu’s participation highlighted critical policy support for India’s AI infrastructure development. His commitment to providing adequate power, electricity, water, and land infrastructure represents essential government backing for private sector AI initiatives.

The minister emphasized “welfare for all, happiness for all” as the ultimate goal of AI implementation, providing important ethical grounding that ensures AI development serves broader social goals rather than purely technical or commercial objectives.

Future Outlook

The panelists outlined a vision for India’s AI future that balances ambitious technical goals with practical implementation challenges. The hybrid AI approach represents a pragmatic path forward, enabling incremental deployment of AI capabilities across the computing continuum without requiring massive upfront investments in centralized infrastructure.

The development of sovereign AI models represents both a technical challenge and strategic opportunity, requiring sustained investment in data infrastructure, model development capabilities, and human capital to compete globally while serving specifically Indian needs.

Energy efficiency improvements offer significant opportunities for reducing environmental impact while controlling operational costs. The combination of edge computing capabilities with strategic data center deployment could optimize India’s AI infrastructure development within existing resource constraints.

Conclusion

This panel discussion illuminated the complex challenges facing India’s AI infrastructure development while highlighting significant opportunities for innovation and leadership. The shift towards heterogeneous, distributed computing represents a fundamental reimagining of AI deployment that could serve diverse user needs while respecting infrastructure constraints and security requirements.

India’s unique position—combining technical talent, diverse datasets, a vibrant startup ecosystem, and supportive policy environment—positions the country to lead in this new paradigm. The collaborative spirit evident in this discussion, where technical experts, policymakers, and industry leaders work toward common goals, provides a compelling framework for navigating the complex challenges ahead while maximizing AI’s transformative potential for all citizens.

The vision articulated by the panelists of AI systems that serve all citizens, respect sovereignty and security requirements, and operate efficiently within India’s constraints offers a roadmap for the country’s AI future that balances innovation with practical implementation realities.

Session transcript

Durga Malladi

with them. 14 languages. Voice is the most natural user interface to devices around you. So the idea is not to actually keep typing and texting, but it’s about the usage of voice, but in native languages, which actually work very nicely. And that means that you have to make sure that the use cases are built on top of it. So that’s what our focus is from a processor standpoint. One final note, and given that I have maybe just one minute, another aspect of heterogeneous computers, disaggregation of compute within the network itself. What I mean by that is, at some point in time, you might have extremely good connectivity to the network. And at some other point in time, you might have zero connectivity to the network.

And the question to ask is, do you want your AI user experience to be invariant to the quality of the communications that you have at that point in time? Or do you want it to depend on it? Obviously, you want it to be invariant. That means you must have the ability to run inference directly on devices. Not that you want to do it all the time, but when you can, why not? today we can run up to a 10 billion parameter model multimodal model state of the art on a smartphone and a sub 1 billion parameter model in your glasses without necessarily charging a device the whole day it’s once every 24 hours so we’ve come a long way in that which means use the data centers use the edge cloud as and when necessary they have a role to play at the same time make sure that we also build for devices where the inference actually occurs and users directly perceive that’s where the data originates so it’s important to think about it that way

Kazim Rizvi

yeah there’s there’s also very strong environmental aspect to this and which often gets unnoticed and undiscussed but that element is also very important in terms of efficiently managing the energy requirements because energy as we also know is finite and so I think you one thing which I was struck to me which is spoke what was inferences and the other is that it’s not just about the energy but it’s also about the energy and the A lot of what’s happening in India is also around inferencing models, right? So, I mean, in terms of the Gen AI story, which we have, we have almost 300 Gen AI startups, which are building on top of the large language models.

And India is definitely leading the way in terms of application layer. There’s no doubt about that. Now, of course, with Sarvam and others, we are also building sovereign large language models, right? So, we are sort of, as Minister Vaishnav has spoken about, every, you know, piece of the puzzles. We are there in terms of fitting that puzzle together. I’d like to come to Mr. Arun Shetty, sir, is with Cisco. And, you know, we just want to take it further from where Durga sir had left in terms of talking about enterprise adoption at scale. And, you know, of course, with Cisco, what are the challenge of bottlenecks, which you see in terms of computer availability, connectivity, which Cisco is trying to do, which you see in generally.

And I think that’s a really important thing to talk about. And I think that’s a really important thing to talk about. And I think that’s a really important thing to talk about. And I think that’s a really important thing to talk about. And I think that’s a really important thing to talk about. And I think that’s a really important thing to talk about. And I think that’s a really important thing to talk about. And I think that’s a really important thing to talk about. And I think that’s a really important thing to talk about. And I think that’s a really important thing to talk about. And I think that’s a really important thing to talk about.

And I think that’s a really important thing to talk about. And I think that’s a really important thing to talk about. And

Arun Shetty

Yeah, so as you know, we connect and protect the… This should be working, right? Yeah, yeah, yeah. As you know, we connect and protect even in the AI era, right? We started in the internet, we came into the cloud, and we are in this era. First of all, thank you very much for having me, and it’s indeed a pleasure to be representing this esteemed panel. So I think what I’ll do is I’ll summarize based on what others have spoken, actually, and I think those are real problems. The first one is clearly the three impediments for AI adoption is one is clearly infrastructure constraints, and we all spoke about it, and they all spoke about it.

The first one is the power. power is a challenge will be a challenge i think usc is expecting it will be 63 gigawatts of power in couple of years what they require okay and then the compute is a problem we did recognize that compute is becoming a problem and then uh kamakoti sir did tell that cisco is in networking what are you doing in networking and networking will be a problem actually and then we need to see how we need to address and clearly it has to be a fit for purpose solutions because you not only do huge data centers and i think what we see is in couple of years you will see there is more inferencing happening at the edge and that’s what we need that’s what the how the world will move and that’s why solutions have to be fit for purpose for sure the second bigger challenge what we have is the security and the safety aspect so that is something what we need to pay lot of attention because as the adage says what if you can’t see you can’t trust right you can’t trust something what you can’t see so you need to have the visibility across the stack and also you need to see whether the models what we are using are the right models for us or is there anything malicious into the models itself actually vulnerabilities in that model so the security aspect becomes where security and safety aspect becomes very very important because the models hallucinate you can inject toxicity into the model so those are the challenges what we need to address as far as what we use so i think it is very very important to build our models and if you look at the models all the models were built using the public data which was the text voice and video data so but however the enterprises the government has the best data sets so why can’t we use those data sets so the third impediment what we have today is the data set so the third impediment what we have today is the data set so the third impediment is the data gap and data gap is essentially i need to have high quality accessible and manageable data and we can build gpts using that what we can call it as a machine gpt what we can build using that use that for inferencing use that for training use that for inferencing and we get a lot of quality use of ai without data the which is the fuel for the ai today you can’t really move forward on the ai and i think these are the typical three problems and the ways we are looking at addressing this is clearly one is i will not be able to build a huge data center for a specific use case so take a use case and then see how fast i can give that infrastructure a comprehensive secure ai factory or a secure infrastructure whether it is in the data center or in the edge actually so that people can focus on building the use cases or the applications on top of it and the second thing comes on the safety and the security aspect of it and how we can do the defense mechanism and the third one is the data so these are the three problems what cisco is trying to address along with the ecosystem partners of course because this is not a problem what you can solve alone actually yeah thank you

Kazim Rizvi

yeah i think i don’t know if my mic okay it’s okay yeah and i’ll i’ll sort of take from the security point which you have spoken and i’ll come back to dr kamakoti i think we have on the clock it shows seven but on my watch it shows 15 yeah so i’ll go by my watch uh yeah so dr kamakoti would like to focus on critical infra and public systems here and as you know that as with the advent of ai we’re going to use it across these sectors as well so how important do you see heterogeneous compute in terms of contributing to national resilience to safeguard and to sort of you know ensure that our critical infrastructure public systems are secure as well

Prof. V. Kamakoti

So today, the type of things that we need to do for each one of these actions, the type of inferencing, type of response time we need, as Shetty mentioned, it’s going to be different. I hope all of you have seen Yes Prime Minister, and always they say, need to know, right? You need to know, right? And now what happens is if I am going to make a model that has understood the entire data, then this that the model, and it is used to be someone that someone should they need to know that data? That’s a very important question. So that’s where the entire aspect of cybersecurity comes in. And that’s why we are all saying that we have need to have sovereign models.

As he rightly pointed out, we can have adversarial AI, we can go poison the whole thing and then make it teach make it tell the things that, you know, should not be told, or need not be told. Okay. This is something that we need to very much look at from a security point where i do an inferencing and my training data set goes for a toss number one so we need to have something for for education at least as a director of one of the premium students in the country what my worry is that for education like how we have since our board for uh you know movies what we should make models for which certain details alone should be fed into it see is a bacha right whatever you teach what it will tell you back probably do a little more uh generative on that so this is number one number two is again coming back to cisco itself right you do deep packet inspection and basically you do it with some signatures today the the whole story is changing dynamically the malware can change its signature so that’s going to be the biggest challenge now and what sort of inferencing they are going to do they have to bring some more different architecture and that will be a heterogeneous architecture now and so so So, ultimately, you know, as you see, you know, what you see, the trust component, I always repeat this, I’ll finish with this with my one minute.

So, trust is, you know, friends, you know, if you want to define A is equivalent to B, that’s the definition, right? If you want to define A, you have to come with B, which is equivalent to A. So, equivalence in discrete mathematics, equivalence relation should satisfy three properties, reflexive, symmetric, transitive. A is trust is not reflexive, I don’t trust myself sometimes. Trust is not symmetric, I trust Sarah, Sarah may not trust me. Trust is not transitive, I trust Gokul, Gokul trust you, I may not trust you. Trust is in addition, trust is context dependent, I trust. I trust you on something, I don’t trust you on something else. It is temporal, morning I trust you, evening I don’t trust you.

So, right? So, the main thing is, we have to build that mathematics. defined trusted and if you go to you know some of these search engine and define trust you get 1 million hits for that so so that is going to be the most important part so specifically on heterogeneous we will have certain different types of security issues something which a can sound something which is originating because of a and that’s where all of us edge connectivity server all the three people have to work together and and we will teach and he’ll put policy so

Kazim Rizvi

but both of you are equally playing an important role in terms of policy dr. Kamothi you’re also you know very influential and important figure in India’s AI policies of course lots to learn from you Goku very quickly would like to come to you and you know just sort of taking away in terms of the practical deployment models and what are the sort of examples you’ve seen which demonstrate that we are moving towards heterogeneous compute right and what needs to be done to also get get to that

Gokul Subramaniam

So I started off with workload and I’ll go back to the same thing. So one of the things that we’re looking at and it’s critical is to see what vertical really needs what kind of domain specific models. And then try to apply that as much as possible as edge inferencing and contain the walls that are there that prevents AI to work efficiently. Primarily it’s like memory, you know, the connectivity, the IO, the thermal and then the power. So from an edge inferencing standpoint, there are quite a few things that are being done, be it an education segment where you want more translation, data being available, transcription. So that the knowledge is being imparted in a way that you have with the right data with the lowest power that’s meaningful for the student.

And more importantly, when we talk security, it’s not only about protecting data. the models we keep talking data and models it’s protecting the user that’s even more fundamental and how you can ensure that that happens second thing is applying it to other verticals be it small and medium business i think there is a great opportunity there where edge inferencing and putting compute with the right kind of power that can translate the businesses into actually using ai more effectively the last aspect that i want to also touch upon is in terms of just power you know as we go from one gig to nine to ten gig in the next five years in the country we have to realize that india is challenged by three physical things that we cannot run away from land water and power and these are very important aspects that it will drive how we set up our infrastructure and you know almost you know in a hundred percent of your power energy that comes into a data center forty percent goes into cooling forty percent into your computer and twenty percent on connectivity and there is this famous metric that you use, the PUE, the power usage efficiency.

It has to be as close to one as possible. All the power that you give goes to the most important thing, which is the computer, not to the cooling and things. And there are a lot of technologies that are being played with with respect to how much you can air cool on a rack, per rack, and that was okay up to about 25 kilowatt, and as you start to get to 100, you have to use liquid cooling, and then how we can set that infrastructure up. And for a country like India, it’s absolutely important to look at what hybrid energy solutions we can go with, because just pure renewable may not be able to address it.

You’ll have to have something that is stable and be able to do something off -grid so that there is that dependency for you to get the data from the data centers and push as much as possible to edge, because edge is all about reach. How can I take it to places across the country where there is no access to connectivity? It’s about how can I leapfrog? How can I leapfrog with verticals that have not used technology as much? We’ve always done a leapfrogging in India, and this is a great moment for us, and total cost of ownership. Those are the big areas.

Kazim Rizvi

Thank you, Gokul. And I think as we are approaching the end of the panel, I’d sort of like to go to Durga and Dr. Shetty also in terms of closing remarks and the way forward. So to both of you, I’ll pose this question in terms of the next two to four years, because I think the AI age, we don’t think too far ahead. We can’t do five -year planning or 10 -year planning. I think two -year planning is sufficient. So what enterprise outcomes are you both looking at? Maybe we can start with Durga in terms of defining India’s access to compute, access to infrastructure, capacity, and also sort of building in scale, cost efficiency and energy efficiency.

Durga Malladi

So I’ll keep it brief. I think what I’m looking forward to with all the conversations here and in other parts of the world as well, where the problems are somewhat similar, is the ability to distribute compute across the entire network. So think of a combination of inference that runs in devices to the largest… extent that’s possible. Edge cloud, on -prem servers, where a lot of the localized processing can be done. And these can be done in air -cooled carts, by the way. The point that was made earlier is absolutely relevant. You don’t necessarily need liquid cooling all the time. You can do air -cooled carts and then just use air -cooled servers and running up to 100 to 300 billion parameter models, which are getting pretty sophisticated.

That’s the edge cloud. And as you go deeper from there onwards, then you have the data centers. It then mitigates the overall requirements of what you need in a data center. And instead of, therefore, concentrating the entire compute in one single location and then building it for just that alone, a holistic approach of devices, edge cloud, plus data center is probably what we are looking forward to. From Qualcomm, we call it as hybrid AI. It’s not just a marketing slogan, but it is something that we truly believe in. Thank you.

Arun Shetty

Since the infrastructure part has been addressed here, so let me talk. A little bit more on safety and security aspects. So I think one of the things what we need to understand about the modern… these models are very intricate and very complex. And it’s also non -deterministic because if you give an input, not necessarily the output will be the same like a standard application, correct? So that’s why it is non -deterministic. So what one should be doing, right? There are two aspects of safety and security. I’ll just touch upon why it is important to know that actually. Safety is all about, we want the models to work in a certain way but it is not working in that certain way or the way we want them to work.

That is the first part of it. That’s where the toxicity part, hallucination, all those challenges come actually. The second part of it is the security part wherein a bad actor from outside can change the behavior of the model. So we need to be careful about both the things actually. So what one should be doing? Say for example, I think Kamakoti sir also told about users to have, that’s it. users also to be secure, right? So it is essential that the organizations or the country has to build that actually. So which means if I’m accessing a chat GPT and sending some confidential info, the system should stop me. So that is the when I’m accessing a third party application, the system should be smart enough to stop me saying that you can’t be sharing that information that’s not allowed for you to share that.

So that’s something which is already happening in organizations today. The second part of it is the first party application, I’m building an application, and I’m using a model. So now the organization should be able to scan what all my AI assets are. Because one of the biggest challenges for enterprise is the shadow AI applications, they don’t know what people are doing actually. So I need to clearly know what all my assets are. That is number one, I detect all my assets or discover all my assets. And next is I should scan. and also ensure that these models and the applications what I’m using are not vulnerable. If it is vulnerable, then I need to put guardrails around it or I need to fix those problems.

And similarly, there are organizations who are already telling that there are a lot of risks. So you need to nist Mitre and OWASP are telling that there are a lot of risks associated with that and we need to ensure that we need to stop that. So that is something what Cisco is focus, our focus to see how we can use AI to defend the, to defend against all these malice and also the vulnerabilities what we see. Thank you so much.

Kazim Rizvi

I think with this, we’ll probably close the panel, but I’d like to invite Honorable Minister once again for his very quick closing remarks that you have sort of. Thank you. us highly motivated to sort of build on this. You’ve heard us in the last one hour. What are your thoughts? We’d love to hear from you in terms of your closing address.

Sridhar Babu

Thank you, Rizvi. And in fact, it’s a great pleasure to be here with the eminent Padmasree Awadi, Professor Kamakoti and Gokul and Durga Prasad and Mr. Vichetti sharing their truly professional experience and how as a policymaker, how we should view the things especially in terms of power, electricity, water and the land. How we should be well equipped to provide all these things where all the eminent panelists over here or the eminent people of the days would be thinking of putting. My primary challenge they have posed before is try to provide all these things. We are here to provide the rest remaining. And in fact, you know, thanks once again for a very apt introduction. very apt dialogue over here.

Ultimately, we have to all, me as a policymaker, and you all technocrats and innovators have to think the basic agenda for this AI impact term is welfare for all, happiness for all. Thank you for inviting me. Thank you so much.

Kazim Rizvi

With this, we will have to close the panel. I’d like to thank all our panelists and also invite colleagues, Sarah from Intel to hand over the gifts. But we’ll just have a group photo. Thank you.

Durga Malladi

Speech speed

197 words per minute

Speech length

Heterogeneous Compute & Edge Inference

Topics

Artificial intelligence | Information and communication technologies for development

Arun Shetty

Speech speed

179 words per minute

Speech length

Data Gap & Quality

Topics

Data governance | Artificial intelligence

Prof. V. Kamakoti

Speech speed

170 words per minute

Speech length

Security, Safety, Trust & Sovereign Models

Topics

Human rights and the ethical dimensions of the information society | Building confidence and security in the use of ICTs

Gokul Subramaniam

Speech speed

186 words per minute

Speech length

Security, Safety, Trust & Sovereign Models

Topics

Building confidence and security in the use of ICTs | Artificial intelligence

Kazim Rizvi

Speech speed

183 words per minute

Speech length

Policy, National Resilience & Welfare

Topics

The enabling environment for digital development | Artificial intelligence

Sridhar Babu

Speech speed

141 words per minute

Speech length

166 words

Speech time

70 seconds

Policymaker mandate for welfare and happiness

Explanation

Sridhar asserts that AI policy must prioritize the welfare and happiness of all citizens, ensuring that technological progress serves broader societal goals.

Evidence

“basic agenda for this AI impact term is welfare for all, happiness for all.” [14]. “policy … power, electricity, water and the land.” [90].

Major discussion point

Policy, National Resilience & Welfare

Topics

Social and economic development | The enabling environment for digital development

Agreements

Agreement points

Distributed AI infrastructure is superior to centralized data centers

Speakers

– Durga Malladi
– Arun Shetty
– Gokul Subramaniam

Arguments

Hybrid AI approach combining devices, edge cloud, and data centers is the optimal solution

Edge inferencing will become more prevalent, requiring fit-for-purpose solutions rather than huge centralized data centers

Domain-specific models should be applied at edge for different verticals like education and small-medium businesses

Summary

All three technical experts agree that the future of AI lies in distributed computing architectures rather than massive centralized data centers, with edge inferencing becoming increasingly important for various use cases

Topics

Artificial intelligence | Information and communication technologies for development

Power and energy constraints are critical challenges for AI infrastructure

Speakers

– Arun Shetty
– Gokul Subramaniam
– Kazim Rizvi
– Sridhar Babu

Arguments

Power consumption will reach 63 gigawatts in coming years, presenting major infrastructure challenges

India faces physical constraints of land, water, and power that will drive infrastructure setup decisions

Energy management is crucial as energy resources are finite, with strong environmental implications

Policymakers must ensure adequate provision of power, electricity, water, and land for AI infrastructure

Summary

There is unanimous agreement that power and energy constraints represent fundamental challenges that must be addressed through policy and infrastructure planning

Topics

Environmental impacts | The enabling environment for digital development

AI security requires comprehensive, multi-layered approaches

Speakers

– Arun Shetty
– Prof. V. Kamakoti
– Gokul Subramaniam

Arguments

Security and safety are major challenges as AI models are non-deterministic and can hallucinate or be injected with toxicity

Adversarial AI can poison models and make them reveal information inappropriately

Protecting users is more fundamental than just protecting data and models

Summary

All speakers agree that AI security is complex, requiring protection against multiple threat vectors including model poisoning, adversarial attacks, and user protection beyond just data security

Topics

Building confidence and security in the use of ICTs | Artificial intelligence

High-quality, sovereign data and models are essential for effective AI deployment

Speakers

– Arun Shetty
– Prof. V. Kamakoti
– Kazim Rizvi

Arguments

High-quality, accessible, and manageable datasets are essential for effective AI implementation

Need-to-know principles should apply to AI models to prevent unauthorized access to sensitive data

India is building sovereign large language models while leading in AI applications with 300+ GenAI startups

Summary

There is consensus that data quality and sovereignty are crucial, with agreement on the need for controlled access to sensitive information and development of indigenous AI capabilities

Topics

Data governance | Artificial intelligence

Environmental impacts | Information and communication technologies for development

Unexpected consensus

User protection as primary security concern

Speakers

– Gokul Subramaniam
– Arun Shetty
– Prof. V. Kamakoti

Arguments

Protecting users is more fundamental than just protecting data and models

Organizations need protection from both internal misuse (sharing confidential info with third-party AI) and external threats

Educational AI models should be curated like movie ratings to ensure appropriate content for different audiences

Explanation

While technical discussions often focus on data and model security, there was unexpected consensus that user protection should be the primary concern, representing a human-centered approach to AI security that goes beyond traditional technical safeguards

Topics

Human rights and the ethical dimensions of the information society | Building confidence and security in the use of ICTs

Mathematical approach to understanding trust in AI systems

Speakers

– Prof. V. Kamakoti
– Arun Shetty

Arguments

Trust is not reflexive, symmetric, or transitive, and is context-dependent and temporal

Visibility across the entire stack is essential for trust, and models themselves can contain vulnerabilities

Explanation

The convergence on a mathematical and systematic approach to defining and implementing trust in AI systems was unexpected, showing alignment between academic and industry perspectives on the fundamental complexity of trust

Topics

Building confidence and security in the use of ICTs | Artificial intelligence

Hybrid energy solutions necessity

Speakers

– Gokul Subramaniam
– Kazim Rizvi
– Sridhar Babu

Arguments

Hybrid energy solutions and off-grid capabilities are essential for India’s infrastructure needs

Energy management is crucial as energy resources are finite, with strong environmental implications

Policymakers must ensure adequate provision of power, electricity, water, and land for AI infrastructure

Explanation

The consensus that pure renewable energy is insufficient and that hybrid solutions are necessary represents an unexpected pragmatic approach to environmental sustainability in AI infrastructure

Topics

Environmental impacts | The enabling environment for digital development

Overall assessment

Summary

The panel demonstrated remarkable consensus across technical, policy, and business perspectives on key challenges and solutions for AI infrastructure deployment. Main areas of agreement include the superiority of distributed AI architectures, the critical nature of power and energy constraints, the complexity of AI security requirements, and the importance of data sovereignty.

Consensus level

High level of consensus with strong alignment between industry experts, academics, and policymakers. This suggests a mature understanding of AI infrastructure challenges and indicates potential for coordinated policy and technical responses. The agreement spans both technical implementation details and broader strategic approaches, suggesting that India’s AI development strategy has broad stakeholder support.

Differences

Environmental impacts | The enabling environment for digital development

Unexpected differences

Fundamental Philosophy of AI Security

Speakers

– Arun Shetty
– Gokul Subramaniam

Arguments

Visibility across the entire stack is essential for trust, and models themselves can contain vulnerabilities

Protecting users is more fundamental than just protecting data and models

Explanation

This disagreement is unexpected because both speakers are addressing AI security concerns, but they have fundamentally different philosophical approaches. Shetty takes a traditional cybersecurity approach focusing on technical assets and system visibility, while Subramaniam advocates for a human-centered security philosophy. This represents a deeper divide in security thinking than might be expected in a technical infrastructure discussion

Topics

Building confidence and security in the use of ICTs | Human rights and the ethical dimensions of the information society

Trust as a Mathematical vs. Practical Concept

Speakers

– Prof. V. Kamakoti
– Arun Shetty

Arguments

Trust is not reflexive, symmetric, or transitive, and is context-dependent and temporal

Visibility across the entire stack is essential for trust, and models themselves can contain vulnerabilities

Explanation

This is an unexpected disagreement because while both speakers discuss trust in AI systems, Kamakoti approaches it as a complex mathematical and philosophical problem that cannot be solved through traditional equivalence relations, while Shetty treats trust as a practical engineering problem that can be addressed through visibility and technical controls. This represents a fundamental divide between theoretical and applied approaches to the same concept

Topics

Building confidence and security in the use of ICTs | Human rights and the ethical dimensions of the information society

Overall assessment

Summary

The speakers show moderate disagreement on implementation approaches while sharing common goals around distributed AI, security, and infrastructure efficiency

Disagreement level

The disagreement level is moderate but significant, particularly around philosophical approaches to security and the optimal balance between centralized and distributed infrastructure. These disagreements have important implications as they reflect different priorities: technical optimization vs. human-centered design, theoretical rigor vs. practical implementation, and comprehensive hybrid solutions vs. focused edge-first approaches. The disagreements suggest that while there is consensus on the challenges facing AI infrastructure deployment, there are meaningful differences in how to address these challenges that could impact policy and implementation decisions.

Partial agreements

All speakers agree that edge computing and distributed AI are important for the future, but they disagree on the optimal balance between edge, cloud, and data center resources. Malladi wants a hybrid approach that includes all three, Shetty emphasizes fit-for-purpose solutions over large data centers, and Subramaniam focuses on domain-specific edge applications

Speakers

– Durga Malladi
– Arun Shetty
– Gokul Subramaniam

Arguments

AI user experience should be invariant to network connectivity quality, requiring on-device inference capabilities

Edge inferencing will become more prevalent, requiring fit-for-purpose solutions rather than huge centralized data centers

Domain-specific models should be applied at edge for different verticals like education and small-medium businesses

Topics

Artificial intelligence | Information and communication technologies for development

Both speakers agree that AI security is a critical concern and that models can be compromised, but they approach the solution differently. Shetty focuses on organizational visibility, asset discovery, and technical safeguards, while Kamakoti emphasizes the mathematical complexity of trust and the need for access control based on security clearance principles

Speakers

– Arun Shetty
– Prof. V. Kamakoti

Arguments

Security and safety are major challenges as AI models are non-deterministic and can hallucinate or be injected with toxicity

Adversarial AI can poison models and make them reveal information inappropriately

Topics

Building confidence and security in the use of ICTs | Artificial intelligence

Both speakers agree on the importance of developing sovereign AI capabilities and utilizing high-quality datasets, but they emphasize different aspects. Shetty focuses on enterprises and governments having better datasets than public sources, while Rizvi highlights India’s comprehensive approach across applications and sovereign model development

Speakers

– Arun Shetty
– Kazim Rizvi

Arguments

High-quality, accessible, and manageable datasets are essential for effective AI implementation

India is building sovereign large language models while leading in AI applications with 300+ GenAI startups

Topics

Data governance | Artificial intelligence

Environmental impacts | Information and communication technologies for development

Takeaways

Key takeaways

Hybrid AI approach combining on-device inference, edge cloud, and data centers is essential for optimal AI deployment, rather than relying solely on centralized data centers

India faces critical infrastructure constraints in power (projected 63 gigawatts needed), land, and water that will fundamentally shape AI infrastructure decisions

Security and safety are paramount concerns as AI models are non-deterministic, vulnerable to adversarial attacks, and can hallucinate or be poisoned with malicious content

Voice interfaces in native languages (14 languages mentioned) represent the most natural user interface, requiring heterogeneous processors to handle diverse use cases

Energy efficiency is crucial with data centers consuming 40% power for cooling, 40% for compute, and 20% for connectivity – requiring optimization toward PUE ratio of 1

India is leading in AI applications with 300+ GenAI startups while also developing sovereign large language models for national resilience

Trust in AI systems is complex – not reflexive, symmetric, or transitive, and is both context-dependent and temporal, requiring new mathematical frameworks

High-quality enterprise and government datasets should be leveraged instead of relying solely on public data for training AI models

Edge inferencing will become more prevalent, with modern smartphones capable of running 10 billion parameter models and smart glasses running sub-1 billion parameter models

Resolutions and action items

Organizations must implement systems to detect and prevent sharing of confidential information with third-party AI applications

Enterprises need to discover and scan all AI assets to address shadow AI applications and vulnerabilities

Policymakers committed to providing adequate power, electricity, water, and land infrastructure to support AI development

Need to develop fit-for-purpose solutions for different verticals like education and small-medium businesses using domain-specific models

Implement guardrails around vulnerable AI models and applications

Develop hybrid energy solutions and off-grid capabilities for distributed AI infrastructure

Unresolved issues

How to mathematically define and implement trust frameworks for AI systems given their complex, non-reflexive, non-symmetric, and non-transitive nature

Specific mechanisms for transitioning from current centralized data center models to distributed hybrid AI infrastructure

Detailed strategies for managing the transition from air-cooled to liquid cooling systems as compute requirements exceed 25-100 kilowatts per rack

Concrete implementation timelines and resource allocation for the projected 63 gigawatt power requirement

Standardization approaches for sovereign AI models across different government and enterprise use cases

Specific regulatory frameworks for curating AI models for different audiences (similar to movie rating systems mentioned for education)

Technical specifications for ensuring AI user experience remains invariant to network connectivity quality

Suggested compromises

Distribute compute requirements across devices, edge cloud, and data centers rather than concentrating everything in centralized locations

Use air-cooled servers for edge cloud deployments (100-300 billion parameter models) while reserving liquid cooling for larger data center operations

Implement hybrid energy solutions combining renewable and stable power sources rather than relying purely on renewable energy

Balance between using sovereign models for sensitive applications while leveraging global models for general use cases

This comment provides crucial context about India’s position in the global AI ecosystem, distinguishing between application-layer innovation and foundational model development. It highlights India’s unique strength while acknowledging the need for sovereign capabilities.

Impact

This observation helped frame the entire discussion within India’s specific AI development trajectory. It influenced how panelists discussed infrastructure needs, security requirements, and policy implications, making the conversation more strategically focused on India’s path to AI self-reliance rather than generic AI development.

Overall assessment

Explanation

This is essential for practical AI deployment across various industries, ensuring that AI solutions are tailored to specific use cases while being efficient enough to run at the edge.

Disclaimer: This is not an official session record. DiploAI generates these resources from audiovisual recordings, and they are presented as-is, including potential errors. Due to logistical challenges, such as discrepancies in audio/video or transcripts, names may be misspelled. We strive for accuracy to the best of our ability.

HETEROGENEOUS COMPUTE FOR DEMOCRATIZING ACCESS TO AI

Summary

Keypoints

Major Discussion Points:

Overall Purpose:

Overall Tone:

Speakers

The Shift Towards Distributed AI Infrastructure

Infrastructure Constraints and Energy Challenges

Security and Safety: Understanding the Distinction

Data Sovereignty and Quality

Practical Applications and Strategic Opportunities

Policy Support and Collaborative Framework

Future Outlook

Conclusion

Voice‑first multilingual UI

Explanation

Evidence

Major discussion point

Topics

Inference on device to keep experience invariant to network quality

Explanation

Evidence

Major discussion point

Topics

Air‑cooled edge carts and selective liquid cooling to improve PUE

Explanation

Evidence

Major discussion point

Topics

Hybrid AI and distributed compute across the network

Explanation

Evidence

Major discussion point

Topics

Cisco networking to address compute and connectivity bottlenecks

Explanation

Evidence

Major discussion point

Topics

Projected 63 GW power demand and compute shortages

Explanation

Evidence

Major discussion point

Topics

Fit‑for‑purpose edge inferencing to reduce data‑center load

Explanation

Evidence

Major discussion point

Topics

Model hallucination, toxicity and need for guardrails

Explanation

Evidence

Major discussion point

Topics

Data gap and opportunity to build “machine‑GPTs”

Explanation

Evidence

Major discussion point

Topics

Heterogeneous security architecture for edge inference

Explanation

Evidence

Major discussion point

Topics

Sovereign AI models and adversarial poisoning

Explanation

Evidence

Major discussion point

Topics

Formal definition of trust (non‑reflexive, non‑symmetric, context‑dependent)

Explanation

Evidence

Major discussion point

Topics

Vertical‑specific edge constraints (memory, I/O, thermal, power)

Explanation

Evidence

Major discussion point

Topics

Projected 63 GW power demand and compute shortages