Waves of infrastructure Open Systems Open Source Open Cloud

Announcements and a lot of activities going on here this week. Excited about it. We are excited about introducing what we do and what we do more in the context of India. We just launched our offering and we’ll be talking more of what we do with our partners in the coming weeks and months. But today, I’d like to introduce ourselves. But before we introduce, we want to set the context of where we fit in, both in the industry trends and the ecosystem and what category we go after from an enterprise private cloud infrastructure. And then we’ll get into sharing some of our partners that we work with and then a Q &A at the end of it with a presentation from Bharat Jain and from Zeta Bolt.

We’ll have an interactive Q &A on some key top three questions or end questions that we think need to be answered. With that, let me start with the first. I want to… thank our sponsors and our collaborators and partnerships at UC San Diego, where they have an initiative for public -private partnership at UC San Diego for AI for education, AI for research, and AI for industry. And we are one of the early industry partners. There’s a newly constituted data science and data center institute called School of Computing, Information Sciences, and Digital Sciences. And we’ll talk a little bit more about it downstream. But this collaboration enables us to not only work on technologies, but also look at key use cases, particularly on health sciences, because San Diego has got one of the largest health science, both hospital system as well as clinical research and variety of health and biotech research.

With the thesis that fundamentally computing is going to be driven by biology and health, it’s a very key partnerships that we hope to work with. going forward. With that, let me step back. This is my standard slide I use in any presentations in terms of long -term reminders, what happens in technology. So where we fit in, we’ll just walk through for the next 20, 30 minutes about what we are doing from a systems innovation, but the systems innovation is going to be punctuated or represented in the context of where the technology shifts that have occurred and will occur as we go forward. So simple reminders are we, as humanity, underestimate. We overestimate what can be done in two years, but we underestimate what can be done in 10 years.

You can go back in history, look at self -driving cars, look at neural link. I remember a slide I had put at UC Berkeley, a conference about programming languages and productivity languages and kind of a very tongue -in -cheek thought, and you just have to think and write and get confused. And I thought, well, I’m going to record out. that was in 2014. I’ll put the slides out later. I thought it would be science fiction, never happened for hundreds of years. But guess what? You can think, you can put a neural link, and probably have cursors generate code for you today. That I never thought about in 2014. So never underestimate what will happen. The big technology shifts that occur every 30 years, 15 years, 7 years.

But the key thing is semiconductors drove the technology innovation in the 80s and 90s, thanks to Moore’s Law. And the cloud phenomenon happened in the last two decades. I do see the pattern now as you are innovating, as you can see, where NVIDIA is innovating tremendously from the silicon side up. And of course, there are innovations going from the top -down, from the use case, from the language models, and higher order functions in AI. And both are coming at the same time, together. A third bullet I would say is, people who are serious about software should make their own hardware. The corollary is, people who are serious about hardware should also make their own software.

So I’m a hardware guy who’s done software, and this venture, I should be doing software first, going to the hardware later, kind of reverse model. this is the last one day one thing I’ll say about myself my professional life has been shaped by luckily I didn’t realize where between 1980s and 2000 there was a peak of Moore’s law there was an exponential part but happened to be lucky to have been part of the semiconductor innovation cycle having developed and delivered a number of world class microprocessors so today we talk about models there are only 4 or 5 guys who could do microprocessors there’s a difficult very small teams 150 person if you look at model foundation models today it’s the same characteristic there are hard problems of course it’s a lot more money you need a billion dollars and lots of GPUs but you still need the same 150 people to do the models it’s not like everybody can do the models so there is a similarity between what happened in the 90s about microprocessors and what I see today in model building it’s the same level of complexity where you need the best and brightest roughly about it’s not me it’s some altman coding that it’s 120 people and I think that’s the difference and you need to have them with the right resources computing.

We also need to have a lot of computing resources to go build the models. So with that let me start I think the next wave and we hope to drive the innovation and disrupt in terms of systems building going forward but the context is why it’s economically interesting and valuable is I think everybody knows if you look at GDP we’ve gone in the last 20 years from 33 trillion dollars to almost 100 trillion dollars by all accounts the GDP could improve by 2x or 4x in the next 20 years but the SAS era was really a productivity improvement so it really scratched the surface about productivity whereas AI is going to impact 95 % of work so the time is much bigger the impact is much bigger, the blast radius is much bigger than the last 20 years.

That’s why the computing also is needed much more. We have 300 billion dollars of infrastructure we’re in from 50 billion dollars I believe in 2000 So we’re going to be in from 50 billion dollars in the next 20 years. So we’re going to be in from 50 billion dollars in the next 20 years. So we’re going to be in from 50 billion dollars in the next 20 years. to now about $250 to $300 million of capital expenditure spent for infrastructure. So power in, capital spent. So every dollar of power you spend ends up being $3 to $5 of capital for compute, memory, network storage. And from there, you do the upper layers of software and then applications.

So that $50 billion was $300 billion, but if you look at all the spending, we’re already at $400, $500 billion, and all accounts in the next 5 to 10 years will be almost $2 trillion of spend. That creates, obviously, there’s a big demand -supply gap. The great thing about programming is every time there is a layer of abstraction, the programming gets simpler, which means it brings more people to the party to be able to compute. I think what LLMs and natural and transform models have done is bring everybody to be able to program. We all are logical. We can algorithmically think. We can program makers, but not everybody could program. finally we have a tool to be able to program in the natural language your mother, your grandmother can also talk to the computer and tell what steps to take and it will do the steps for you or it will tell you what steps to do so that’s the fundamental shift which means at population scale you’re going to have computing for everybody, that creates a huge gap, it’s not even 1000x as Jensen would say it’s a billion x absolutely true, but it creates a big technology gap, supply gap and increasingly because of model and languages and data the sovereignty gap also that’s appeared, that’s the theme of the conference that continues to drive tremendous amount of demand now we have seen a little bit of this before, I have been through the first two cycles of innovation in semiconductors in my first job as at Sun Microsystems and then the dot com era and then now and there was always a demand supply gap in one of these transitions and But we solved it in one way.

It doesn’t mean you can solve it the same way, but we are at the crux of solving it also in a similar way, but with a different set of boundary conditions, if you will. So what we solved between 1990 and 2000, if you look at, we went from clock rate, single CPU, to fundamentally shifting to multicore threaded and distributed systems, and that was the cloud phenomena. I have a slide later to show what the transition was. I’ll probably skip this slide. I think everybody knows we need lots of power, and one interesting point is I think India is going from almost nothing less than a gigawatt to about 10 gigawatt buildup, while the U.S. is going from 25 to 125 gigawatt in other regions, and China.

EMEA is going to be on a comparative basis, on a relative basis, a lot less. But the need for AI -ready geolocal data centers we already see. Everybody is building out. And what is the infrastructure? What is the architecture to support that? there’s certainly reference architectures inside the hyperscalers Google has got a TPU based and AWS has got Tranium based infrastructure Tranium plus general purpose computing and of course Microsoft has got Maya and of course NVIDIA so those are probably and of course AMD but increasingly over time you want to have an open multi -vendor strategy that’s probably where we’ll check we’ll talk more about so why do I believe these transitions and distributed systems are drivers of new innovation up and down the stack this is not new this has happened in history starting from VAX 11 780 was disrupted by of course at that time PC but more so in the enterprise side was Sun and the workstation if you think of the first distributed system in the modern era it was Sun Microsystems where Ethernet was used to build a distributed system network file system and that was version one and over time it’s like evolution you gain more mass, more momentum more weight in your capabilities and you end up building big monstrous machines in E10K that drove the internet and the dot com era but that was also was an Achilles heel because that was not going to enable the scale that people had to go build at much bigger so Google was probably the epitome of the next big shift I’ll talk about that and similar thing we see today is we’ve gone from CPU only dual socket x86 memory clusters to heterogeneous compute but also gone to a fairly large scale up now the interesting transition today was then is training and inferencing as you’ve seen the news lately with the Grok acquisition by Nvidia and others there’s clearly a separation between the training kind of workloads and inference type of workloads and what kind of systems you want to support because inference is going to drive a lot more of the compute so the one way I think about is inference and biology or workloads related to biology and healthcare are going to be the drivers of computing like it was for graphics in the 1990s.

So this is back again to reinforce the point that between 1994 and 2005, we saw the shift going from the version 1 .0 of distributed system to version 2 .0, which is open source. So the first one was open systems in the first 20 years. And open source came and enabled a new way to build distributed system because from an economics, it removed the cost of middle age of software. Everybody got access. In this case, Linux. This is Solaris. But that also enabled to build truly hundreds of thousands of machines in a single cluster. And out of it came Borg, Kubernetes, a whole bunch of other distributed file systems, all kinds of innovations that happened.

So the proposition here is I think we are at the cusp of similar things on the infant scale computers. And I think we are at the cusp of similar things on the infant scale computers. And I think we are at the cusp of similar things on the infant scale computers. And I think we are at the cusp of similar things on the infant scale computers. So just a reminder. And the punctuation that happens every time turns out to be, if you look back in history, it’s Ethernet. Yes, the network is the computer, but more important is 10 megabit was the onset of replacing big mainframes or miniframes like VAX to workstations and network of workstations.

Then right at the point of 10 gigabit Ethernet coming around 2000 to 2002 timeframe, along with it was multi -core, was enabled the new distributed building block. We are at the same point. We have got 800 gigabit Ethernet going to 800 gigabit Ethernet and probably a terabit Ethernet networks. And that’s hopefully, and that will be the enabler, and that’s a bet we are making. So the other element of the system is its network and then the memory. And do you build a full scale -up system at data center -wide? Certainly you need for training for backpropagation and forward. but inference can be much more distributed, shardable and it’s time to rethink what kind of systems you want for inference only dominating infrastructure.

The other dimension to think about is we’ve gone from a single memory type to multiple memory types so do we need four light types of memory to deal with a variety of layers or just two or one? That’s a lot of debate in the technical community but that’s a critical decision that will happen. So a way to think about this, we think of the entire system not from flops and GPU and compute. GPU compute, CPU compute are needed but really what does the memory hierarchy or memory system look like because there is a physical view because that dominates the cost function and the power function but equally at the same time you have to represent that especially from a performance standpoint you are caching lots of different data.

for computing. So think of the KV caches for the LLM side, the in -memory representations of many of the data from a performance standpoint. So that’s a layer that is continuous, is rich in innovation and technical innovation that we hope to have an influence as well as probably make a mark. And then the large part is the logical view of memory, especially deep context. You want to go from session to session, location to location, and you want to have your memory state. You want to be able to switch models and have some state of the memory state. All of these consume various layers of the logical and the physical layers of memory. So that’s what we think about.

So net, putting all this together, we think of taking a bet with interrnet, taking a bet with memory, and build an infant -scale compute for population scale, like in this case India, but also in certain key verticals like health sciences and others. So… So there’s another important element we want to highlight. I can’t take a quote from Jensen.

One of the applications that my favorite is just good old -fashioned data processing, structured data and unstructured data, just good old -fashioned data processing. And very soon we’re going to announce a very big initiative of accelerated data processing. Data processing represents the vast majority of the world’s CPUs today. It still completely runs on CPUs. If you go to Databricks, it’s mostly CPUs. If you go to Snowflakes, mostly CPUs. SQL processing at Oracle, mostly CPUs. Everybody’s using CPUs to do SQL, structured data.

So taking a cue from what he’s saying, historically, databases, SQL, all run on CPUs. And that will remain the case for a variety of reasons. so that’s an important metric in terms of why we believe the new systems that we compose going forward needs to have a happy blend especially the ways to design systems for the hyperscalers but also the whole category of use case and customers in the private side where they don’t need to have 100 ,000 machines but smaller scale machines but it needs to have a happy blend of CPUs and GPUs that’s the main point in terms of so in that context we have taken a position to start working in partnership with AMD because they’ve got the x86 CPU assets and a compelling GPU roadmap as well as an architecture that supports both from the network side as well as the memory side they have higher memory capacity for LLM so it started with 256GB of HPM which supports 128 billion parameter models at least now it’s going to go to 288 and 512 and no time which means we can fit fairly sizable models so that enables one to do more kind of classical distributed systems principles of single node that captures most of the workload for most customers and be able to optimize it on that.

So coming back, before we get into what we do in Proximal, I want to emphasize the partnership with UC San Diego. They have a data center, as well as I told, it’s a supercomputing data center for research for NSF and DARPA, where we are doing some of the work in terms of the hardware level at the middle layer, in terms of the compute kernels, as well as in the inference engines, as well as the use cases, as I said, because there’s a data science institute, AI for education, to transform the undergrad and graduate level programs using the same tools to have advanced research capability, as well as for health sciences. So with that, I think that is a part to set the motivation for the future of the field.

Thank you. what we’re doing in Proximal Cloud. The next phase we want to go into specifically what we are launching in the four layers, the key components of what we are building and delivering to many cloud partners in India, starting with. There’s also a why India question. I think I’ll say one aspect is India demands an extremely low -cost infant -scale compute at population scale, and that’s a challenge. So we really are excited to work on that problem to start with. So the first thesis, why do we need compute other than the cloud? I think the best way to quote is Michael Dell telling you what he sees. To the beginning here.

Yeah so we in the last year you know delivered a little over 3 ,000 of these still AI factories and you know those are increasingly to enterprise and commercial customers that want to bring the AI to their data not the data to the AI and you know there’s just a ton of data that is still on -prem and being generated on -prem and it turns out to the beginning here

If you have a particular question in the domain that you understand, we can try it out after this. So we enable with this interactive learning for the students, contextualized intelligence, and, of course, instructor empowerment in that. And the way it will look and feel will be like a Jupyter notebook on the extension side will be the research content, the archive papers for them to use. It’s an add -on thing. It doesn’t have to be integrated. It’s a commercial AI chat, if you will. Then the next example would be MRI images. Unfortunately, I’m not able to log in remotely onto that right now. The other one I had a local copy. I’m not able to show the MRI images right now.

So at this point, I want to summarize saying that what is Proximal? The word Proximal brings compute closer to your data. The word Proximal means it’s sovereign to the nation or the region or the business that cares about it. And the word Proximal also means we bring compute closer to memory. We bring compute closer to where the business is. so that was the thesis we are not doing this alone we are doing it with some technology partners as well as we have some key customers and partners so with that let me give an example for a given education use case. Let’s go to I’ll bring Lalit Bhatt here to talk about who is director here heading the India office for PharmEx the key partner

Thanks Renu So what I’ll do is and thanks to Proximal Cloud for giving the stage out here what we’ll do is that basically first I’ll little bit talk about what PharmEx stands for and then why in this space and different space why local compute and all these things are becoming important so PharmEx is basically a comprehensive AI stack so if you see on the left hand side we have lot of infield sensors and So we have a complete comprehensive platform in terms of not only soil moisture sensor but dendro meters and multiple sensors. We also have imaging capabilities where we can take images using satellite, using drones. And we also have an autonomy stack and we just now have acquired an autonomous electric tractor.

Basically these are pretty big machines. They might look like transformers but they are like almost 70, 80 horsepower machines. And we are putting our autonomy stack here so that they will go completely as autonomous ones. So what I’ll do is that I’ll just run a small clip. And I’ll just run a small clip. Thank you. Thank you. Thank you. again I think this is probably very standard everyone understands this you need to do AI you need data these two things we need that one then what becomes important is how efficient you are in terms of running those inferences using those data and we are also dealing with huge amount of data and that’s where we are looking into this technologies where we can reduce our cost everyone understands that in agriculture it is very difficult to ask lot of money from the farmer so where we can really make our operation more efficient if we start like making sure that we are very efficient very effective in terms of dealing with a large amount of data and able to run inferences on top of that one but essentially that’s what happens we get a lot of data both from the imaging side both from the sensor side and then we have our all engines running which basically leads to diagnostic and recommendations and this is just an example of the kind of thing that we do with our customers.

You would see here like complete or autonomous irrigation scheduling. A lot of data points would go into those models basically to create those schedules, anomaly identifications, crop stresses, yield predictions, frost predictions, and even we have worked on this soil percolations model as well. It depends on what all sensors you take. So in India I can tell you like we sell one, there is two feet four sensing probe, which is like four sensing, it goes two feet one, and with the whole controller unit it says 45 ,000 per unit basically. Usually in India we recommend one unit in one hectare, but again it will change based on the variation of the soil and these things like that, but this has been a good ballpark basically.

So yeah, I guess that’s it. And I think the whole theme is that we also are looking for really reduce our inference cost and that’s where Proxima Cloud comes into picture. Thank you.

Thank you, Bharat. Okay, next we’ll have Bharat, Director at Divium, who is a key partner, and as I mentioned earlier, about model selection and runtime optimization that is integrated or will be integrated into a stack. So, Bharat.

Hey, good afternoon, everyone. So, let’s address the… hard truth out there. 90 % of Gen -AI pilots never make it to production. Not because the demo was bad or the models were weak or bad. It’s primarily because of three reasons. Number one, quality is undefined. What’s good for one use case is not necessarily good for another one. There’s no standardized way for evaluation or regressions. Number two, the costs are unpredictable. Be the cheapest model or the best model, you can see the price of these models ranging different from like 10 to 50x. The moment your application goes into production and hits real traffic, the costs spike up. There are AI engineers who are running experimentations and trying to tune this.

But model selection is always a moving target. There are always new models coming which are trying to fix something and are breaking something else. So without addressing all three, it’s very difficult for an enterprise to take their pilots to production. And that’s why we built Divium. So Divium is the only inference layer built on quality. Thank you. Divium defines measurable evals aligned against each use cases And it optimizes every incoming query to select the model Which is giving you the best quality per dollar The other part is that Divium automates the entire model selection process By continuously evaluating new models Deprecating the previous old ones And migrating you to new ones If we find something better, we auto -upgrade without breaking production Evals first, routing second And that’s what makes Divium different from every other routing platform out there Divium is the only inference layer with customer -specific intelligence Your apps can be AI agents, rack pipelines, or multi -agent workflows And the LLMs can be from the standard OpenAI, Anthropic Be your own fine -tuned models or deployed open -source models We sit right in between We provide you a single API.

We are continuously evaluating each and every incoming request, routing it to the model which is giving you the most optimal performance and also giving you detailed visibility on what models are working, how is your agent performing and what’s the overall quality. Remember, DVM is trained on your data, your agents and your quality. There’s nothing generic out there. And this is just a theory. We’ve already proven it across multiple deployments. For the India’s largest travel aggregator, which runs a conversational shopping assistant in their application homepage, we were able to cut down the cost by more than 60%. For one of the leading e -pharmacies of India, the customer support chatbot had a little bit lower latency.

So we ended up reducing the cost by 30%, reducing the latency, latency by 30%. leading to a case resolution improvement of 95%. As you can see, different use cases, different industries, but the result is the same. Lower cost and better outcomes. And we understand enterprise realities. You can keep your data secure. We have flexible deployment options, be it SaaS, privately hosted, or on -prem clusters. You stay in control. If you’re trying to take your AI pilots to production, feel free to reach out to us. Thank you.

Thank you, Bhatt. So we talked first about application use case, one in education and agriculture. Second one, how we are bringing optimization to the system stack. Some of it we do and some of it with our partners. Third, we want to bring in how do we get customers, many of them mid -market, small, as well as large ones, enabled on our platform. I’m happy to introduce Sandeep Kumar. Coming is part of… venture builder instances. It’s a company that we partnered with in Delhi here to take this to a variety of customers, small, medium, large, with a higher velocity. Let him describe what they can do and how we partner.

Hello, everyone. I’m Sandeep Kumar from Instant System. We are a Silicon Valley -based venture builder. We do not just build startups. We grow them. We are partners in every domain of a startup, be it engineering, be it product, be it marketing. We just give them full blueprint to be a successful startup. We co -invest in the startup so that we are there in every journey of them for them to be a successful startup. We are a venture builder. Sometimes it’s often confused with the incubator, but we are a venture builder. We are a venture builder where we actually help in every step of your startup to be a successful startup. Part of the engine system I am mostly responsible for a company called VanEye though we usually do not disclose the name of the company that we are partners with to protect the IP and the confidentiality but just to give you a use case that you know what our capabilities are and what we have been able to build so far so this is a use case that I am taking this company has got nearly around 200 million dollars funding from the top investors including South Bank we are building an AI conversation software here and we are dealing with real use case real challenges of you know for a mostly like financial domain or financial based industries but all of these solutions are also generic for the analytical based industries as well so I am going to talk about you know some of the challenges that are actually common to every problem or every AI -based solution.

But we’ve been able to identify these challenges and we’ve been able to solve these challenges for this particular use case. So one of the most challenged that every AI -based software face is hallucinations. So LLMs always try to answer to your question irrespective of how much of the context it does have. We’ve been able to solve this problem up to a very good extent and our system is almost 99 % reliable. They do not hallucinate. That was the biggest problem that we’ve been able to solve. Next challenge that we face is disambiguation. So in spite of providing the context, sometimes the system is not able to understand how to disambiguate between some specific terms which may exist in different domains.

So that’s also the problem that we’ve been able to solve. As the theme of the system, and it’s very closely related to the theme of the system, because data security and the data privacy is one of the major industry concerns that we’ve been able to solve. So we’ve been able to address and challenge this problem so that the data privacy and the data access control is being managed at the raw level or in a very technical term I would say at the object level. We’ve been able to tackle that problem and solve that problem efficiently and that’s already running there and working fine. Evaluation and the quality management, that’s also one of the key areas.

That we need to solve as part of the venture that we are building. That’s also that we’ve been able to solve very efficiently. Another thing is the reliability because since we are talking about the financial system, the system has to be reliable. It has to be reliable every time. You cannot send a million dollars to someone’s account by mistake. That doesn’t work in the financial world. Or you cannot report data where you could show losses. instead of revenue or vice versa because you cannot survive in that world with hallucinated or the data which is not correct or factual. So being able to, with our advanced architecture system, we’ve been able to solve that problem as well.

There’s a long list of the pointers that we’ve been able to solve, but I’ll just cut down short. The system that we’ve been building, our performance, the reliable, we’ve been able to keep a check on the cost and efficiency of the system. That’s how we’ve been able to serve to the different audience, different customers from the different niche. So that’s what our theme is. We are a venture builder. Please feel free to reach out to us, and we’d love to talk to you about your startup. We don’t pick a selected startup to work with, but you all are free. You’re welcome to reach out to us, and we can discuss all the stuff that we are doing.

working on. Thank you so much.

Thank you, Sandeep. I think that ends what we’re doing in Proximal and what our partners and our customers are working with in the early phases. We have partners in the U.S. like UCSD and Life Sciences, Health Sciences, Education, and here in Agriculture and soon to other, particularly we’re going to focus more on, it turns out to be that the Government of India initiative of Education, Health and Agriculture coincidentally aligned. It was not planned. It turned out to be that way. With that, I can go back to any questions. We’ve had a small panel session we can go to. I don’t know if Piyush has come here or not, but I think there’s a question here.

I’m here from ARIA, from Infosys, Senior VP at Infosys. please

Hello excuse me my name is Arya Bhattacharjee I am from entrepreneur from Silicon Valley so right now like Renu said I am driving this semiconductor and AI vision for Infosys from the United States and India also so the reason I am here is because like Renu said very correctly a very important question that what’s in future for India how can India capitalize or make a mark in this journey so not a small answer but I can tell you what we are trying to do at Infosys because if India is going to win this Semicon 3 .0 or 4 .0, 2 .0, I don’t know, it has to be in software, it has to be in AI.

The chip building -wise is going to take some time. So Renu said that 80 % of the data is on -premise. And what we are working on together is to see on the semiconductor price, this is true, absolutely true, more than 90 % of the data is on -premise. Yes. So the whole journey of how to take the data and how to create solutions through agentic AI approach, through distributed computing, and actually by owning the architecture to lower inferencing cost is a main challenge. So to answer the question which Renu asked me, what’s the future of India? I think that India… what we’re going to do is we’re going to look at a domain. So at Infosys at least we have selected domain and semiconductor, I was talking to him also, that is a large domain and we have taken the leadership with some major clients right now, I can’t talk about details, we’re using an agentech AI on premise and delivering productivity solutions, AI solutions and by cutting our productivity for chip making at least by 25 % and every day in a semiconductor fab you save $10 million, benchmark for a 7 nanometer type of technology, not even 1 .9.

So with that, good luck to Renu and I look forward to collaborating. Thank you.

Thank you, Arya. Now we welcome Abhishek Venjan but before, just come on board. To summarize what we do, graph, that is underneath what I think is the most important AI factors. Organizing the data layer turns out to be probably the most complicated thing, which spans the enterprise such that it can meet the intelligence. And so that’s the stuff that I think we’ll probably do a lot of. We still don’t really have deep research in a corporate context. We do. That’s what Copilot is about. But most people day -to -day do not have this. So are they just underusing AI that exists? Yes. In fact, it’s interesting you brought that up because to me that is the killer feature.

So the biggest thing we did was we took this graph that is underneath what I think is the most important database in any company, which is underneath your email, your documents, your Teams calls, what have you. It’s the relationships that, by the way, own AI factors. Organizing. Organizing the data layer. so that’s a best summary obviously Satya wants to do it in the cloud and that will happen but also you need to have it in your on -prem, near -prem isolated from other sovereign as well and have the same capabilities, in a sense that’s what we bring to the enterprise if you will any other questions before we go to a panel session

Thanks for having me here this is Abhishek, founder of ZetaVault we did a lot of work on the LLM acceleration, what it means is that we offload the large language models to the specific chips and custom silicon and thereby get the inferencing states and all we have Renu here who has wealth of experience on the distributed computing side and we were supposed to have a panel discussion but I thought I would pick his brain on what the challenges and what kind of changes he has seen in the industry. So, Renu, like, you have been part of, like, Sun Microsystems and early sort of, like, pioneers of distributed computing. So from Sun, which was maybe the distributed systems 1 .0 to Linux, which pretty much democratized the entire competition space and brought the Linux and x86 and now almost every embedded device, every competition pretty much happens on Linux.

So that was the distributed systems 2 .0. And now coming to the distributed computing space with the open models, right? Open source has played a lot of sort of role in the proliferation of the distributed computing. What do you foresee or what do you envision the open models are going to do for the distributed computing? Are we going to see a distributed computing 3 .0?

Hello. Yeah. So that’s a fundamental thesis in that. I mean, we are, in a way, in part of that continuum to some extent. If you look at… not to take anything away from how NVIDIA designs, but there is a clear bifurcation going on right now as we speak on, as we said, training versus inferencing. And then there is open source models, and a variety of customers’ use cases would use and need the open source models. There’s always been the history of open and close in every transition. I mean, if you go back in the 80s and you go back, I mean, if you look at what enabled the cloud was hypervisors. There was KVM and VMware.

The same thing will apply. There will be open models and closed models. But the way I like to think about it is models is a new abstraction layer that separates between the underlying computing needs and everything above. Hypervisors separated the physical machine to a virtual machine, and then operating systems unix at that time also did that. The same thing is models are the abstraction layer that provides a higher degree of innovation both from closed and open models. The closed one will probably be innovated within OpenAI and Google, but the rest of the world will take the open source model, like what happened with Linux, and innovate. It’s not just going to be an NVIDIA GPU or AMD GPU, or there could be a plethora of GPUs, country -specific, region -specific, domain -specific.

Anything can happen over time.

That’s a very wonderful take. One of the things that we have been wondering is about the latency you talked about in the various scientific and other applications you’re working. When we build the solutions for our customers, we build a lot of, actually, natural language to query processing kind of solutions. Like, we have been able to do maybe a sub -minute kind of a solution, which is acceptable to the customer because from weeks or days, he is able to answer or get the answers to their queries in less than a minute, right? But even a minute is not sufficient, right? When you talk about, like, really interactive queries and all, you want, like, sub -millisecond. Or maybe, like, sub -second kind of response.

what are your thoughts on that? Is it even possible that to a population or to a large customer base that we have in India about 1 .5 billion people at a very low cost, maybe like 200 rupees per month, you can provide query processing at a scale which is like within sub second?

I think that’s a very good question. Sometimes scaling the problem is more important than the answer. This is an interesting way to frame the problem is, if you go back and look in history, why did Google succeed? A fundamental decision that was made by them on the toolbar is every query response has to be in 20 milliseconds. Now, nobody thought about it prior to that. It’s obvious today. But that key proposition or definition or the question that was asked, maybe by Larry Page or Sergey Brin, whoever it was, led to what we see as Google today in the back end, which is a huge amount of infrastructure to satisfy the 20 millisecond response to any query.

so to me the same thing applies today, maybe 20 is too hard I’m just going to arbitrarily pick I have a simple demo or animation thing I was trying to show every 120 milliseconds you want to have the answer today if you go ask a question it will take seconds sometimes longer than that we are all impatient, we want the answer in quick order, when I ask you a question you don’t say let me think and come back, you want to give the answer if you don’t know how to think and come back, ok, you can but that’s a very deep question you can think and come back but we can throw more computing resources to get that so what it tells you is you can throw a lot more computing to get to the answer, it’s not just hardware, it’s going to be algorithmic improvements other improvements, but to me that’s the benchmark, get 120 milliseconds to any query for anybody so there’s a global context and India context, India provides an ample opportunity for 1 .4 billion people if you can deliver at a cost point and you can deliver at a cost point like 200 rupees a month at 120 seconds and any query to be handled which is a long road to go but if you can meet the objective in 10 to 20 years it serves a lot of people but it also will drive tremendous amount of innovation that’s why when somebody says population scale unique India has a unique thing about the population scale problem and the cost problem so hopefully there are enough people within as Arya said here semiconductor 3 .0 and other innovations that can drive to build India’s own sovereign lowest cost, shortest answer to any language to the question that you asked

interesting take on that one of the things that keeps coming is about the scale of the global corporations or the size that they have been able to reach with AI gaining mainstream in India. And there is a parallel actually theme going on, which is on the semiconductor side, right? Like we are putting, the government is putting a lot of focus. Private players are putting a lot of focus. We have an audience like esteemed, like our Infosys guest. And the question that everybody keeps wondering about is with the AI and AI speeding up things, he’s talking about productivity gains and all. Like, can we, like what kind of corporations can come out of India? Can we see like NVIDIA is coming out or, I don’t know, like Palantir or Supermicro or even a new version of some microsystems coming out just because there is so much emphasis on AI or the Semicon side.

I’ll let Renu talk and then maybe you can also have your take on this particular question, right? What kind of corporations can come out, right? Your take.

transition, can there be an SAP coming out of this transition in the AI? Why not? To give you some raw numbers, every gigawatt of power will require $25 billion worth of compute memory network storage. So if India is going to do 10 gigawatts, that’s $250 billion of hardware. That brings multiple super micros, or that sustains a semiconductor ecosystem at that scale. So certainly the investments going in for power, which is a long lead item, is important, but the next layer provides the economic value to host the hardware systems company, the HP, the Dells, and Supermicro can emerge. You can go each layer of the stack. The next layer is the application in your tier. Proximal is that.

Maybe we could become the SAP of tomorrow. That could be a Palantir, which is the application tier, not just Palantir, any other company. So both the scale and if you can solve the technology, the scale, and the cost economic it’s not just restricted India it can be global unlike the China model where it ended up being a very close garden wall I think India has the opportunity to be make in India and make for global it’s much better but you just have to think bigger take more important is take bolder bets and go for the long haul not just work for it for 5 years 7 years these are 10 20 years cycles to change very interesting take and thanks a lot for actually and I would like to have your opinion on this particular question do we see NVIDIA SAP’s and Oracle’s coming out thanks sir so I think the semiconductor data for example recently working with more than large companies I want to give some specific examples they are just ingesting data right now just I’m talking about a fab not design they have got 7 petabytes of data ingested and they don’t know what to do with it And like I said, a typical fab manufacturing facility is worth at least $10 billion.

And it’s got thousands of steps. It takes about 120 days to make a chip. So $10 billion for 120 days producing a wafer, and there are defects, design issues, things. So if you take the data just from basic information, run -time, real -time data, defects, soft defects and hard defects, because, you know, just because a chip fails doesn’t mean it’s slow. Slow means no money. That’s a failure. So collecting all that data, classificating the data, understanding and using agents in an edge computing way. You cannot solve this in a server. And then feeding it back to design infrastructure. So the design time also has shrunk a lot. And the yields are going up. 30 years ago when I was in Intel, we were talking about die sizes of like maybe one centimeter by one centimeter.

Today, in a 300 millimeter wafer, NVIDIA’s latest wafer level ship is about 20 centimeter by 20 centimeter. That level of yield and reliability is unimaginable without the use of AI. So I can go on and on, but I think if India has to win, I don’t think India needs to become a Palantir. And India does not want to become a slave shop. So the way I explain that in a one level page, the Palantir’s gross margin is 95%. Indian company’s gross margin is 30%. Can we build a business at 50 % gross margin where the amount of domain expertise India provides with the amount of data is available to take these technologies we talk about to implement them in real practice?

That’s what India can win. it is the execution with the best technology thank you thank you

thanks everyone I have one question for the venture side so like all these like technologies require a lot of investment before they can actually become fruitful right I heard somewhere that the government of Karnataka and I don’t want to demean them by the way I put like 20 crores of fund for actually funding the startups and all that right the single engineer which actually Meta is hiring right now they are throwing how much 100 million dollars at that engineer 20 crore for funding like hundreds of startups versus 100 billion dollar like being given to one engineer right there is a huge mismatch now the question is do we like for Indian companies do the venture capitalists or this or the private equity do they have such deep pockets to fund them for like continuously fund them for hundreds of millions of billions of dollars so that an Nvidia or AMD or I don’t know like the Sun Microsystems can

I want you to take. Answer this question. Actually, I would like to take your. I don’t want to answer. I want you to answer the question. Answer your own question.

I want to answer my own question. Yes, it will require that kind of investment, right? I think this was one topic I touched upon a long time back. ISRO has been funded like continuously, right? Initially, the ISRO rockets would all land up in the ocean, right? Or the sea or whatever. But over a period of time, they gained competence. They are among the top four in the world right now, right? I think that kind of like continuous and continued support is needed for whatever industry we are picking, whether it is AI or whether it is Semicon. Like we need the private players. We need the government to support it like till the end, right? And that’s when maybe the key players and the winners will emerge.

I think your question has got two parts. I think the first part is that the government is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going to be the one that is going Sorry, there was a public announcement.

There’s an interruption here. I think there are two parts. One is there’s a mismatch between what a demand supply gap and skills in the model companies, if you

Hopefully, yeah.

So why did you do it and what do you think? That’s why I asked the question back to you.

It’s good. It’s fun to build for India, by the way, and build from India, right? Build for India, build from India. That’s why we are here. And that’s why all this conference is here. All the discussions are happening.

But thanks a lot, Renu, for all the wonderful insights. Last call. Anybody from the audience wants to ask any question to Renu?

Yes, sir. Thanks, sir. So my question is that you shared that if 10 gigawatt business comes to India, that means $250 billion worth of equipment will be purchased or something will happen in India. so how can we ensure that 10 gigawatts just leave 10 let’s start with 1 so how will that business come to India how will that I am just sharing that if 1 gigawatt business you said will come so how will that business come how will that business come

today we already see most of the hardware is either broadened by the hyperscalers who got some capacity and then Dell HP are the largest OEMs as I understand super micro is behind I guess most of the hardware level systems are manufactured in Taiwan and other places and brought here and there are emergent players like VVDN Sanmina has got a manufacturing plant in Chennai who is going to come and do make in India I don’t want to steal the thunder so there are emergent ones seeing the economic value of that scale to start designing but we have already seen I don’t know the details of all the phone manufacturing that’s happened So the ecosystem of building chassis systems, board design, design capability was there, but manufacturing and operations support and all that was not there.

So I do expect that to start happening. That’s why we started working with CDAC and VVDN to some extent. We do see the opportunity that there’s at least a $300 to $500 million opportunity. If you look at the interesting aspect is the Indian public market is also valuing these things fairly high. Look at NetWeb and others. So you can’t go and raise money in the public market for these kinds of businesses in NASDAQ. You can certainly do that in India. So it’s an interesting point in time where there’s a demand, there’s a need, there needs to be enough people willing to invest. And there’s also probably a way to scale the business. I don’t view going public as an exit.

But really? I’m viewing going public as a way to raise money to scale the business. So there’s enough financial muscle that’s getting built at all stages. But the question is, are there enough people funding at the early phases to fund some of these, right? That I think they have to come together. I’m on the entrepreneur side, not the venture side. I’ve played both, but that has to come. That’s my point of answer to Abhishek’s question is, at a 10 gigawatt, it’s going to be multi -hundreds of billion dollars worth all the layers of the stack, and there should be enough investments going in. And if you look at what has happened in China, there’s a different way to drive that capitalistic structure, right?

They have taken a centralized model, but enabled a lot of districts and regional people to go invest. Look at the cars. How many car companies are there? I’m not saying you should follow the same model, but there should be enough early stage at various layers of the stack to be invested. So, the opportunity, the exit

Thank you. Thank you, Renu, for all that, and everybody who has participated. Thanks for coming, guys. We’ll close the session here, so thanks a lot. Thank you. Thank you.