Good afternoon everyone and thank you for joining this town hall discussion where we will be talking about a topic that university and education leaders are all buzzing about which is namely dilemmas around knowledge.

This has been a topic for us since schools were first invented, libraries were first invented and it’s still with us today. It’s extremely relevant today in an age in which AI is changing, is making knowledge available broadly to everybody all the time, but it doesn’t mean that there aren’t still dilemmas around knowledge and we’re going to probe these today.

I’m Professor Deborah Prentice and I’m the Vice-Chancellor of the University of Cambridge. I’m very pleased to introduce you to our panellists for this session. So we have Aidan Gomez who is the co-founder and Chief Executive Officer of Cohere, an enterprise AI company developing advanced language models for use by business and we also welcome Hugo Sarrazin who is President and Chief Executive Officer of Udemy which provides a wide range of business and leadership development courses including AI courses to businesses and organisations around the world in fields such as financial services, higher education, government, manufacturing and technology.

We have some fascinating questions to discuss this afternoon around knowledge, misinformation, AI, attention spans and even the nature of expertise. We’re going to bring the audience in early and often so I hope that you’ll all participate with us. We as panellists come from very different perspectives.

Aidan and Hugo run very successful businesses selling a product. They are from the for-profit educational technology sector and I’m from the not-for-profit sector so there are different pressures, different opportunities, different challenges that we face in this space.

Before we get started with our panel discussion, I’d like to remind the online audience that if you are sharing with us through your social channels, you should use the hashtag. Hashtag WEF26 and whether you’re joining online today or here in person and it’s great to see so many of you here. Thank you so much for coming.

Please feel free to get involved in the session by reacting to the questions we discuss in our conversation and also by submitting questions to panelists via the Slido app. Okay? Okay, so our first question is, in a world of instant answers and AI assistance, what is becoming the scarcest resource?

Okay, the answers are from a list of options. Is it sustained human attention, independent judgment and critical thinking, deep understanding and mastery, motivation to learn in the first place, or trust in what we know and who to believe? And actually I said or, that could be and.

You can choose as many of these as you as you want. Okay? So you can see on the screen, actually, as people are responding via the Slido app, but I want to ask our panelists.

What would you say? So you can see the answers on the screen. What would you say, Hugo?

Well, I think it’s a complicated question and I think there’s a lot of all of the above. If you take a historical perspective, knowledge was scarce. That was a source of power.

You know, countries fought for that. And we also had experts that built knowledge over time, but very few polymath. Very few.

Those ones that were, were very, very, very important. Now today you have LLMs that can learn everything and they can learn across different domains and they can become the polymath. So every data center, every time we say there’s a new infrastructure that’s being added, we’re adding millions and millions of polymath.

And that becomes a democratization of that knowledge. The problem is, and you know, there’s, there’s an amazing quote from Herbert Simon, when you have a wealth of information. you have a poverty of attention and I think that’s what’s happening for a lot of learners and that’s why traditional methods need to change and we’re gonna come up and talk I’m sure about how learning needs to evolve, what the process, what’s the role of traditional institution and changing, what’s the role corporation need to and what individual needs to do.

So I think attention is one big component. The second is a lot of when you go to LLM and AI and you ask for a question, it will give you an answer. It will feel very comfortable with that answer.

It doesn’t explain. Explainability in AI is a whole field, a whole domain and most of these LLMs don’t give you that. So if you have a society that begins to rely on products that give you an answer but don’t tell you where that answer came from, how do you learn and what do you have in terms of trust?

So I think the trust piece is also equally important. So I’ll stop at that we can go well further.

Yeah I was looking at the the poll up there and for whatever reason that the first one that came to me was deep mastery which seems to be the most unpopular choice. So I think you know when you exist in a world where it’s so fast and easy to get answers to whatever question you might have or to get a very surface level answer to even a complex question like whatever like how does quantum mechanics work?

It’ll give you a four paragraph response but that’s not deep understanding of the subject matter and so I think LLMs can’t chatbots. They can fool you into thinking that you understand something when you don’t and I view that as a core risk as we integrate these LLMs into an education environment is this false sense of mastery or understanding.

You know we can discuss the the different solutions to that. I think that testing is essential to it. The idea that you need to take away the tool.

and see what the human alone understands and has retained. The ability for you to assess depth has to take away those tools. And I think that is, from my perspective, what’s most at risk.

Yeah, it’s interesting. You know, my answer is a variant on yours. I wanted to, I, of course, wanted to reject all five.

But I think it’s because of where I come from, coming from the university sector. I mean, I want to say self-knowledge for the learner, right? And it’s part of what you’re saying.

I mean, you don’t know if you’re mastered and you don’t know if you’re interested in it and you don’t know if you get it, right? It comes to you. So much of what you learn, so much of what you learn comes from what is difficult and what is compelling and what it, so for that, for those cues to no longer be actually, you know, useful cues for self-understanding means how will you even know?

But that’s my answer anyway. So we can see what the, whoops, it went away. I think critical thinking was the one that won out at the end.

It looked like critical thinking was actually the audience preferred. We can keep coming back to this. But I want to use this as a jumping, oh, there we go, okay, yeah.

Critical thinking and then sustained attention. They were neck and neck for the most of the time, yeah. And then trust and then deep, deep mastery, right, it’s interesting.

So I want to talk a little bit about each of what you do. So we can start with you, Hugo, you know, tell us about Udemy.

So Udemy is a 15-year-old company that at the time did a pretty cool thing around introducing online learning. There was a great innovation to change accessibility and the cost of reaching out to millions and millions of people and created a creator economy around that. So we now have 250,000 courses.

80 million learners on a regular basis. We serve 17,000 large enterprises. We have 85,000 instructors that kind of come to this marketplace to offer their ware.

They’re very deeply committed. They know stuff and they want to, you know, share it to the world. And we do it in, you know, about 40% of our revenues are in the U.S.

The rest is around the world. So we’re in tons of languages, 46 plus. And the funny story, I’ve only been in the role for less than a year.

When I came in my first town hall and the people who may be listening online who were on that town hall, I came in and I said, we’re going to exit online learning. That is a wonderful innovation. It did a bunch of great things, but it doesn’t solve the problem of today.

And with AI, we can do so many different things. So I want to make a hard pivot of the business toward becoming an AI platform to reskill the workforce of the future. And we can talk about that.

And I couldn’t, I don’t want to take too much time, but there’s a lot of, you know, ways you can use AI to do some of the things Aidan, you were suggesting to kind of help build the mastery, how to do assessment using AI, how to use AI role play to immerse people.

And it also does the thing that I think is so, so important, traditional online learning and actually traditional learning. You’re an instructor and you teach to the average, right? You create your curriculum and you think you’re going to hit like the most of the people you can’t get for the super fast, you can’t get for the super slow.

And the same on online learning. And then different people have different starting points. And we don’t have an easy way to accommodate that.

Now with AI, you can do a quick assessment, you can break apart the class, you can have feedback loop and reinforce that in a very, very powerful way. And I think that’s one of the things that’s going to emerge of using AI to kind of reskill the workforce is going to build on that previous generation of online learning to do something pretty remarkable and quite different moving forward.

Thank you. Aidan?

Yes, so COHERE builds large language models, so we’re one of the developers of this core piece of technology that powers things like ChatGPT and all these different applications. We’re focused purely on the enterprise side of house, and so we work with businesses to put those models to work inside the organization. We give them access to internal data and systems that the humans have access to.

And then we teach or we work with our customer to teach the workforce to shift their role from being the ones individually doing the work to managing a team of these models or agents to carry out that work.

Our big differentiator is on the security side, so there’s no data exiting our customer’s perimeter. Instead, we send all of our models and software to them, and they keep it self-contained. Yeah, so you have certain customers who will only subscribe to you, right?

Yeah, certainly critical industries, financial services, telco, healthcare, and then of course government applications as well. Anything that’s a national security concern, and arguably education is within that remit, that’s a place that we do extremely well.

That’s interesting. So, Hugo, what can we learn from the arc of progress from MOOCs and online education to now AI-driven?

I think a few things. The first one is, you know, if you look at the traditional learning processes and methods that we had, there was a void. And that’s why online learning took off, and that’s why there’s a whole industry.

And it addressed a bunch of problems around, you know, getting to skills, specific skills. and also getting to certification and then helping organization reskill. So that was a very, very, very powerful thing.

What is now becoming a lot more a priority, and in the last six months I spent an enormous amount of time, I spoke to 400 CHROs and Head of Learning and Development in large enterprise. So the pattern that I saw is they had an enormous proliferation of tools and things that were bothering the COVID era. Very few could explain the ROI.

How do you measure the ROI of learning? It’s a really good question. And everybody kind of defaulted to, did they take the class?

Did they complete the class? Hours of learning? And as a business leader, it’s not particularly helpful.

And it gets even worse. When they get certification in Google Cloud or AWS or cyber something, to know that you’ve certified yourself two years ago, I’m a business leader, I want to know, are you current? Are you relevant today?

So I think the arc now is moving in the enterprise to an ability to do in the flow of work learning, do it at bite size, do it in an adaptive way, and we can come back to what adaptive means, and with an ROI, an ability to measure what skills people are deploying in real time.

So you’re now beginning to create a workforce management tool that is powered by an operating learning system.

So, Aidan, you said that you were not as worried about sustained human attention as you were some of the others. How does Cohere solve the attention problem?

Well, I mean, I don’t know if Cohere solves the attention problem. I think it’s definitely a concern. There’s lots of pressures on our attention span.

I think social media, short-form content, driving a lot of that. I’m certainly on the receiving end of that, you know, after 30 seconds because of TikTok, my attention span ends and I need to talk about something else. And also just the way that we do business now are in these short 30-minute meetings where you completely swap context.

And so I think those are difficult challenges not related to AI that are still applying pressure on human attention span. But it has a pretty strong consequence on how people learn and how students can learn when they’re constantly being distracted, when they struggle to sit with material over time. I think AI can perhaps assist in resolving that by its ability to personalize the experience to the individual and engage them more effectively.

And so if you have a generic education offering which, you know, bores some part of the population, excites the other, you’re missing, you’re underserving that population that gets bored. But if we can have a very targeted, scalable approach to, for each individual, giving them something that’s engaging, exciting, if they are auditory learners or visual learners, we can tailor it to them and hopefully keep their attention better than we might otherwise would.

So AI might be part of the solution as opposed to the source of the problem.

Hugo, does your vision of AI comport with that?

It completely matches. And I think, you know, there’s a well-known piece of research from the 80s from a University of Chicago professor. It’s the Bloom Two Sigma problem.

And they did some research where they looked at the ability to learn with one-on-one coaching. It was two sigma higher than the classroom. But the economics of doing that was not there.

That’s why we have these big classrooms, and that’s why there are bigger classrooms for first years, and that’s why it doesn’t deliver the same learning experience. Now, to Aidan’s point, with AI, you can personalize the experience. You can adapt it and you can create feedback loops that a professor cannot today, you know, you’ve got 40 students, you cannot pick up who’s, you know, not easily, some teachers are amazing and they have the ability to do incredible things, but now you have the ability to have that feedback.

So I think, you know, we’re going to see a lot of AI expert tutors and coaches that will have context and that will have been trained on a body of knowledge that is hopefully trusted, hopefully accurate, and will help, you know, in the way that you like to learn.

So if you’re an auditory learner, we’re going to give it to you that way. And if you’re a visual, we’ll give it to you that way. I think that’s a really exciting and promising world we’re entering from that point of view.

So we’re going to go to questions from the audience in just a second. So start thinking about your question. I’m just going to ask one more question of our panelists, myself, which is where do humans fit in, in this brave new world of AI-based education?

I think all of us who are educators know that at some point we need human intervention in the process, even with the most fabulous technology. Where do you think they need to come in?

I think they’re the customer, right? So they’re the ones that we’re serving with this technology. And so we need to create the best possible product for them.

If we just do surface-level education that’s very confirmatory, oh, yeah, you’ve got it, great, you know, a bit sycophantic, then they won’t be effective in the real world when they actually enter the job market.

And so there’s a burden on us as product creators to create the most effective product, to teach people skills and give them knowledge. And I think that AI is actually an incredibly effective tool. tool towards that.

But I do still believe that it’s a tool. It’s like a calculator. It’s something that you can lean on to give you faster answers, more thorough answers.

But we still need to ground ourselves in the human without the tool. And so testing becomes, it’s always been important of course, but I think it becomes absolutely critical now because you can fake your way through an education system much more easily. And so having very strict text testing regimens is going to be essential.

Yeah. I have a variation on this. I do think the teachers, the instructors are part of the, partly the customers, but I do think they’re, you know, they need to be in the loop.

There’s amazing storytellers. They have a way. I mean, if I ask anybody in this room, who was your favorite teacher in high school?

And I paused for five seconds. There’s somebody in your mind right now. What was special about that person?

And you cannot replicate that, but you can augment that. You can make that person now be able to maybe teach you on something that they were not that, you know, maybe like my favorite teacher in high school was a physics teacher. I love the way he presented.

I love the way he engaged. And it was so motivating. My chemistry teacher was not that, but now I can augment, you know, with AI and have the voice, not just the voice, but the way he thought, the way he presented the information be applied to a different topic.

And I think that gets pretty, pretty exciting as well. You finally understand chemistry. I may finally understand chemistry.

I stayed away from chemistry because of that, but physics I love.

Okay. I want to, I want to open up to questions from the audience. So I will call on you the old fashioned way.

If you raise your hand. Oh, you have to, sorry, you have to speak.

Thank you. Anna Van Eels, director of the Levium Trust. I guess learning is a bit like working out it’s got to hurt to be effective.

How do you think AI-enabled tech of various kinds can help with that motivation issue? You’ve talked about the teacher being the one that absolutely motivates, but a lot of the systems we’re talking about in the workplace, etc., you’re not going to have that human in the loop. So can we do things with AI and tech that could prompt that?

Yeah, I’m going to offer a few suggestions. And this is not like future, this exists today. So you can do AI role play in a way that makes you go through the learning process.

And I’m going to use business example. So if you’re a new salesperson and you have a new product that you need to sell, you can load up the specs of that product into an AI role play and practice selling to a person. And there will be a rubric against which we’re going to score you.

And we’re going to discover whether or not you are competent at selling this product that you’re responsible for. So that’s a business example. I can do the same thing in a call center.

We have one of the largest call center outsourcers. There are 20,000 call center agents they need to onboard every month. That is incredibly complicated.

But now you can load the most common error cause, the most common tickets, the product specs, and instead of taking three weeks to onboard somebody, through the process of learning, of experimenting, you can do a role play and get to accelerate that learning by doing a lot of practice.

So it’s simulation. So that’s one powerful example. I think the other one is AI can give you feedback and monitor the progress you’re making in the way that we can bring you back to that point in the gym where you’re struggling with whatever exercise you’re doing.

We’re going to make you do that exercise more and more and get that repetition in a way that reinforces the gap that you have.

Hi, I’m Nathaniel. I run an education company in Australia. Now, as a region, Australia has an interesting relationship with technology.

As many of you may know, we’ve just recently had a social media ban for young people under 16. And in a similar vein, we don’t really have a good consensus around the role of AI in classrooms. So my question is, what do you believe the role is for AI in physical classrooms?

And what would you say to people who might be on the side of banning versus not banning it?

Yeah, I think I’m interested to hear your answer. But from my side, I think it’s a tool like a calculator. I think also, a duty of the education system now is to teach people how to use this AI, how to engage with it, how to most effectively use that tool.

And so it certainly should exist as part of the classroom and as part of schooling. But like I said, it can become a crutch, and it can be used to cheat. And so we have to come up with ways to ensure that students aren’t misusing it or using it in the ways that are unproductive to their learning.

I’m excited to hear what you think.

I’ve got two-part answer. The first one is any business process or any endeavor, you have the problem statement asking the right question, you have the solving, and then you have the quality assurance in the back. It’s a feedback loop that you go through a circle all the time.

And education is no different. What AI does well is that middle part. It doesn’t do a whole lot in the front end and the back end.

So what we need to teach young student and adults is how to ask the right question, the critical thinking. I love that it came out at the very top. Super, super important.

But you can, as you said, the calculator is a calculator. Like the fact that I can’t do multiplication table all the way to 100. is not that relevant for my day-to-day job.

But the fact that I can be critical in my thinking, I can summarize, I can contextualize, I think those are the skills you want. Second part, for those who are curious, I have no relationship, but I am just fascinated. There’s a school in the U.S.

called Alpha School. And they’ve got a really powerful model. They are using AI.

They’re encouraging students to use AI. And they’re demonstrating that I’m gonna get all the stats wrong, but they get twice the learning in half the time or three times the learning half the time. And then the kids in the afternoon, they go learn and learn how to be a civic leader or a leader in all sorts of other contexts instead of spending all their time where historically you would have learned various dates.

It’s not that relevant to know the dates of specific things, but it’s relevant to understand the context of those events. And I think that’s where we can focus a lot of the effort.

Thank you. Terrific topic to be discussed at Davos. I’m Pranjal Sharma.

I’m from India. I’m an author and analyst. We’re looking at a lot of the micro pieces, but I’d like to focus on the macro.

We have a situation today where we’re all skilled up but nowhere to go, right? Last year, I think ILO says 7 million fewer jobs were created, not to mention the existing jobs that disappeared. So there is a cry from the industry.

Firstly, they don’t know who to hire and why to hire and what to hire, and they don’t even know what to test that credentials on. The second part is there’s a huge disconnect between what they want and what academia is offering. Plus the concept of a degree shouldn’t exist, and even continuous learning in terms of applied knowledge is missing.

So I think the core phrase to be used here is applied knowledge. How do you create? information for a person to be able to earn a livelihood irrespective of white gray blue color and I think that’s the gap of applied knowledge delivered in the right way to the right people at the right time.

From a labor market perspective I think there’s a a good case to be concerned about the impact of AI and what might happen and reskilling is going to be an essential component of that. The mismatch in the market between what education institutions are offering and what the market is demanding I think that is a major issue that we need to figure out how to solve. I think AI can be a part of speeding up delivery of new programs and courses and keeping up with changes in demand much faster than we have in the past.

The process of scaling up educational infrastructure to meet a shift in market demand has been historically extremely slow and laborious but with AI we’re able to create programs much faster the models are you know infinitely scalable they’re always awake 24-7 they’re extremely they never get annoyed at the student right so we have these incredibly compelling tutors to deploy at scale against the problem of teaching the population the skills that we need but I think the issue might be in identifying the skills that we need and that’s still gonna have to come from us the humans the business leaders the the policymakers so that might be the core constraint we need a direction to be set against to start building the solution.

I think too I mean what I would say is I think that that you know universities aren’t teaching to what businesses need necessarily they’re teaching we’re teaching things that we believe are fundamentally important and I would defend that I mean we’re teaching critical thinking and we’re teaching deep mastery and we’re teaching them to people at a critical moment in their lives most of them where they actually really need to have a go and and learn these skills they may need additional skills when they go out into the workplace and that as far as I’m concerned is what the kinds of products that you’re talking about are for.

Let’s go back to the critical thinking because now in the university the students widely use the AI assistance and get the instant answer. In that case, how can we teach them to increase their capability of critical thinking to make factual check, logical check, scientific check, ethical check, to the instant answer they got from models?

From my point of view, you’re on to one of the core issues. We need to start teaching a very different set of skills. You need to, in my little model of ask the right question, get the answer, and then check the answer.

The middle part, AI is going to outdo the human. It is already a foregone conclusion. The AI will outdo the human.

Where we can be competitively differentiated versus the AI is in the front and in the back end. We need to adapt the curriculum to make sure that people are asking the right questions with the right context. It is critical thinking.

It is critical thinking, but we need to expand and we need to have a better way to evaluate the level of critical thinking these students have when they hit the workforce so that you can evaluate. Then, the same on assessing, AI is marvelous right now. It generates codes like there’s no tomorrow, but it’s mostly garbage.

We have bottlenecks in quality assurance in the back end. How do you create the new tools and teach people to have the critical thinking to see if this is using the right library, is it using the right pattern, is it using the right data? I think that’s one of the core change that you know academic institution organization like me an individual need to do you know as you do your self-development you need to kind of really lean into this this ability to ask the right question because the middle part you don’t have a competitive advantage you will be outgunned and the thing that is even more crazy historically like people did PhD I have a PhD I went like super deep on one little topic and I got buried somewhere in the sinkhole and it took my entire you know body of effort to get there and to be a polymath is very hard to be able to understand I don’t I know nothing about chemistry I know nothing about biology psychology my dad did that so I got something rubbed up on me maybe so but AI is a polymath by design it has the data set across all of that so the middle part is a foregone conclusion folks you need to get good at the front and the back end

yeah well I was gonna say another thing which is teaching is a skill in the same way coding is a skill or doing math is a skill and so it’s a core capability that we as model developers need to invest in and it’s not something that is easily benchmarked and it’s not something that is accurately tracked at the moment but I think the more this rules I mean it’s already in the hands of every student on the face of the planet it’s going to become imperative that we’re able to track the performance of models in teaching tasks to ensure that they’re actually effective and improve that over time that’s just so like a technical level that is not done presently I don’t know of a teaching benchmark but I can point to probably 30 code ones 50 math ones you know biology etc right

I think that psychology is rubbing off well. When you say AI is a polymath by design, it’s a brilliant thought, you know, it was, you articulated it very well, which also means that by definition humans cannot compete. So we basically have to end the session and say that doom is nigh.

Well I don’t think so. I mean I’m more optimistic. So the polymath thing is real.

I mean, you know, like if you do, again, historical perspective, he who had Leonardo da Vinci on his team, you know, had an advantage to build war machine or, you know, a better court or whatever. Now there’s going to be a similar debate like who assembles these polymath AI thingy has an advantage. Okay, that that is a foregone conclusion.

That’s why there’s all these, you know, battle for, but I think we cannot as the human race give up that ability to influence. I think that we made a point, I think you did at the very beginning, like these models typically are not designed, though some of them can be designed to explain their reasoning. So if as a society, we begin to rely on this thing that is super facile, that gives us an answer and we don’t have the questioning and we don’t kind of do the checking and the validating, we lose agency on important decision.

And I think that is one of the things that we need to focus on deeply as a society. It also leads to the guardrail, the ethical things and all that other stuff. We need to go there because in the middle, it’s going to come up with answers that will be amazing in biology and will solve things in biology because it got trained in English language.

I don’t know. It’s going to take pretty, pretty while, but we cannot lose agency around this polymath. I mean, every data center is going to have hundreds of millions of polymath in there.

Yeah.

I just want to share a thinking. I believe there’s a type of paradox within companies about this critical thinking. Let me say it this way.

We senior professionals, we know how to judge what the AI is doing. So I ask her one day for the AI to do a model, whatever, and I could judge. My juniors, they were not able to judge because they don’t have the experience.

But to some extent, I could fire them because I don’t need them anymore because of these AI technologies. But maybe there will be a gap. So at some point in time, AI can enhance a lot what I do.

But if you don’t train, let’s say, the new generation, the junior, who will be the future? Who will be in the future able to do this critical thinking on what AI is doing? I don’t have the answers.

Obviously, companies need to take efficiency, and we need to do our best to reduce costs, whatever. But I think it’s something we, as a society, will have to think a lot about.

It’s fair. Here, we’ve got one here. You wanted, you were up, right?

Yeah. I didn’t just cold call you.

Hi, thank you for your insights. I’m Kian. I’m the CEO of an AI company called Workera.

I really like what you said, Aidan, on testing the human. And I think in the world of testing right now, there is almost two camps, one that says you can test them with the calculator, or you can test them without the calculator. And there’s also, overlaid on top of it, the risks of proctoring and understanding who’s cheating, who’s not cheating, and what can you tell about it.

So how are you thinking about that idea of testing with or without the calculator?

Yeah, can you tell whether a piece of text was written by AI? It’s really tough. A lot of the detectors out there are total scams.

They’ll say 100% AI, even when it’s not used at all. So they’re extremely overconfident, very high error rate on both sides, false positive, false negative. and but the the answer to that question is you can like you can insert into language models subtle cues to indicate for the reader this was written by an AI you can not sample from natural language language that I’m drawing from right now you can sample from a slightly shifted distribution and use certain words much more than any normal like any human would use and then as soon as those words appear you have a good piece of evidence that this was written by a language model and so us language modeling companies do that we we shift the distribution of the language model so that when its text gets read we have some ability to say you know I can assign a likelihood that was generated by my model so you can detect that to some extent but many of the tools are scams and so I think we need to make better tools and put them in the hands of educators more readily.

On testing with and without the calculator I have a pretty strong focus on without the calculator like I think everything needs to be ripped away and you standing alone as yourself need to prove your knowledge that is like the gold standard test of what you have what you have learned retained but of course like I was saying earlier using the language model is a skill itself and we should have space to test that in which case of course you’re gonna need the LLM in the loop.

Let me let me seize the chair’s prerogative here to ask because I’m curious what you’d both say to this question what happens in this in this brave new world of polymaths and and not showing your work and not explaining your answer to expertise or authority so you know there we have at Cambridge a you know library after library of big books that tell you the truth or that that was always the the idea right you would go look it up somewhere what do you do in a world in which looking it up is no longer There’s not a dictionary.

There’s not a truth

I’ll start. I think most technology go back and forth. There’s a pendulum.

We’re in the pendulum that bigger is better We’re throwing everything under the Sun every reddit quote is now part of you know training every large language model and That is good. It’s gonna give you an average answer for average problem now over time I think we’re gonna come back and say you do need you know specialized trust it and We need to have Confidence that we did use the right source, and I think there will be a space for that at least I want to hope that that will be the case that we’re gonna come back, and we’re gonna have these specialized Model that will not only be rag, but there are gonna be you know Define from from scratch with the right intent and they don’t need to be a bazillion trillion Function points and whatever I mean they they just need to be trained on The expertise and then you do need to trust it and it’s it’s gonna be incredibly important I think we also need a lot of research on explain ability These and and Ben Geo And University of Montreal one of the guys who got the Turin Awards Has been very vocal around this we need to kind of go back and explain a lot more of these These are statistical model, this is all this is is huge matrices, and they’re like weights assigned to different things So the people so this is not a piece of software where you say if then this that this is just statistics So it on averages gives good answers but it depends on the data and You need to come back and put a bunch of tools to put the explain ability into the model and there are ways to do it It’s not yet super advanced, and I think we need to invest in that so that we do You know have the confidence build a trust, and I do think it’s part of the learning question you have because if the models are black box you lose the ability to learn from the deduction process which doesn’t exist it’s just a statistical model there’s no deduction so anyway those are my two two ideas.

yeah over the course of last year there was a paradigm shift in the type of model that gets used now we don’t just use input output direct response models like you were alluding to every model now is a reasoning model and so before it actually responds it has a internal monologue where it thinks through the problem tries to reason about it and then delivers a response it is primitive it’s a year old but it’s getting much better and so I think exposing that to the user and showing these chains of thought this reasoning is an important solution and then like you say rag which is retrieval augmented generation where the model isn’t just drawing on its own knowledge but it’s actually making direct and specific reference to external knowledge so we can plug it into the Cambridge Library I went to Oxford so the Bodleian but and it can cite directly back from from those sources and that that provides some degree of both reasoning and rag provides some degree of auditability so you can have a little bit more confidence in the response because you can check its work just out of curiosity what’s driving that what’s what’s driving the need for reasoning yeah because the models were brittle they would very confidently answer with the wrong solution and it turns out you know humans don’t put the same amount of energy into answering every question but that was the previous the prior expectation on these models you would ask them what’s one plus one and it would immediately respond and put the same amount of effort into answering that question and you would ask it you know to prove some unsolved air dose problem or something and it would put the same amount of effort as one plus one into that that was obviously wrong you know there are some problems that we should spend days, weeks, months, years, decades, putting effort in to solve.

And there are others that can be responded to instantly. It’s just a better, more robust intelligence.

That’s fascinating. We have time for one more question. Anything pressing?

Yeah, I’m very interested to ask the question of, just circling back to the beginning where we said we have like public sector university as well as a technology, a tech platform being in the same room.

The question I have on my mind is that with right now, like in the U.S. especially, education cost is so astronomically high and prohibitive. Lots of people are saying the narrative goes is like there’s no point going to university anymore.

And I would see in that world, there would be a lot of attention turned to online education. I think we’re all very familiar with Udemy. Has, what is the gaps between an online education and an accredited college or elite college?

Has there ever been customer or market demand for online education to move towards a model or imitate a traditional college experience? Like has that ever surfaced as a need? And yeah, just like comparing the gaps there.

I’m going to say something maybe controversial, but it’s fun. The university degree is a bundle. It’s a convenient bundle that as a society we chose to create.

So you learn something, you get an accreditation and you have a right of passage. You know, these kids are at the moment, they leave home, they go and that bundle is a convenient and we bundle that with research because the same people could now pass on their knowledge to others. It is a convenient bundle as a society.

It has worked well for a long time, Oxford and Cambridge are examples of long-standing institutions that had a version of this bundle, it changes over time. Is it time to revisit whether all of these components need to fit together because of the economics and what AI can do to change the economic of delivery? Maybe.

I think the second-

Think it quickly.

Yeah, quickly.

Because we’re running out of time.

And the second piece is just the adaptability. If you have the labor market that moves so fast, you’re now going to begin to put more weight on addressing a specific need for a specific skill. So I think that is a reality in addition to that potential unbundling of that whole experience.

Do you have a good word for the university?

I’m actually interested to hear from the university’s perspective.

Then I’ll just end by saying, I think that they are currently serving very different functions that are, right now university does so much more than provide knowledge that it still is worth its weight in gold, and it is gold.

But we’ll see how the space develops, right? And with that, I’m getting all kinds of signals from the producers, so we’ve got to end it. But thank you very much.

Thank you for your questions. And thank you to our panelists. Thank you.