Andrew Ng: Artificial Intelligence is the New Electricity at Stanford GSB (Transcript)

So as Ravi mentioned, right now I lead a large AI team at Baidu, about 1300 scientists and engineers and so on. So I’ve been fortunate to see a lot of AI applications, a lot of research in AI as well as a lot of users in AI in many industries and many different products.

So as I was preparing for this presentation, I asked myself what I thought would be most useful to you. And what I thought I’d talk about is four things. I want to share with you what I think are the major trends in AI. Because I guess the title of this talk was AI is the New Electricity. Just as electricity transformed industry after industry 100 years ago, I think AI will now do the same. So I share with you some of these exciting AI trends that I and many of my friends are seeing.

I want to discuss with you some of the impact of AI on business. Whether, I guess, to the GSB and to the Sloan Fellows, whether you go on to start your own company after you leave Stanford, or whether you join a large enterprise, I think that there’s a good chance that AI will affect your work. So I’ll share with you some of the trends for that.

And then talk a little bit about the process of working with AI. This is some kind of practical advice for how to think about, not just how it affects businesses, but how AI affects specifically products and how to go about growing those products. And then finally, I think for the sign up of this event, there was a space for some of you to ask some questions and quite a lot of you asked questions about the societal impact of AIs. I’ll talk a little bit about that as well, all right?

So the title of this talk is projected, no, I guess not, all right. I think on the website the title was listed as the AI is the New Electricity. So it’s an analogy that we’ve been making over half a year or something. About 100 years ago, we started to electrify the United States, right, develop electric power. And that transformed transportation. It transformed manufacturing, using electric power instead of steam power. It transformed agriculture, right I think refrigeration was a really, a transformed healthcare and so on and so on.

And I think that AI is now positioned to have an equally large transformation on many industries. The IT industry, which I work in, is already transformed by AI. So today at Baidu, Web search, advertising, all powered by AI. The way we decide whether or not to approve a consumer loan, really that’s AI. When someone orders takeout through the Baidu on-demand food delivery service, AI helps us with the logistics. They route the driver to your door, helps us estimate to tell you how long we think it’ll take to get to your door. So it’s really up and down. Both the major services, many other products in the IT industry are now powered by AI, just literally possible by AI.

But we’re starting to see this transformation of AI technology in other industries as well. So I think FinTech is well on its way to being totally transformed by AI. We’re seeing the beginnings of this in other industries as well. I think logistics is halfway through its transformation. I think healthcare is just at the very beginnings, but there’s huge opportunities there.

Everyone talks about self-driving cars, I think that will come as well, a little bit, that will take a little bit of time to land, but that’s another huge transformation. But I think that we live in a world where just as electricity transformed almost everything 100 years ago, today I actually have a hard time thinking of an industry that I don’t think AI will transform in the next several years, right?

And maybe throughout this presentation, maybe at the end of doing Q&A, if you can think of an industry that AI won’t transform, okay, like a major industry, not a minor one. Raise your hand and let me know. I can just tell you now, my best answer to that. So I once, when my friends and I, sometimes my friends and I actually challenge each other to name an industry that we don’t think would be transformed by AI. My personal best example is hairdressing, right, cutting hair. I don’t know how to build a robot to replace my hairdresser.

Although I once said this same statement on stage, and one of my friends, who is a robotics professor, was in the audience. And so my friend stood up, and she pointed at my head, and she said, Andrew, for most people’s hairstyles, I would agree you can’t build a robot. But for your hairstyle, Andrew, I can.

All right. So despite all this hype about AI, what is AI doing? What can AI really do? It’s driving tremendous economic value, easily billions. At least tens of billions, maybe hundreds of billions of dollars worth of market cap. But what exactly is AI doing? It turns out that almost all this ridiculously huge amounts of value of AI, at least today, and the future may be different, but at least today almost all this massive economic value of AI is driven by one type of AI, by one idea. And this technical term is that it’s called Supervised Learning. And what that means is using AI to figure out a relatively simple A to B mapping, or A to B response. Relatively simple A to B or input to response mappings.

So, for example, given a piece of email, if I input that, and I ask you to tell me if this is spam or not, right. So, given an email, output 0 or 1 to tell me if this is spam or not, yes or no? This is an example of a problem where you have an input A, you can email, and you want a system to give your response B, 0 or 1. And this today is done with Supervised Learning.

Or, given an image. Tell me what is the object in this image and maybe of a thousand objects or 10,000 objects. Just try to recognize it. So you input a picture and output a number from say, one to 1000 that tells you what object this is. This, AI can do.

Some more interesting examples. When you’re given an audio clip, maybe you want to output the transcript. So this is speech recognition, right. Input an audio clip and output detects transcript of what was said, so that’s speech recognition. And the way that a lot of AI is built today is by having a piece of software learn, I’ll say exactly in a second what I mean by the word learn, what it means for a computer to learn, but a lot of the value of AI today is having a machine learn these input to response mappings.

Given a piece of English text, I’ll put the French translation, or I talked about going from audio to text or maybe you want to go from text, and have a machine read out the text in a very natural-sounding voice. So, it turns out, that the idea of supervised learning, is that, when you have a lot of data, of both A and B both. Today, a lot of the time, we have very good techniques for automating, for automatically learning a way to map from A to B.

So for example, If you have a giant database of emails, as well as annotations of what is spam and what isn’t spam, you could probably learn a pretty good spam filter.

Or I guess I’ve done a lot of work on speech recognition. If you have, let’s say, 50,000 hours of audio, and if you have the transcript of all 50,000 hours of audio, then you could do a pretty good job of having a machine figure out what is the mapping between audio and text. So, the reason I want to go into this level of detail is because despite all the hype and excitement about AI, it’s still extremely limited today, relative to what human intelligence is. And clearly you and I, every one of us can do way more than figure out input to response mappings.

But this is driving incredible amounts of economic value, today. Just one example. Given some information about an ad, and about a user, can you tell me, will the user click on this ad? So leading Internet companies have a ton of data about this, because of showing people some number of ads that we saw whether they clicked on it or not. So we have incredibly good models for predicting whether a given user will click on a particular ad. And by showing users the most relevant ads this is actually good for users because you see more relevant ads and this is incredibly lucrative for many of the online internet advertising companies, right. This is certainly one of the most lucrative applications of AI today, possibly the most lucrative, I don’t know.

Now, at Baidu, you have worth of a lot of product managers. And one question that I got from a lot of product managers is, you’re trying to design a product and you want to know, how can you fit AI in some bigger product? So, do you want to use this for spam filter? Do you want to use this to maybe tag your friends’ faces? Or do you want to use this, where do you want to build speech recognition in your app, but can AI do other things as well. Where can you fit AI into, you know, a bigger product or a bigger application?

So, some of the product managers I was working with were struggling to understand what can AI do and what can’t AI do. So I’m curious: How many of you know what a product manager is or what a product manager does? Okay good, like half of you. Is that right? Okay, cool. I asked the same question at an academic AI conference and I think only about one fifth of the hands went up, which is interesting.

Just to summarize in the workflow, a lot of tech companies, it’s the product manager’s responsibility to work with users, look at data, to figure out what is a product that users desire, to design the features and sometimes also the marketing and the pricing, as well. But let me just say design the features and figure out what the product is supposed to do, for example, should you have a light button or not? Do you try to have a speech recognition feature or not? So it’s really to design the product. If you give the product spec to engineering which is responsible for building it, right, that’s a common division of labor in technology companies between product managers and engineers.

So product managers, when I was working with them, was trying to understand what can AI do? So there’s this rule of thumb that I gave many product managers, which is that anything that a typical human can do with, at most, one second of thought, right, we can probably now or soon, automate with AI. And this is an imperfect rule. There are false positives and false negatives with these heuristics so this rule is imperfect but we found this rule to be quite helpful.

So today, actually at Baidu, we have some product managers running around looking for tasks that they could do in less than a second and thinking about how to automate that.

I have to say, before we came up with this rule, they were given a different rule by someone else. And before I gave this heuristic, someone else told them product managers, assume AI can do anything. And that actually turned out to be useful. Some progress was made with that heuristic, but I think this one was a bit better.

A lot of these things on the left you could do with less than a second of thought. So one of the patterns we see is that there are a lot of things that AI can do, but AI progress tends to be fastest if you’re trying to do something that a human can do. For example, build a self-driving car, right? Humans can drive pretty well, so AI is making actually pretty decent progress on that. Or diagnose medical images. If a human radiologist can read an image. The odds of AI being able to do that in the next several years is actually pretty good.

There are some examples of tasks that humans cannot do. For example, I don’t think, well, very few humans can predict how the stock market will change, right? Possibly no human can. And so it’s much harder to get an AI to do that as well. And there are a few reasons for that. First is that if a human can do it, then first, you’re at least guaranteed that it’s feasible, right? Even if a human can’t do it, like predict the stock market, maybe it’s just impossible, I don’t know.

A second reason is that if a human can do it, you could usually get data out of humans. So we have doctors that are pretty good at reading radiological images. And so if A is an image and B is a diagnosis, then you can get these doctors to give you a lot of data, give you a lot of examples of both A and B, right? So things that humans can do, can usually pay people, hire people or something, and get them to provide a lot of data most of the time.

And then finally, if a human can do it, you could use human insight to drive a lot of progress. So if a AI makes a mistake diagnosing a certain radiology image, like an x-ray scan, like an x-ray image, then AI makes a mistake. Then if a human can diagnose this type of disease, you can usually talk to the human and get some insights about why they think this patient has lung cancer or whatever and try to code into an AI. So one of the patterns you see across the AI industry is that progress tends to be faster when we try to automate tasks that humans can do.

And there are definitely many exceptions, but I see so many dozens of AI projects and I’m trying to summarize trends I see. They’re all not 100% true, but 80% or 90% true. So for a lot of projects, you find it if the horizontal axis is time and this is human performance, in terms of how accurately you can diagnose x-ray scans or how accurately can classify spam email or whatever. You find that over time the AI will tend to make rapid progress until you get up to human level performance. And if you ever surpass it, very often your progress slows down because of these reasons.

And so this is great, because this gives AI a lot of space to automate a lot of things. The downside to this is the jobs implication, right. If we’re especially good at doing whatever humans can do, then I think AI software will be in direct competition with a lot of people for a lot of jobs. I would say probably already a little bit now, but even more so in the future. And I’ll say a little about that later as well.

The fact that we’re just very good at automating things people can do and we’re actually less good at doing things people also can’t do. That actually makes the competition between AI and people for jobs laborious.

So all right, let me come back to the AI trends. And one of these I’m going to delve a little bit deeper into the AI trends is, I bet some of you will be asked by your friends afterward, what’s going on in AI? And I hope to give you some answers that let you speak intelligently as well, to others about AI.

It turns out lot of the ideas about AI have been around for many years, frankly, several decades. But it’s only in the last several years, maybe the last five years, that AI has really taken off. So why is this? When I’m asked this question, why is AI only now taking off? There’s one picture that I always draw. So I’m going to draw that picture for you now. Which is that, if on the horizontal axis, I plot the amount of data, and on the vertical axis, I plot the performance of our AI system. It turns out that several years ago, maybe ten years ago, we were using earlier generations of AI software, earlier generations of most common machine learning algorithms, to learn these A to B mappings.

And for the earlier generations of — so this is an earlier machine learning. Sorry, let me call this traditional machine learning algorithms, all right. It turns out that for the earlier generations of machine learning algorithms, even as we fed it more data, its performance did not keep on getting better. It was as if beyond a certain point, it just didn’t know what to do with all the additional data you are now giving it. And here by data, I mean the amount of A, comma B data, right? With both the input A as well as the target B that you want to output.

And what happened over last several years is because of Moore’s law and also GPUs, maybe especially GPU computing, we finally have been able to build machine learning pieces of software that are big enough to absorb these huge data sizes that we have. So what we saw was that, if you feed your data into a small neural network, we’ll say a little bit later what a neural network is, but an example of machine learning technology. If you’ve heard the term deep learning, which is working really well but also a bit overhyped, neural network and deep learning are roughly synonyms.

Then with a small neural network, the performance looks like that. If you build a slightly larger neural net, the performance looks like that. And there’s only, if you have the computational power to build a very large, neural net that your performance kind of keeps on going up, right? Sorry, I think this line should be strictly above the others, something like that, right?

And so what this means is that in today’s world, to get the best possible performance, in order to get up here, you need two things. First, you need a ton of data, right? And second, you need the ability to build a very large neural network. And large is relative, but because of this I think the leading edge of AI research, the leading edge of neural net research is today shifting to supercomputers, or HPCs, or high performance computers or super computers. So in fact today, the leading AI teams tend to have this old structure where you have an AI team and you have some of the machine learning researchers, right? Abbreviated ML. And you have HPC, or high performance computing or super computing researchers are working together to build a giant, to build the really giant computers that you need in order to hit the levels of today’s performance.

I’m seeing more and more teams that kind of have an old structure like this. And the old structure is organized like this because, frankly, one of the things we do at Baidu, for example, it requires such specialized expertise in machine learning and such specialized expertise in HPC that there’s no one human on this planet that knows both subjects to the levels of expertise needed, correctly right?

So, let’s see. So let me go even further into – and the questions that some of you asked on the website signing up for this event, some of you asked about what, evil AI killer robots taking over humanity and so on. No, people do worry about that. So to kind of address that, I actually want to get just slightly technical and tell you what is a neural network, right?

So a neural network is loosely inspired by the human brain, right? And so a neural network is a little bit like a human brain, all right. So that analogy I just made is so easy for people like me, right, to make to the media, that this analogy tends to make people think we’re building artificial brains, just like the human brain. The reality is that today, frankly, we have almost no idea how the human brain works. So we have even less idea of how to build a computer that works just like the human brain.

And even though we like to say, neural network is a little bit like the brain, they are so different that I think we’ve gone past the point where that analogy is still that useful, right? It’s just that maybe, we don’t have a better analogy right now to explain it.

But so, let me actually tell you what a neural network is, and I think you’ll be surprised at how simple it is, right. So let me show you an example of the simplest machine learning problem, which is, let’s say you have a data set where you want to predict the price of a house, right? So you have the data set where the horizontal axis is the size of the house, and the vertical axis is the price of the house, square feet, dollars. So you have some data set like this, right? And so well, what do you do? You fit a straight line to this, right?

So this can be represented by a simple neural network, where you input the size, and you output the price, okay? And so just this straight line function is represented via a neuron, which I’m going to draw in pictures as a little circle like that, okay. And, if you want a really fancy neuron, maybe it’s not just fitting in a straight line, maybe it’s I don’t know, at this smart you realizes that price should never be negative or something, but the first approximation, let’s just say is, cutting a straight line, right? Maybe you don’t want it to be negative or something.

Now, so, this is maybe the simplest possible neural network, one input, one output with a single neuron. So what is a neural network? Well, it’s just to take a bunch of these things, where you take a bunch of these things, and stringing them together. So instead of predicting the price of house just based on the size, maybe you think that the price of a house actually depends on several things, which is, first, there’s the size, and then there’s the number of bedrooms. And depending on the square footage and the number of bedrooms, this tells you what family size this can comfortably support, right. Can this support a family of two, a family of four, a family of six, whatever, right, and then, well, what else?

Based on the zip codes of the house, as well as the average wealth of the neighborhood, maybe this tells you about the school-to-school quality, right. So, with two little neurons, one that tells us a family size, a house can support; one that tells us his group quality and maybe the zip code also tells us, how walkable is this, right? And maybe if I am buying a house maybe ultimately I care about my family size and support, is this a walkable region, what’s the school quality.

So let’s say these things and string them into another neuron, another linear function or something like it that then outputs the price, okay? So this is a neural network, and one of the magics of a neural network is that, I gave this example, as if when we’re building this neural network, we have to figure out that family size, walkability and school quality are the three most important things that determine the price of a house, right. As I drew this neural network, I talked about those three concepts.

Part of the magic of the neural network is that when you are training one of these things you don’t need to figure out what are the important factors, all you need to do is give it the input A and the response B and it figures out by itself what all of these intermediate things that really matter for predicting the price of a house.

And part of the magic is when you have a ton of data, when you have enough data, A and B, it can figure out an awful lot of things by itself, all right? I’ve taught machine learning for a long time, I was a full-time faculty at Stanford for over a decade, now I’m still adjunct faculty in the CS department. But whenever I teach people the mathematical details of a neural network, often I get from the students like almost a slight sense of disappointment. Like is it really this simple, you gotta be fooling me, but then you implement it and it actually works when you feed it a lot of data. Because lot of complexity, lot of the smarts of the neural network comes from us giving it tons of data. Maybe tens of thousands or hundreds of thousands or more of houses and their prices, and only a little bit of it comes from the software, so the software, well known trivia. Software is really not that easy, right. The software is only a piece of what the neural network kind of knows. The data is a vastly, larger source of information for the smarts of the neural network, than the software that we have to write.

So, and let’s see, yeah. One of the implications of this is, when you think about building businesses, we think about building products or businesses, what is the scarce weasels, right? If you want to build a defensible business that deeply incorporates AI, what are the moats? Or how do you build a defensible business in AI?

Today, we’re fortunate that the AI community, the AI research community is quite open. So almost all, maybe all of the leading groups, tend to publish our results quite freely and openly. And if you read our papers at Baidu, we don’t hold anything back. If you read our state of the art speech recognition paper, our state of the art face recognition paper, we really try to share all the details. And we’re not trying to hide any details, right. And many leading, researchers in AI do that, so it’s difficult to keep algorithms secret anyway.

So how do you build a defensible business using AI? I think today, there are two scarce resources. One is data, it’s actually very difficult to acquire huge amounts of data, right, A come a B. Maybe to give you an example, one of the projects, well a couple examples, speech recognition, I mentioned just now we’ve been training on 50,000 hours of data. This year, we expect to train about 100,000 hours of data. That’s over 10 years of audio data, right? So literally, if I pull my laptop and start playing audio to you to go through all the data our system listens to, we’ll still be here listening until the year 2027, I guess, right? So this is massive amounts of data that is very expensive to obtain.

Or take face recognition. We’ve done lot of work on face recognition. So to say some numbers, the most popular academic computer vision benchmark slash competition has researchers work on about 1 million images, right, and the very largest academic papers in computer vision published papers on maybe 15 million images, right, of the kind of recognizing objects from pictures or whatever. At Baidu, to train our really leading edge, possibly best in the world, but I can’t prove that, definitely very, very good face recognition system, we trained it on 200 million images, right, so this scale of data is very difficult to obtain.

And I would say that, honestly, if I were leading a small team of five or ten people, I would have no idea, frankly, how to replicate this scale of data and build a system like we’re able to in a large company like Baidu, with access to just massive scale data sets. And in fact, at large companies, sometimes we’ll launch products, not for the revenue, but for the data, right? We actually do that quite often.

Often I get asked, can you give me a few examples, and the answer, unfortunately, is usually no, actually. But I frequently launch products where my motivation is not revenue but is actually data, and we monetize the data from a different product. So I would say that today in the world of AI, the two scarcest resources are, I would say the most scarce resource today is actually talent because AI needs to be customized for your business context. You can’t just download an open source package and apply it to your problem. You need to figure out where does the spam filter fit in your business or where does speech recognition fit in your business and what context, where can you fit in this AI machine learning thing?

And so this is why there is a talent war for AI because every company, to exploit your data, you need that AI talent that can come in to customize the AI, figure out what is A and what is B, where to get the data, how to tune the algorithm to work for your business context. I’d say maybe that’s a scarce resource today. And then second is data is proving to be a defensible barrier for a lot of AI-powered businesses.

So there’s this concept of a virtuous circle of AI that we see in a lot of products as well. Which is, you might build a product, right? For example, we built a speech recognition system to enable a voice search, right, which we did at Baidu. Because the US search companies have done that, too, some of the US, anyway. The speech recognition system, whatever, some product, because it’s a great product, we get a lot of users, right? The users using the product naturally generates data, right, and then the data through ML feeds into our product to make the product even better. And so this becomes a positive feedback. That often means that the biggest and the most successful products, the most successful product often — the best product often has the most users. Having the most users usually means you get the most data, and with modern ML, having the most data sometimes, usually, often means you can do the best AI, best machine learning and therefore have an even better product, and this results in a positive feedback loop into your product.

And so when we launch new products, we often explicitly plan out how to drive this cycle as well. And I’m seeing pretty sophisticated strategies in terms of deciding how to roll out the product, sometimes by geography, sometimes market segment, in order to drive this cycle, in order to drive the cycle, right?

Now this concept wasn’t around for a long time, but there is really a much stronger positive feedback loop just recently, because of the following reasons, is traditional AI algorithms work like that, so there was kind of beyond a certain point, you didn’t need more data, right? This is data performance. So I feel like ten years ago data was valuable, but it created less of a defensive barrier because beyond a certain threshold, the data, it just didn’t really matter. But now the AI works like that, the data is becoming even more important for creating defensible barriers for AI kind of businesses.

Let’s see, all right. Then several of you asked me about, actually Robbie was kind enough to take the audience questions from the sign-up form and summarize them into major categories. So he summarized the questions into your major heading categories, right? So one of them was AI society impact. One was your practical questions for AI. One of the headings that Robbie wrote was scared as in, will AI take over the human race or kill humans or whatever?

So I feel like there is — so this is a virtuous circle of AI. There is a, I’m not sure what to call it, I’m going to call it the non-virtuous circle of hype. When preparing for this talk, I actually went to a thesaurus to look up antonyms, opposites, of the word virtuous, and vile came up. But I thought, vile circle of hype was a bit too provocative, I don’t know. But I feel like that we are, unfortunately, there is this evil AI hype: will AI take over the world, slaughter humans, whatever. Unfortunately, some of that evil AI hype, right, fears of AI, is driving funding, because what if AI could wipe out the human race? Then sometimes wealthy individuals, or sometimes government organizations or whatever they now think, well, let’s fund some research, and the funding goes to anti-evil AI. And the results of this work drives more hype, right, and I think this is actually a very unhealthy cycle that a small part of AI communities are getting into.

And I’ll be honest unfortunately, I see a small group of people, it’s a small group, with a clear financial incentive to drive the hype, because the hype drives funding to them. So I’m actually very unhappy about this hype. And I’m unhappy about it for a couple of reasons. First, I think that there is no clear path to how AI can become sentient, right? Part of me, I hope that there will be a technological breakthrough that enables AI to become sentient, but I just don’t see it happening. It might be that that breakthrough might happen in decades. It might happen in hundreds of years. Maybe it’ll happen thousands of years from now; I really don’t know.

The timing of technology breakthroughs is very hard to predict. I once made this analogy that worrying about evil AI killer robots today is a little bit like worrying about overpopulation on the planet Mars, right? And I do hope that someday we’ll colonize Mars and maybe someday Mars will be overpopulated. And some will ask me: Andrew there are all these people, all these young, innocent children dying of pollution on Mars, how can you not care about them? And my answer is We haven’t landed on the planet yet, so I don’t know how to work productively on that problem.

So, maybe the dilemma is, if you ask me, do I support doing research on X, right? Do I support research on almost any subjects? I usually want to say yes, of course. Doing research on anti evil AI is a positive thing. But I do see that there’s a massive misallocation of resources. I think if there were two people in United States, maybe ten people in United States where I can go and anti evil AI is fine, that ten people working on over population of Mars is actually fine, form a committee, write some papers.

But I do think that there is much too much investment in this right now, right? So yeah, so sleep easy. And maybe the other thing, quite a lot of you asked about the societal impact, what I found is varying. The other thing I worry about is this evil AI hype being used to whitewash a much more serious issue, which is job displacement, right?

So frankly, I know a lot of leaders in machine learning, right? And I talk to them about their project. And there’s so many jobs that are squarely in the cross hairs of my friends’ projects, and the people doing those jobs, frankly, they just don’t know, right? And so, in Silicon Valley, we’re being responsible for creating tremendous wealth, but part of me feels like we need to be responsible as well for owning up to the problems we cause and I think job displacement is the next big one, thank you.

Thank you. And I’m going to say just a little bit more about that at the end. And then we shouldn’t whitewash this issue by pretending that there’s some other futuristic fear, to fearmonger about and try to solve that by ignoring the real problem. Let’s see.

So the last thing I want to talk about is, AI product management. So AI is evolving rapidly, it’s super exciting, there are just opportunities left and right, but I want to share with you some of the challenges I see as well, right? Already some of the things we’re working on that I end up bleeding as well, I feel like our own thinking is not yet mature. But that you run into if you try to incorporate AI into your business.

So AI Product Management. So maybe many of you know what a PM is, but let me just draw for you a Venn diagram. That’s my simple model of how PMs and engineers should work together, right? So let’s say this is the set of all things that users will love, right. So the set of all possible things, all the possible products that users will love. And this is a set of all things that are feasible, right, meaning that today’s technology or technology now or the near future enables us to build this, right?

So for example I would love a teleportation device, but I don’t think that’s technological feasible, so teleportation device will be here, but we’ll all love one, but I don’t think it’s feasible. There are a lot of things that are feasible but then no one wants it. But we build of those in Silicon Valley as well. And I think the secret is to try to find something in the middle, right?

And so, roughly, I think the PMs job as figuring out what is this set on the left ánd research and engineering’s job as figuring out what’s in this right side, and then the two kind of work together to build something that’s actually in the intersection, right?

Now, one of the challenges is that AI is such a new thing that the workflows and processes that we’re used to in tech companies, they’re not quite working for AI tools. So, maybe for example, in Silicon Valley we have pretty well established processes, there are product managers and engineers to do their work. For example, for a lot of apps the product manager will draw a wire frame, right? Where, so for example, actually for the Baidu search app, right? The PM might decide well put a logo there, put a Search bar there, put a microphone there, put a camera there, and then put a news feed here, and then actually, well we actually moved our microphone button down here and we’ll have a social button, this button, this button.

So a product manager would draw this on a piece of paper or on the cat thing, and an engineer would look at this drawing that the product manager drew, and they would write a piece of software and this is actually a rough for the Baidu search app, yeah? The search bar in terms of news here, right? Little bit like a — it combines the search as well as the social newsfeed. Not very social, a newsfeed, both in one.

But, so this works for if you pull open your app or you build a lot of apps like a news app or a social feeds app or whatever, this type of working together works with established process of doing this. But how about an AI app? You can’t wire frame a self-driving car that runs by wire frame from a self driving car or if you want to build a speech recognition system, the PM draws this button, but I don’t know how good, how accurate, is my speech recognition system need to be.

So while the processes are not — so what if this wire frame was a way for the PM and the engineer to communicate, we are in still frankly trying to figure out what are good ways for a PM and an engineer to communicate a shared vision of what a product should be. Does that make sense? So PM does a lot of work, goes out, figures out what’s important to users and they have in their head some idea what this product should be.

But how do they communicate that to the engineer? All right. And so, as a complete example of that, let’s say that you’re trying to build a speech recognition system, I did a lot of work on speech recognition, right. So my team and I, did a lot of work on speech recognition, so I thought about that a lot.

If you’re trying to build a speech recognition system, say to enable voice search there are a lot of ways to improve the speech recognition system. Maybe you want it to work better even in noisy environments, right? But a noisy environment, it could mean car environments, or it could mean a cafe environment, people talking versus a car noise, a highway noise.

Or maybe you really need it to work on low bandwidth audio, right? Maybe sometimes users are just in a bad cell phone coverage setting, so you need it to work better on low bandwidth audio. Or maybe you need it to work better on accented speech, right? I guess US has a lot of accents. China also has a lot of accents. What does accented speech mean? Does it mean a European accent, or Asian accent? European does it mean British, or Scottish? You know what does accented really mean? or maybe you really care about something else, right?

So, one of the practices we’ve come up with, is that one of the good ways for a PM to communicate with an engineer, is through data, and what I mean is for many of my projects we ask the PM to be responsible for coming up with a data set. For example, give me, let me say 10,000 audio clips that really shows me what you really care about, right? And so, if the PM, comes up with a thousand or ten thousand examples, of a people of recordings of speech, and give us data, to the engineer, and just the engineer has a clear target to aim for.

So, found that having a PM responsible for collecting really a test set is one of the most effective processes for letting the PM specify what they really care about, and so if all 10,000 audio clips have a lot of car noise, this is a clear way to communicate to the engineer that you really care about car noise. If it’s a mix of these different things, then it communicates to an engineer how exactly, what mix of these different phenomena the PM wants you to optimize for, right?

I have to say, this is one of those things that’s obvious in hindsight, but that surprisingly few AI teams do this. One of the bad practices I’ve seen is when the PM gives an engineer 10,000 audio clips, but they actually care about a totally different 10,000 ones. That happens surprisingly often in multiple companies, right?

And then I feel like we’re still in the process of advancing the bleeding edge of these workflow processes, for how to think about new products. So, here’s another example. We have done a lot of work on conversational agents, right? So, the conversational agent I might say to the AI, please order takeout for me? and then the AI says well what restaurant do you want to order from? And you’d say I feel like a hamburger. So you’d go back and forth like a conversation or a chat bot to help you order food or whatever.

So again if you were to draw a wire frame, the wire frame would be while you say this, the chat box says this, you say this chat box says this, but this is not a good spec for the AI right? The wireframe is the easy part, the visual design, you can do that, but how intelligent is this really supposed to be? So the process that we developed at Baidu is we asked the PM and the engineer to sit down together and write out 50 conversations that the chat box is meant to have with you, right?

So for example, if you sit down and you write the following, let’s say the user, U for user, says, please book a restaurant, right, for my anniversary next Monday. I’m abbreviating this just to write faster. Please book a restaurant for my anniversary. If the PM then says, well in this case, I want the AI to say, okay, and do you want flowers? Right? Do you want me to order flowers? What we found is that this then creates a conversation between the PM and the engineer where the engineer asks the PM, wait, do you want me to suggest an appropriate gift for all circumstances and all possible, for Christmas I would suggest some other I don’t know what to buy for Christmas, I guess, or is it only for anniversaries you want to buy flowers? and I don’t have just buy any other gift and not for anything other than anniversaries, right?

But we found that the process of writing out 50 conversations between counselor agents and engineer PMs sitting down together and work through these conversations, that those are good process to enable the PM to specify what they think is the set on the left, what the user will love and for the engineer and to tell the PM what the engineer thinks is feasible given today’s chat box technology, right?

And so this is actually a process that we’re using in multiple products. So I think that AI technology is advancing rapidly and there’s so many shiny things in AI. The things you see the most in PR are often the shiniest technology but the shiniest technology is often not the most useful, right? But I think that we’re still missing a lot of the downstream parts of the value chain of how to take the shiny AI technology that we find out in research papers and how to think about, how do the product or business, and, we’re definitely, it definitely feels you know, software engineering today has established processes like code review and you know agile development. Some of you know what those are, right? But these was established processes for writing code. I think we’re still in the early phases of trying to figure out how on earth to organize the work of AI and the work of AI product. And this is actually a very exciting time to enter this field.

Let’s see. All right. I want this time for questions so, all right, real quick. I want to share with you some specific examples of short term opportunities for AI. These are things that are coming in the very near future. Let’s see I think I mentioned, well, I mentioned Fintech, I’m going to talk about that, in the near term future, I think speech recognition will take off, it’s just in the last year or two that speech recognition reached the level of accuracy, was becoming incredibly useful.

So about four, five months ago, there was a Stanford University led study done by James Landay, led by James Landay, who is a professor of Computer Science, together with us at Baidu and the University of Washington, and showed that speech input on a cellphone is 3x faster, using speech recognition than typing on the cell phone, right? So, speech recognition has passed the accuracy threshold where you actually are much faster and much more efficient using speech recognition than typing on the cell phone keyboard, and that’s true for English and Chinese.

But I think, and at Baidu over the past year, we saw 100% year on year growth on the user speech recognition across all of our properties. So I think we’re beyond the knee of the curve where speech recognition will take off rapidly, and so, I guess in the US, there are multiple companies doing small speakers. Baidu has a different vision, but, I think that the device that you can come on with your voice in your home also take off rapidly, so whenever an operating system that would release, the hardware makers enable that.

What else? Computer vision is coming little bit later. You know, I see some things take off faster in China than in the US, so because all of us living in the US are familiar US ones, I am like, mean to a little bit even sharing things I see from China. One thing that’s taking off very rapidly is Face Recognition, so I think because China is a mobile first society right?, and all of us, most of us in US first on the laptop or a desktop, then we got our smartphone. Lot of people in China really just have a smartphone or first get a smartphone, then a laptop or a desktop Or laptop I guess, I’m not sure who buys this house anyone.

But because of that in China a lot of people, let’s see, you can apply for an educational loan on your cellphone in China. And just based on buttons, just based on using your cellphone, we will send you a lot of money, right, for your education. So because of these very material, financial transactions are happening over your cellphone, before we send you a lot of money we would really like to verify that you are who you say you are, right, before we send it to someone that claims to be you but isn’t you.

So, this in turn has driven a lot of pressure for progress in face recognition, and so face recognition on mobile devices as a means of biometric identity verification is taking off in China. And then we’ve also done things like, today in Baidu headquarters, instead of, do I have it, no, I don’t right, instead of having a swipe an RFID card to get inside the office building, today at Baidu’s headquarters, I can just walk up and there’s a face recognition just to recognize my face, and I just walk right through. Just yesterday or the day before, I posted a video on my personal YouTube channel demoing this. You can look that up later if you want.

But we now have face recognition systems that are good enough that we trust it with pretty security protocol applications, right, if you look just like me, you can actually get inside my office at Baidu. So we really trust our face recognition system, so it’s pretty easy.

So let’s see, and I think both of these have been obvious to us for some time, so our capital investments and data investments have been massive. These are well beyond the point where a small group could be competitive with us unless there’s some unexpected technological breakthrough. I’ll mention some things a little further out.

I’m personally very bullish about the impact of AI on healthcare. I am spending quite a bit of time on this myself. And I think, well, the obvious one that a lot of people talk about is medical imaging. I do find it challenging. Yeah, I do think that a lot of radiologists that are graduating today, will be impacted by AI, definitely, sometime in the course of their careers. If you’re planning for a 40-year career in radiology, I would say that’s not a good plan.

But beyond radiology, I think there are many other verticals, some of which we’re working on, but there’s a huge opportunity there. And anyway, and on and on and on, right?, and I think Fintech is there. I hope education will get there, but I think education has other things to solve before reading this huge impact by AI, but I really think that AI will be an incredibly impactful in many different verticals.

So let’s see. And what I talked about today was kind of AI technology today, right, so really supervised learning, and I will say that the transformation of all of these industries, there’s already a relatively clear roadmap for how to transform multiple industries using just supervised learning. There are researchers working on even other forms of AI, you might hear one say unsupervised learning or reinforcement learning or transfer learning, there are other forms of AI as well that maybe don’t need as much data or maybe has other advantages. Most of those are in the research phase, most of them are used in very relatively small ways, they’re not what’s driving economic value today, but many of us hope that there will be a breakthrough in these other areas and if that comes to pass, then that will unlock additional ways of value.

So, let’s see, the few that AI has had several winters before, right? I think the some overhype went down – some overhype went down. So we think they were maybe two winters of AI, right, but many disciplines undergo a few winters, winter and then eternal spring, and I actually think that AI has passed into the phase of eternal spring.

I think one of the questions someone asked was, when will AI no longer be the top technology or something, right, and I feel like if you look at silicon technology, right? I think we’re in the eternal spring of silicon technology, or maybe some other metal, some other material will surpass it, but the concept of a transistor in computational circuits, that seems like it’s going to be with the human race for a long time. And I think we have reached that point for AI where AI neural networks, deep learning, I think it will be with us for a long time. Can’t predict conscious of yourself, but it could be a very long time, because it’s creating so much value already and because there is this clear roadmap for transforming several industries even with the ideas we have, but hopefully there will be even more breakthroughs and even more of these technologies.

All right, very last topic, you know the jobs issue. I think that’s, to the extent that we’re causing these problems, we should, the job displacement issue, I think we should own up to it. Just as AI displaces jobs, similar to the earlier ways of job displacement, I think that AI will create new jobs as well, maybe even ones we can’t imagine. So that’s why I’ve actually seen development course for a long time. I think one of the biggest challenges of education is motivation, right? And it’s really good for you to take these courses and study, but it’s actually really difficult for an individual to find the time, and the space, and the energy to do the learning that gives them these long term benefits.

So after the automation replaced a lot of agriculture the United States built its current educational system, your K-12 and university. It was a lot of work to build the world’s current educational system. With AI displacing a lot of jobs I’m confident that there will be new jobs but I think also we need a new educational system to help people whose jobs are displaced, reskill themselves to take on the new jobs.

So one of the things that some governments, well, one of the things I think we should move toward is a model of basic income but not universal basic income where, you’re paid to quote do nothing, but I think government should give people a safety net, but pay the unemployed to study, right, to provide the structure to help the unemployed to study so as to increase the odds of gaining the skills needed to re-enter the workforce and contribute back to the tax base that is paying for all this basic income.

So I think we need a new, new deal in order to evolve society towards this new world where there are new jobs, but job displacements are also happening faster than before. We’ll have to see more about that.

Finally, really, final, final thing, I know that over in the GSB, many of you have fantastic product business, or social change ideas, one of the things I hope to do is try to connect, frankly connect GSB and CS. I think GSB and CS are really complementary sense of expertise, but for various complicated reasons we didn’t get into, the two communities don’t seem very connected.

Yeah, I’m in the process of organizing some events that I hope will bring together some CS, some GSB, maybe also some VC, some capital investments to those of you interested in exploring new opportunities that AI creates. So if you want to be informed of that, sign up for this mailing list at bit.ly/gsb-ai. There are some things being organized. They’re already underway, but actually instead of taking a picture of this, if you just go and sign up for this on your cellphone, right now. You can do this while I’m taking questions.

And some of these things are already underway, but when they’re ready to be announced, I’ll announce it to the mailing list there, so that you can come in and be connected to some of these other pieces at the campus.

So with that, I’m happy to take questions, but let me say thank you all very much.

Question-and-answer session

Introducing speaker: Thanks so much, Andrew. It’s a great talk, and a lot of us, I know, want to be engaged in product development and product management in the field of AI. And you’ve given us a lot of good frameworks to think about these conversations. And the mailing list is right there, in case you wanted to note down.

So Andrew has gracefully accepted to fill some questions until about 5:30. So if you have any questions there are going to be some Sloan fellows that are going to be moving around the room, so please attract their attention. But I can kick off with a question; I really wanted to ask this question, because it reminded me of my TSBSE, which is what scares you about AI and why? But I guess you already answered part of that, so maybe you can touch on that. And another question, which I felt was interesting was, what is the role of known technical leaders in development of AI? Who’s in charge of the ethical decisions being made in directing AI?

Andrew Ng: All right, what scares about AI is definitely the job displacement. I think that, honestly, part of me, I am really honest with you guys, right? Part of me wonders with the recent presidential election, part of me really wonders if many of us in Silicon Valley, have we really failed a large faction of America, and we’re being really honest. I’m not saying I agree with everything happening with politics right now, but part of me actually wonders if we create a tremendous wealth, but also frankly, if we left a lot of people behind. And I think it is past time for us to own up to it, and also take responsibility of addressing that.

Let’s see, what was the other question? It was about – ethical. And I think in terms of ethical issues, there are some things, but I think that — I think jobs are so important, I’m just tempted not to talk about anything else. But I think that AI is really powerful, and can do all sorts of things. And we see lots of — I think there are some small issues such as, is AI sometimes biased, right? For example, if you do a web search, right? We want to make sure that if you search for a certain ethnic group, you don’t get lots of results that says well, this is, check out their criminal record or something like that, right? We don’t want AI to exhibit bias. Or that AI thinks you’re male versus female, we don’t want to show you very different types of information that they confirm is gender stereotypes. So I think there’s some cultural bias issues.

I think that openness, AI community is very open today. I think we must fight to make sure to keep it open. I think the number one by far is actually jobs.

Maybe take some questions? How does the microphone work?

Audience: Hi, Catherine Shen here, I’m MSX 16, graduated last year. And thanks for the talk. I had a question around, you mentioned the defensibility of AI as the three things, so access the data, talent scarcity and positive feedback loop. And one in three, so accessing data and positive feedback loop seems to really benefit large companies or companies that already have the AI technology. And so I’m wondering at what point is it going to be really tough for startups to, well, become a AI startup. And secondly, for investors, at what kind of scale do those investments need to make for a startup to be successful?

Andrew Ng: Sure, yeah, and just to clarify, I think the scarce resources are data and talent. And then a positive feedback loop is a strategy or a tactic to drive the data, right? So I think that for the problems I talked about, like speech recognition, face recognition is going to be really difficult for a small company to acquire enough data to tailor or whatever the computer effectively. Unless there’s an unexpected technological breakthrough that small groups do stuff that can’t be done with today’s technology.

But I think there’s lots of small verticals. So for example, take medical imaging. There are some medical diseases where there are so few cases around the world that if you have 1,000 images or something, that might be almost all the data that exist in the world. So that’s one. There are just some verticals that there isn’t that much data.

But I think the other thing it that there are so many opportunities in AI today. Honestly, my teams regularly write full fledged business plans, do the market research, size of the market, figure out the economics and all of that is good. With a full fledged business platform and a new vertical, and we decide let’s not do it. Because we just don’t have enough talent to go after all the big options. So we decide, let’s not do it, because there’s something even bigger we want to do, right?

So I think today, we’re fortunate to have so many opportunities that there are plenty of opportunities that the large companies are, frankly, not pursuing because today’s world has more opportunities than talented AI researchers.

Audience: Hi, Andrew, what do you think of the use of AI in the creation, sorry, over here — in the creation of inventions? So it’s something that’s usually the reserve of what’s, I don’t know, the human mind, the use of AI to create inventions, even patentable inventions?

Andrew Ng: Yeah, I am seeing very early phases. You know, creativity is a very funny thing, right? So can AI compose music, it’s so subjective. I feel like even with a 20 year old technology, automated music composition by computers, a lot of us thought that the automatic composition sounded horrible. But there were some people that loved it, like the 20 year old technology. So I don’t know. We’re seeing a lot of cool work with AI doing special effects on images, synthesizing, make this picture, if it was painted by a certain painter. I don’t know, it feels like a small, but very interesting area right now.

But making complex inventions, like inventing a totally new, very complicated system with many pieces. I think that’s beyond what I will see a clear path to today.

Audience: [Inaudible Questions]

Andrew Ng: Sure, so scalability drives a lot of problems in AI. But if Moore’s Law is coming to an end, how does that affect the scalability of AI? It turns out that, let’s see, so I think that as I’ve seen the roadmaps of multiple high performance computing hardware type companies. And whereas, Moore’s law for single processor doesn’t seem to be working very well anymore. I have seen specific and I think credible roadmaps of microprocessing companies that show that for the types of computations we need for deeper, for neural networks, I am confident that it will keep on scaling for the next several years. And so this is same day processing, single instruction multiple data. It turns out it’s much easier to paralyze than a lot of the workload. Your word processor’s actually much harder to paralyze and your network is actually much easier to paralyze. So I feel there is still a lot of headroom for faster computation.

I will say that when I look across a mix of problems, many of the problems, AI problems, are bottlenecked by data. But many of the problems are also just bottlenecked by computational speed. There are some problems where our ability to acquire data exceeds our ability to process that data inexpensively. So further progresses in HPC, which I think there is a roadmap for, should open up more of that value.

Audience: Hello. Hi, Andrew, my name is Erica Lee. I’m a startup founder working machine learning. So two questions, you mentioned that algorithms aren’t like the special sauce to being successful in AI. What do you recommend for people, though, building and working on AI about IP protection or best ways to get around that to still build a valuable product?

And then two, you mentioned the relationship between the PM and an engineer about the cycle of data and how to communicate. That’s for building a product, though. What about people doing some R&D research on reinforcement on supervised learning? Is there a certain lifecycle of strategy would go for research breakthroughs or to improve the research processes?

Andrew Ng: Yeah, maybe, sure, right. Boy, all right, so I think, yeah, IP protection is one of those things that we give advice on and I get in trouble with lawyers or something. Honestly, I don’t have a strong opinion. I see a lot of companies file for some patents, but how much you can rely on them for defensibility is an open question, check with your lawyer. I actually don’t have a strong opinion on that. We do tend to think strategically about data as a defensible barrier, though, we rely on data.

In terms of, you said processes for R&D, right? The research academic committee tends to favor novelty, anything novel and shiny, you can get a paper published. I would say that, maybe if you want to train up a team of engineers, I’ve supervised PhD students at Stanford for a long time. I feel like if you want to be a deep learning researcher, and if you go to published papers, the formula I usually give people is this: read a lot of papers. Go beyond reading papers but go and replicate existing research papers yourself. This is one thing that is underappreciated, actually. Even pull back a little bit from trying too hard to invent a new thing. I spend a lot of time replicating published results. I found that to be very good training process for new researchers.

And then the human brain is a marvelous thing. It works every time, I’ve never seen it fail. But if you read enough papers and really study them and understand them and replicate enough results, pretty soon you have your own ideas for pushing forward the state of the art. I’ve mentored enough PhD students to ascertain with high confidence that this is a very reliable process. And then go submit your paper and get it published.

Audience: Thank you. So I’m a mechanical engineering student aspiring to be a roboticist when I graduate. I was wondering what are the best opportunities for mechanical engineers to go into as it relates to AI and robotics? Would you know that?

Andrew Ng: Yeah, so I’ve seen a lot of ME people take up very successful careers in AI. Actually some of my PhD students, actually one of my PhD students was an ME PhD student, and he transferred to the CS Department and he did very well. So I think that robotics has many opportunities in specific. Well, you’re a Stanford student, right? Cool I would say, take some CS-AI classes and try to work with the AI faculty. I do think that there are a lot of opportunities to build interesting robots in specific verticals. So I think precision agriculture is a very interesting vertical. Right, so there are now multiple startups using AI.

Actually, for example, some of my friends are running Blue River, which is using computer vision to look at specific plants, specifically, heads of cabbages, and kill off — have an AI decide which heads of cabbage to kill and which to let live so as to maximize crop yield, right? So there’s one application where AI is letting you make, well, this is life and death decisions, but this is life and death of heads of cabbage, not of humans. But it is letting you make one at a time life and death decisions by heads of cabbage. But I think that precision agriculture is one vertical. I don’t know, yeah, I think actually it’s interesting work on surgical robotics as well, but that has a bigger kind of FDA process approval. So that’s a longer cycle. But I’m seeing less of a, actually one of the things taking off in China, the love of companionship robots, more social companionship robots that are being built in southern China. It’s not really taken off in the US yet, but there are surprisingly many of these things in China.

Audience: Hi, I’m Phil. I’m cofounder of Eurobaby, it’s a Palo Alto based startup that helps parents to understand the developmental needs of their child and pair with baby products. I’d love to hear your take on pairing AI with humans. If you think it’s usually for most applications the faster way to focus on an AI only approach right away or actually have a hybrid solution of AI and humans. It’s, for example, in self-driving cars or chatbots and some –

Andrew Ng: Yeah, I don’t have a general rule for that. It’s so case by case, I guess. A lot of speech recognition work is about making humans more efficient in terms of how you communicate with or through a cell phone, for example. And then for self-driving cars, we know that if a car is driving and it wants you to take over, you need maybe 10, 15, maybe even longer seconds to take over. So it’s incredibly difficult to bench the attention from the distracted human back to take over a car. So that’s why I think full autonomy — level four autonomy will be safer than trying to have a human take over at a moment’s notice when the car doesn’t know what to do. So that might be one case where this mix between full and partial automation is challenging from a user interface point of view. So I don’t have a general rule for that.

Audience: Okay, so when you talk about opportunities for AI, you mentioned the online education. I just wanted to know more about this. You mentioned that the motivation problem is one of the problems for online education. But do you think this is the biggest challenge that online education is facing that AI could probably solve? Or do you think there are some other challenges for online education? On motivation, I mean, people don’t want to spend enough time to finish the whole course.

Andrew Ng: Yeah, so I think that’s actually — AI is helping education and people talked about personalized tutors for a long time. And today Coursera uses AI to give you customized course recommendations and there’s AI for auto-grading. So I would say it’s definitely helping at the margins, but I would say that education still has a big digital transformation to go through, maybe even without that much involvement of AI. Maybe one pattern that is true for a lot of industries is first comes the data and then comes the AI.

So healthcare needs this pattern. Over the past year, thanks to, well, partially, Obamacare, right, there’s a huge movement in the United States, a movement in other countries, too, towards electronic health records EHR, so the rise of EHR and the fact that your X-ray scanners all went from film to digital x-rays, so that wave of digitization has now created a lot of data that AI can eat to create more value. I would say that a lot of education still feels like it’s first undergoing the digital transformation. And while AI can certainly help, I think there’s still a lot of work to do for just a digital transformation.

Audience: Yeah, if we could talk a little bit about how Baidu is using AI for managing your own cloud data centers, primarily IT operations management use cases.

Andrew Ng: Sure, so I guess, boy, let’s see, I’ll give one example. We talked about this. Several years ago, almost three years ago, we did a project showing that we can detect hardware failures, especially hardware failures a day ahead of time using AI. And so this allows us to do preemptive maintenance, a hot swap of hard disks. Copying the data off even before it fails, thus reducing constant easing reliability. We’ve also been working to reduce power consumption of the data centers, something low balancing uses AI. I can’t point to one big thing, but I feel like in many places, AI has had an impact on optimizing various aspects of data center performance.

Introducing speaker: We’d like to stay for longer but we have to leave the room for the next event, so it will probably be the last question.

Audience: Hey, how are you here? I actually studied at both CS and the GSB before. So my question is you actually mentioned that that’s the sweet spot for AI progress, if human can process for less than a second, then that would be a good problem set for AI to solve. Can you comment on the other way, on the other side of spectrum? In your experience, a problem would take a lot more seconds or a long time for humans to process, yet after careful modeling or careful planning, you are able to solve the problem by AI. Can you give some examples on that?

Andrew Ng: Yeah, so there are things that AI can do that humans can’t do in less than a second. So for example, I think Amazon today does a way better job recommending books to me than even my wife does, right? And the reason is Amazon has a much more intimate knowledge of what books I browse and what books I read than even my wife does.

Advertising, honestly, leading Internet companies have seen so much data about what ads people click on and don’t click on, could remarkably be good at that task. So there are some problems where a machine can consume way more data than any human can and model the patterns and predictions. So this is something that AI surpasses human performance because it consumes so much data, right, like Amazon knowing my book preferences better than my wife.

Let me finish. And then the other thing of tasks that take a human more than one second to do, a lot of the work of designing AI into the workflow is piecing many small AI pieces together into a much bigger system. So for example, to build a self-driving car, we use AI to look at a camera image, radar, LIDAR, whatever, the sensor data. Let me just say, a picture of another car and supervised learning estimates the position of the other car. Supervised learning, estimates the position of the pedestrians.

But these are just two small pieces, well, two important pieces, of the overall AI. Then there’s a separate piece that tries to estimate, well, where is this car going to be in five seconds? Where’s this pedestrian going? There’s another piece that plans, well, given that all of these objects are moving in this way, how do I plan my car so that I don’t hit anything? And then after that, there’s then how do I turn the steering wheel? Do I turn the steering wheel five degrees or seven degrees to follow this path? So often a complicated AI system has many small pieces, involve the ingenuity is figuring out where to take this superpower, supervised learning, and put it into this much bigger system that creates something very valuable.

Audience: I’m Mahidhar, I’m a solutions architect in a company called OTP. My question was, you mentioned about jobs and wealth distribution as well. Since it’s a management forum, I wanted to ask, what sort of role do you see for product managers when interacting with for example sociologists or legal profession, based on examples, if you are building a car, self driving car, if there’s a collision which is about to happen, where the developer or the AI has to take into consideration the person driving the car, or the pedestrian who it’s about to hit. That’s a legal question. So but there’ll be a lot of questions like these. What do you see the role of management interacting with different function areas?

Andrew Ng: Yeah, so the most famous example of a variation of what you said is then called the trolley problem, is a philosophy that cause ethical dilemma, where I guess I think your car is — the classical version, you have a trolley running on rails. And the trolley is about to hit and kill five people. And you have the option of yanking on the lever to divert the trolley to kill one person. So the ethical dilemma is do you yank on the lever or not, because if you do nothing, five people die. If you do something, one person dies, but you killed that person. So are you going to kill someone, right, versus not doing anything?

So it turns out that the trolley problem wasn’t important, even for trolleys, right, when we built trolleys and whatever, several hundred years in the history of trolleys, I don’t know that anyone actually had to decide whether or not to yank the lever. It’s just not an important problem outside the philosophy classes. And I think that when the self-driving car teams are not debating this, philosophers are debating this.

Frankly, if you’re ever facing a trolley problem, chances are you made a mistake long ago. Now, when was the last time you faced a trolley problem, right, when driving your car? I expect a self-driving car to face it about as often as you have driving your car, right, which is probably pretty much never. So I think right now the problem with self-driving cars is there’s a big white truck parked across the road. Your options are slam the truck and kill the driver or brake. And we don’t always make the right decision for that. So I would solve that first before solving the trolley problem.

Introducing speaker: That’s, I think, a good point to end this great talk. Thanks a lot.

Related Posts

• Transcript: Sam Altman on AGI, GPT-5, And What’s Next — the OpenAI Podcast Ep. 1
• Transcript of Godfather of AI: I Tried to Warn Them, But We’ve Already Lost Control!
• Transcript of Anton Korinek on What Happens When AI Replaces Every Job?
• Transcript of Sundar Pichai on Future of AI, Antitrust, and Privacy
• AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference (Transcript)