Transcript: Jeff Dean Interview on The 1% Rule for Building in AI with Diana Hu

That was the origin story of the TPU.

JEFF DEAN: Yeah. We were starting to see really good quality results on the deep learning-based speech models we were training, but they were computationally expensive compared to the old speech system — though they halved the error rate. That was the equivalent of 20 years of advances in speech recognition in just a few months of fiddling with the model, scaling it up a bit, and getting better data.

So we started to get worried that if speech worked a lot better, people would use it more. That back-of-the-envelope calculation was really about that — what if people start to use speech recognition more to dictate emails or talk to their phone? We realized we needed some better solution than running on CPUs at the time. So we came up with TPUs, which are very specialized for essentially low-precision dense linear algebra, which is at the heart of nearly all the modern machine learning algorithms we use today.

If you build a specialized chip for low-precision dense linear algebra and it can’t do anything else, that turns out to be really useful for machine learning inference, even though it can’t run Chrome or Word or whatever. That system produced a chip a couple years later that was 30 to 80 times more energy efficient than the CPUs and GPUs of the day, and also much, much lower latency — like 20 to 30x lower.

DIANA HU: Which is incredible, what the foundation that TPU has become today. No way you would’ve predicted that TPU would be so foundational now with transformer architecture, which was invented way later — before you actually invented the TPU.

JEFF DEAN: Yeah, that’s sort of why we built a general-purpose linear algebra system, which is what a TPU really is, because we knew ML algorithms were still evolving and you didn’t want to over-specialize. But you wanted to specialize enough that you got the dramatic performance benefits — very big multiplier units, high-speed memory, high-speed interconnect for later TPUs that brought many, many chips to bear on the same problem efficiently. We’ve continued to scale those up and improve their performance over many, many generations now.

DIANA HU: Incredible napkin math. So what’s a good napkin math that everyone here who wants to be a future founder should run tonight, to potentially build something as consequential as the TPU?

JEFF DEAN: It’s always hard to say. I think about what problems you see in whatever it is you’re thinking about, what bottlenecks you see. Are there very different ways of thinking about solutions to some of those problems that would get you an order of magnitude — or two orders of magnitude — better performance or capability?

Sometimes if you just squint at a problem, and think about it not necessarily anchored on exactly how that problem is solved today, but how you would solve it from first principles, you can come up with really good ideas that maybe aren’t what other people are thinking about.

An AI Edition of the Latency Numbers Every Engineer Should Know

DIANA HU: That’s a good tip. Now, for everyone here who doesn’t know — years ago, Jeff wrote a very famous list called “Latency Numbers Every Engineer Should Know.” These are numbers around things like how long a cache miss takes, a disk seek, a network packet traveling from California to the Netherlands — lots of numbers like this about distributed systems and systems engineering. It’s been taped up and become the Bible for a lot of distributed systems engineers.

JEFF DEAN: Okay. Yeah.

DIANA HU: Now, fast forward — that list is up for an update. Give us the AI edition for 2026.

JEFF DEAN: If you looked at what’s important in AI systems these days, you’d want to know things like the bandwidth between your main memory system on your accelerator and the on-chip memory, and the multiplier unit. You want to know how much energy it takes to do a single multiplier operation, what the interconnect bandwidth is between chips and how many chips you can connect with that bandwidth. And beyond that, what’s the fall-off in network bandwidth when you need to talk to 10,000 chips instead of 500? These are all really important numbers to learn, and they really affect how you think about solving particular kinds of problems.

DIANA HU: One interesting thing I’ve heard you talk about is that nowadays the unit you measure everything in is energy.

JEFF DEAN: Yeah.

DIANA HU: You pointed out that doing a calculation or math costs about one picojoule, but moving the data — doing data I/O — costs 1,000 times that.

JEFF DEAN: Yeah, just bringing it in from HBM on an accelerator into the processor so it can actually compute on it.

DIANA HU: That gap kind of quietly decides what products are possible and how these AI algorithms are built. So what are the kinds of problems that founders keep calling model problems, but are in fact energy or data I/O problems?

JEFF DEAN: The example you raised — a 1,000x difference between moving data and actually computing on it, in terms of energy — is pretty significant. It shapes a lot of aspects of what we do in machine learning. If you didn’t have that 1,000x difference, you wouldn’t have to do batching. But you have to batch many examples, or many tokens, at once in order to amortize that data movement so you don’t pay a 1,000x slowdown, but instead pay a 1,000x-divided-by-batch-size energy cost. For really low latency, batching is not very good. So these kinds of energy considerations behind decisions in computer hardware really affect a lot of decisions we make in building higher-level systems.

DIANA HU: A very concrete example is just how training models is done. There’s this whole concept of batching the datasets and running epochs — people may confuse that as a model problem, but it’s really a systems data I/O problem, right?

JEFF DEAN: Yeah. You have to assemble batches to get better efficiency in your hardware. Ideally you might do batch-size-1 training, but it’s not as good in terms of efficiency. So people use pretty large batches these days.

DIANA HU: Do you think it’s possible — I know you’re well known for taking off on a long weekend and coming up with a brilliant solution — is there such a thing as Geoff Hinton going off and working on it for a couple of weeks and getting batch-size-equals-1 training done?

JEFF DEAN: I’ve been thinking more about inference, actually. I think inference is a pretty interesting problem because you want very low latency. Training, you don’t necessarily need incredibly low latency for. I think there’s a lot of room for specializing hardware more for inference than we do today.

DIANA HU: What are some of those interesting things around inference that you’re thinking a lot about?

JEFF DEAN: Just trying to minimize data movement, trying to think about incredibly low precision operations, and maybe not supporting lots of different kinds of precisions. If you feel like you have a good answer for what kinds of precision you need, maybe just build that into the hardware and not much else.

DIANA HU: Which brings it down to a core analogy I heard from a famous computer scientist — that the whole process of AI is a big compression problem, because in order to have data be fully lossy compressed and then restored, you basically need to understand it.

JEFF DEAN: If you truly understand the data, you should be able to compress it really well.

DIANA HU: And now transformer architecture is basically one of the ways that has turned out to work really well.

JEFF DEAN: Yeah, I would say.

DIANA HU: Working pretty well so far.

JEFF DEAN: Good work by my colleagues.

Beyond the Model: Context Engineering

DIANA HU: Now let’s zoom out a bit. AI progress used to mean just better models — more data, bigger models with more parameters. But increasingly in the last year or so, it’s everything around the model, not just size or amount of data. It’s things like retrieval, tools, memory, agent tools. It might get consolidated into what people call context engineering, right?

JEFF DEAN: Yeah. I think the model is really only one piece of what you’re trying to do, which is build an overall system that can solve really interesting problems. That involves a model that knows how to use various tools, knows how to retrieve relevant information, maybe has a history of other information it has retrieved for past problems, and can put information into the context of the model.

The nice thing about that is the information is really clear to the model, unlike the training data the model was trained on, where it’s all trillions of tokens stirred together into a soup of hundreds of billions or trillions of parameters. The actual context the model sees directly for a particular problem or use case is much clearer.

Then there’s understanding what tools are available, which ones will help the model solve the next phase of the problem, how to decompose the problem into a sequence of tool calls, maybe trying multiple approaches and seeing which ones work. This is the whole orchestration of complex agent and multi-agent systems, which I think is going to be more and more important. Super exciting times, I’d say.

DIANA HU: I think the fun thing about this particular problem domain is that everyone in this room can actually do it, because before, to train a model, you needed an incredible amount of resources — access to GPUs and data. But for context engineering, everyone here could do it. You just need the API to something like Gemini and then work on your own setup for retrieval, tool calls, and so on. So what are some tips for everyone here — how does one get better at, and become exceptional at, context engineering?

JEFF DEAN: I think a really good way to do it is to use these models, and harnesses and tools, to try to solve problems. Sometimes you can actually see where the models are failing. Often you can make the model work better and succeed at that kind of problem — not by adjusting the model parameters, which is hard to do from the outside, but by creating better guidelines for the model, writing skills for the model to know how to use different tools that would be incredibly useful for solving this particular class of problem.

As you do that, you end up on this kind of self-improving loop with the setup you’re trying to use to solve things. That’s a really good way to get better at understanding what additional information the model would want in order to become more capable.

DIANA HU: Can you give an example of some context engineering you personally have done? Skills you wrote, tools that made a huge difference in your workflow?

JEFF DEAN: Sanjay and I were working a few weeks ago — we often do some amount of performance improvement for very low-level libraries. We have a microbenchmark library we’ve written at Google where you can write microbenchmarks of how long different kinds of operations take, or how long it takes to populate a data structure. Sometimes those data structures are used on millions of processes across Google, so it’s actually pretty important to make sure they’re high performance.

So you write microbenchmarks, but without an agent-based system, what you usually do is measure the current performance on benchmarks you care about, make modifications to hopefully improve performance, then rerun the benchmarks and see where things improved, run a broader set of benchmarks, measure the cache footprint of things. We wrote a skill that taught the model how to do most of those things in various sequences, so it could do self-improving benchmark measurement — benchmark, improve code changes, measure the performance improvement, and iterate on that.

That seemed to work pretty well for some kinds of problems. It really is just us giving the approach we would use as people to the model, in a form it could use.

DIANA HU: Wow, that seems very impressive. So you’re saying you have a skill that, if someone got access to it, could perform optimizations like Jeff Dean? Seems like the world would love this — it’s worth an infinite amount of money for someone to have access to this.

JEFF DEAN: We actually published a document a few months ago called “Performance Hints” that Sanjay and I wrote. It’s about a 30-page document about various kinds of performance tricks. Some people have taken that and given it, in summarized form, to various models, and seen that the model can now get better at reasoning about performance issues in code.

DIANA HU: You heard it all here — you could actually optimize your own code like Jeff Dean if you take this paper he published, “Performance Hints.” It’s all freely available, so you should all try it. Very cool.

Now we talked about agents. Everyone here is probably building one or has built one at some point, and I’m sure everyone has seen their agent go off the rails at perhaps step 30 or 40. Agents are great up to about step 10, and then it gets shaky around step 50. What do you think is the constraint today? Is it context, evaluators, or just errors that compound because it’s basically an open-loop system?

JEFF DEAN: Obviously we want agents to be able to run for very long periods of time, because that’s how they’re going to solve more and more complicated problems. But as you observe today, they sometimes stop working well after 10 interactions with the tools and so on. Sometimes that’s because the model is trying to do something it doesn’t have a lot of experience doing. It’s been trained on a whole set of things, and as soon as you get a little bit off the distribution of things it knows how to do, then, like most machine learning models, its performance will suddenly start to degrade. The farther you get from the comfort zone of what it knows how to do, the more likely it is to not work as well.

There’s a bunch of things you can do. One is give the model skills and hints that tend to keep it on the more brightly lit path of things it knows how to do. Having multi-agent systems, where multiple agents try different approaches and you have another model or agent evaluating which ones seem promising, is another way to search the space of possible solutions and stick to the ones that seem most promising, discarding the ones that didn’t work or went off the rails. That’s a very useful general technique — using inference-time compute to search over plausible ways of solving the problem, which can get you much higher performance or reliability in long-running agent flows.

DIANA HU: How have you implemented this particular workflow for your agents internally?

JEFF DEAN: We have harnesses, and a whole set of skills, particularly in the internal Google development environment. We have skills so agents can use lots of our internal tooling for coding, code reviews, measuring performance, or fetching log files. Those are just skills you can add to make the base model more capable. Even though it hasn’t necessarily been trained on exactly the way Google internal engineers would fetch log files from our proprietary systems, with the right skill definition you can actually get it to work. That improves the usefulness of the agents.

Where Startups Can Win

DIANA HU: Now let’s talk about where startups can win. This is a section I personally care a lot about, because everyone in this room needs to decide what to build in the future if you’re a future founder. Google co-designs everything in the system, from the processors to the products. Which layers will someone like Google keep building and compounding and getting better at? And where can a two- or three-person team still win?

JEFF DEAN: Google, our Gemini models, and our hardware infrastructure are really trying to build very general models that can do almost anything. In a lot of cases that means we don’t have a lot of attention on particular domains where perhaps a really well-designed surface — and maybe a model and set of skills, or a specialized model that isn’t in the general mix of things our models do well — can have a significant advantage, because you can build something delightful, really high accuracy, really high quality, for a domain you’re really passionate about. That’s where two or three people in a room, building something they’re really excited about, can have an advantage.

But I’d also caution that the general models are definitely getting better at a broader and broader range of things. So you have to figure out: is the thing you’re working on going to be durable, or are forefront models going to get better at that in the next 6 or 12 months? Or is it something they’re not going to be able to do for a couple of years, or three years? You want to weigh that as you’re deciding what to work on.

DIANA HU: Let’s dive deeper into this. The general models, of course, you’re going to keep making better. How should the audience reason about the areas they don’t cover? How should founders think about what to pick and work on?

JEFF DEAN: The most important thing is to pick something you’re super excited about, that you want to build, and that you think would be useful in the world. If you do that, you’re already way ahead of someone who wakes up thinking “I don’t really want to do this” or who’s building something that isn’t actually that useful to the world or to many people. That’s the number one selection criterion I’d apply for what problem to work on next.

Second, look at what current general models can do in that problem domain. Test them — are they able to do this thing very well? If they’re completely failing, that’s probably a good sign. If they’re able to do some of it but not very well, that’s maybe not a great sign, because that’s probably a sign the capability is starting to be present in those models, and with more training data or larger-scale models, it’s likely to get better. So look for something where the model succeeds 0% or 1% of the time, not 20%.

DIANA HU: How do you find those? Are those things effectively out of distribution from the training set? What exactly is the problem shape that fits that?

JEFF DEAN: Sometimes it’s a product you build that might have access to a particular kind of data that the underlying general model might not. It might be that you’re building something to help users organize their own personal information, and the model won’t necessarily have access to that. There you can have a big advantage, because all of a sudden your product has visibility into important data.

It could be some incredibly hard problem where, if you get the right training data, you can train a more specific model than a general-purpose one, in a very affordable way. Maybe it doesn’t take that much compute to train a niche model for a particular problem, but you can get something that’s highly accurate. That can be a really good building block for solving an important problem that isn’t handled very well by the general model.

DIANA HU: I think that’s interesting. There are basically two paths. The first is a little funny — you guys are organizing the world’s information, which is probably kind of well covered, but organizing your personal information is open, which is funny. And then the second path — you talked about more specialized models in certain domains. Can you tell us more about what some of these domains are?

JEFF DEAN: If you look at my colleagues’ work on AlphaFold, that was a very specific model for protein folding, and it was highly successful — able to handle that domain quite well, so that all of a sudden you have this amazing tool and model that can give you answers to questions about proteins and their structure really effectively. But it’s not a general model, it’s a very specific one. There are other domains where that kind of approach can work really well — maybe materials science, or chip design — things that let you leverage the capabilities of a very accurate but niche model to do things that are hard today.

Becoming an AI-Native Founder

DIANA HU: That’s a good example. So if some of you find a problem with a similar shape to AlphaFold, that could be a good problem to work on.

Now let’s assume you’ve found a problem to work on. Let’s talk about how to become an AI-native founder — how do you really get good at it? You’ve said in the past that managing a fleet of agents — say 50 or 100 — is all about writing really good, crisp design docs or specs. How do people get good at that? What do those look like?

JEFF DEAN: You’ll have a lot more success working with your virtual agents if you can clearly specify what it is you want. The clearer you are, the more the agent has guidelines — an outline of what it’s trying to accomplish. If you don’t specify very much, the agent has to infer what you meant, and in many cases it might infer something different than what you imagined.

We’ve always told computer scientists from the very beginning that it’s really important to specify what the software you’re writing is trying to accomplish before you go and write it. Now we have agent-based systems that can do the writing, but the importance of specifying what you want has actually gone up, because before you’d be handing it off to a very intelligent human who might have context or could ask follow-up questions. Agents can sometimes do that too, but clear specifications are still a really good idea.

A great example of a coding agent use case that works extremely well is translating software from one computer language to another. In that case you actually have an incredibly detailed specification — the whole software that says what the system is supposed to do. So if you have a Python implementation of something and want a Go implementation, that’s something today’s models seem incredibly capable at, because they can take all the tests in Python, make sure they pass in the Go version, translate the tests to Go, compare behavioral differences between the implementations until there aren’t any — and be highly effective, because that spec is so clear.

DIANA HU: Now let’s assume every founder gets good at running hundreds of agents at the same time, and all the code is written for them by the agents. What becomes the scarce skill?

JEFF DEAN: I think it’s really having incredibly good taste in what you ask your agents to work on. That’s the crux of it, from my background as a research problem. A researcher can have all the tools and techniques, but often most of the battle is what problem you’re going to spend your time on. If you pick the problem well and succeed in solving it, that’s way better than delightfully executing a research investigation into a rather boring problem.

That high-level wisdom of what to work on is incredibly important. Models are not necessarily going to be that good at it, so you’re going to have people steering a lot of AI-assisted computation to accomplish great things more quickly. But that essence of what you want your models to do is the key thing to focus on.

DIANA HU: Let’s talk a bit more about taste, because it gets talked about a lot right now in this era of agentic coding. How do you build taste, and make it concrete? It sounds so esoteric.

JEFF DEAN: It is a difficult thing. There isn’t a measurable objective for taste in a lot of cases. Some of it comes from experience — working on a lot of different problems in the past teaches you about what kinds of problems might be interesting in the future, or what kinds of things might be just barely possible by cobbling together previous approaches, with some open problems you might have to work on to get to something magical or highly useful.

Another way to get more experience for yourself is to write down a bunch of things you think might be important in the next 12 months. Maybe you pick one to work on, but go back and evaluate in 12 months which of the others actually seemed important, or which ones other people in the world went and created, and which didn’t seem to happen yet. That gives you a lot more samples for your own taste-creation capability, which is an important skill to have.

DIANA HU: I think a third way we were talking about earlier was doing very crazy thought experiments.

JEFF DEAN: Oh yeah, that’s another good way. I think sometimes it’s good to not take as given things that most people seem to take as given. I was doing a crazy thought experiment with some colleagues the other day about how, for 60 years, the whole silicon chip design and fabrication industry has done tremendous work to make smaller and smaller-scale transistors with very low error rates — because the assumption is that every chip we manufacture of the same design should be identical to every other chip.

DIANA HU: You don’t want any bits to flip.

JEFF DEAN: No bits should flip.

DIANA HU: Deterministic.

JEFF DEAN: There’s all kinds of error margins built in — memories have ECC these days. But at the macro scale, we don’t make that assumption when we’re building large-scale distributed systems. We build reliable large-scale distributed file systems out of unreliable parts — individual disks can fail, but your data should be safe. We have mechanisms at a higher level, like three copies of the data on three different machines and three different racks, so if any rack switch or individual machine or disk fails, you still have your data. We have Reed-Solomon encoding techniques.

But we don’t seem to do this at a really extreme level at the transistor scale of the technology we’re working on. So an interesting thought experiment is: what would happen if you tried to build a system out of transistors that might have 20 errors per day?

DIANA HU: Oh my God.

JEFF DEAN: Rather than one every million years. That would be a very different design point, and might enable really interesting things on the fabrication side. You’d have very different design methodologies, because if you want to get a signal from here to there with super unreliable transistors, you might have very different ways of signaling — sending it along multiple redundant paths to make sure it gets through on one of them. I think that would be a pretty interesting set of thought experiments. I’m not saying we should go do this — often these thought experiments don’t work out, because there are very good reasons we’ve done things a certain way for the last 50 years. But it’s good to revisit those assumptions every so often.

DIANA HU: That is so wild. It’s starting to rhyme a lot with neuromorphic computing, or the human brain and how nature works.

JEFF DEAN: Exactly — signals in our brain are not especially reliable getting from one place to another. So in brains, when really important things need to get from one place to another, there are multiple pathways that let you do that.

DIANA HU: You have such an impressive career. What’s one of these crazy assumptions you threw out the window that actually built a consequential system in the past?

JEFF DEAN: I guess—

DIANA HU: That worked out, actually.

JEFF DEAN: TPUs are a good example — being able to specialize hardware for a very niche problem domain before that domain seemed as important as it is today. The origin of MapReduce is another good example. Sanjay and I, and a number of other colleagues, had worked on various iterations of the crawling and indexing system at Google. We’d written lots of hand-parallelized code with lots of checkpointing to make sure it would be robust and reliable running on 100 or 1,000 computers, some of which might die.

But that code tended to be intermixed with the actually relatively simple thing you were often trying to do — like, I just want to look at the contents of all the webpages and compute, on the side, a mapping from URL to what language the page is in, or the text of the page. It would get obscured by all this other code for parallelization and reliability.

We remembered our training in functional languages and realized we could squint at those problems and develop the MapReduce abstraction that you could layer above the implementation, with all the checkpointing and reliability mechanisms in a lower-level library everything could build on. That became a hugely successful way of dealing with very large-scale computations at Google, in a robust and reliable way — from that thought experiment of, well, if we squint at it, could we find lots of problems that fit into this abstraction?

AI That Builds AI

DIANA HU: That’s impressive — that thought experiment led you to create MapReduce. Now let’s go back to your interest right now in customized hardware. AlphaChip lays out chips. AlphaEvolve proposes solutions, evaluates them, and keeps the ones that work. It seems like you’re starting to build systems that can compound and build AI that builds AI.

JEFF DEAN: More generally, there’s the foundation of the scientific method — you propose an experiment, implement what you need to run it, and evaluate it, then get results. There are more and more problems where you can now automate that whole loop — not just running a few experiments, but running many, many experiments, because you can automate the loop and make its latency extremely low. That’s going to be really important. It’ll enable us to tackle lots of different problem domains in science, engineering, machine learning, model design itself, and engineering tasks like designing chips.

If you can do those things in an automated way, with an orchestration framework that takes very high-level objectives and breaks them into subproblems — each subproblem being one of these automated exploration loops, with an orchestration framework that puts subproblem solutions together into the overall solution — that’s going to be really impactful. It’ll accelerate machine learning progress, accelerate science, and accelerate engineering. I think that’s going to be amazing.

DIANA HU: That sounds awesome. It sounds like a lot of fields where you can have very good evaluators, and maybe things adjacent to what can be formally verified, are ripe for AI systems that can self-improve.

JEFF DEAN: In a lot of cases your evaluators need to be made much faster. As an example, my colleagues did some work maybe a decade ago on problems in quantum chemistry, where you’re trying to understand the properties of a particular molecule. You generate a molecule configuration and want to understand its properties, so you run a very computationally intensive density functional theory simulator, which might take a night of computation to give you the answer for one thing.

What my colleagues did was take a bunch of output from those simulation runs — the input molecule configurations and the outputs of the expensive simulator — and use it to train a neural approximation of the simulator. That validation device, instead of taking a night, was 300,000 times faster,

DIANA HU: Wow.

JEFF DEAN: and nearly as accurate as running the full-scale simulator. That completely changes how you’d do science, because now you have 10 million things to screen, and you could do that while you go to lunch rather than it being a six-month endeavor scraping together enough compute to run all these simulations. I think there’s a lot of room in many domains for much faster — possibly learned — validation models that can get you an approximation to the true answer much more rapidly. That changes how those experimental loops can be thought of, and how quickly you can go around them.

DIANA HU: What are some of the spaces and problems you’re super excited that this sped-up scientific method is going to solve or achieve?

JEFF DEAN: Well, clearly machine learning itself is one. Can we have a model that’s able to recursively self-improve by running lots of experiments? If you think about how models are improved today in large research teams, people think of ideas, run a bunch of small-scale experiments, see if those worked out well, take the most promising ones and try them at larger scale, evaluate them, and integrate the results into a new recipe for the model.

I don’t think there’s any real impediment to making that a much more automated loop, where the model itself decides to explore — maybe with a nudge from people at the highest level, like, “why don’t you try some new ideas around model architectures that incorporate this?” It goes and runs lots of experiments, sees which ones work, and those get incorporated at a much more rapid rate. You want to optimize your discoveries per unit of compute input.

Rejection, Persistence, and the Distillation Paper

DIANA HU: Very cool. Now, going back to the room — as all of you become founders or start your careers, you will probably collect lots of rejections. That will happen. It happened to you too, Jeff.

There’s a story that in 2014, you, with Geoffrey Hinton and Oriol Vinyals, wrote a paper on distillation — taking a big teacher model to train a much smaller, more efficient model that’s a lot cheaper to compute, with fewer parameters. It’s become a trick everyone is using right now in industry. And this paper got rejected at NeurIPS.

JEFF DEAN: Yeah. I don’t fault the program committee, because a lot of times a paper gets three reviews, and one of the reviewers said it was unlikely to have significant impact.

DIANA HU: Unlikely to have significant impact.

JEFF DEAN: But when we wrote the paper, we actually saw this was a super important problem, because we knew making cheaper, highly capable models from larger-scale models was something we desperately wanted to do — we wanted to serve models to more and more people in many different domains, like speech or vision. But sometimes the reviewer maybe didn’t have that experience, because maybe they weren’t thinking about large-scale AI services, and were thinking instead about whether it’s a fundamental advance.

So it gets rejected every so often — that’s fine. We put it on arXiv, people read it, people use it, it’s all good. We do use it in making our Flash models, for example, from our larger-scale Pro model. That’s partly why our Flash models in Gemini are so capable relative to their size and speed.

DIANA HU: They’re some of the best in the benchmarks for their model size class, which is impressive. I think part of the lesson is that even if you get rejected, keep going.

JEFF DEAN: Yeah, that’s the lesson I would distill from that.

Advice for a 25-Year-Old Jeff Dean

DIANA HU: Now, I think the fun thing is that you basically joined Google when it was a 20-person startup, back in 1999. If you were to take the young Jeff Dean from way back then and teleport him to today, with your skills—

JEFF DEAN: I’m feeling so vigorous and young now.

DIANA HU: What would you do? Join a frontier lab, start a company? What would the Jeff Dean of today, a 25-year-old Jeff Dean, do?

JEFF DEAN: It’s always hard to say, and it’s a very personal choice of what you want to spend your time on. To me, some of the most important questions are: are you going to work on something you really care about? If you’re able to make progress on it with a bunch of colleagues you like working with, will that make a difference in the world in some positive way? Will you be able to offer something that tremendously helps — maybe it’s a very niche thing that helps biochemists, or maybe it’s broader, helping programmers or all consumers on the internet.

What you should strive to do is have a positive impact in the world, work with people you enjoy working with, and work hard and do your best. In terms of the trade-off between joining a frontier lab versus starting a company with one, two, or three close friends — those are different experiences. In a large established organization, you have structure, lots of amazing colleagues who know things you don’t, lots of interesting problems to work on, and you already have a platform for impact through your work influencing lots of people in the world.

As a very small startup, you have to be passionate about something, and there’s a lot of risk in taking on that problem in a way that you’ll succeed and grow the endeavor. But that can also be incredibly rewarding. It’s really up to personal taste, but at the very least, regardless of the path you take, ask yourself: if I work on this problem and the best possible outcome happens, will the world be a lot better in some way, or will the world just go, “eh, that’s kind of cool, but whatever”? That’s not the kind of thing you should spend your time on.

Building and Finding Great Teams

DIANA HU: Now let’s talk more about that second path — working with people you really like in a small team. You’ve been an incredible mentor and manager to many engineers, and you’ve built huge systems. What are some lessons for everyone here on how to get the most out of working with smart people, or finding smart people?

JEFF DEAN: You always want to find people who have really good skills in some area needed by the team you’re trying to form, whether that’s inside a company or starting a company. But you also want to find people you delight in being around, because you’re going to spend a lot of time together working on really hard problems. You want people who are low-ego, who are team players, who have complementary skills to your own, perhaps.

I always find working in a small team, where people know things I don’t and I have skills others don’t have as much of, super fun — because you’re collectively building or working on something none of you could do individually. In the process, you gain a lot of new knowledge and new skills yourself, and so do they. You want to view your engineering or research career as an amazing tool belt of techniques, and you always want to be adding new tools to that belt, because you never know when you’ll come across a problem where you need four specialized tools rather than three. Adding more tools makes it more likely that the problems you encounter in the future will be solvable by you.

What’s Next

DIANA HU: One last thing. I’m pretty sure someone in this room, or multiple people, will eventually build something as consequential as what you’ve done with MapReduce, TPU, distillation, and so on. What problem do you hope they’ll be working on?

JEFF DEAN: Oh yeah. There are a lot of interesting problems in the world, and I’ll just rattle off a few — this isn’t exhaustive, because the world is a very big place, full of problems.

I’m particularly excited about new approaches to hardware, like the thought experiment we discussed — much more efficient inference hardware. I think there are radically different kinds of algorithms for machine learning that might be much more data-efficient than the approaches we use today. If you think about our large-scale models today, they probably see 1,000 times as much data as a human does by the age of 18, yet a human by 18 is better at a lot of things, and on par with those frontier models that have seen way more data. Could you come up with much more data-efficient systems that learn continuously, learn from their own actions? Continual learning is a really interesting problem.

Multi-agent interactions are interesting too. I think creating better ways of having discourse among people in the world could be interesting — are there ways to have much more civil conversations, and help people meet others all over the world they should know based on their interests? These are interesting things. There are lots of cool things in the world, and we should all go strive to make even cooler things occur.

DIANA HU: That sounds wonderful. Thank you so much, Jeff Dean. That’s all we have today.

JEFF DEAN: Thank you. Appreciate it. Thank you all.

DIANA HU: Thank you all.

Related Posts

• Transcript of Dr. Fei-Fei Li Interview on Huberman Lab
• Transcript of Kim Dotcom Interview on Surveillance: Tucker Carlson Show
• Full Transcript: Zanny Minton Beddoes Interviews Elon Musk
• Transcript of Daniel Kokotajlo Interview: Diary Of A CEO Podcast
• Transcript: Why a Social Media Ban Won’t Save Teens – Candice Odgers