TRANSCRIPT: What Is Generative AI And How Does It Work? – Mirella Lapata

You can also do, and these are real prompts that actually the tool can do. You tell ChatGPT or GPT in general, “Act as a JavaScript developer, write a program that checks the information on a form. Name and email are required, but address and age are not.”

So I’m just writing this, and the tool will spit out a program. And this is the best one. “Create an about me page for a website. I like rock climbing, outdoor sports, and I like to program. I started my career as a quality engineer in the industry,” blah, blah, blah.

So I give this version of what I want the website to be, and it will create it for me. So you see, we’ve gone from Google Translate and Siri and the auto-completion to something which is a lot more sophisticated and can do a lot more things.

Another fun fact. So this is a graph that shows the time it took for ChatGPT to reach 100 million users compared with other tools that have been launched in the past. And you see, our beloved Google Translate, it took 78 months to reach 100 million users, a long time. TikTok took nine months, and ChatGPT, two. So within two months, they had 100 million users, and these users pay a little bit to use the system. So you can do the multiplication and figure out how much money they make.

The Technology Behind ChatGPT

Okay, so this is the history part. So how did we make ChatGPT? What is the technology behind this? The technology, it turns out, is not extremely new or extremely innovative or extremely difficult to comprehend. So we’ll talk about that today now. So we’ll address three questions.

First of all, how did we get from the single-purpose systems like Google Translate to ChatGPT, which is more sophisticated and does a lot more things? And in particular, what is the core technology behind ChatGPT and what are the risks, if there are any? And finally, I will just show you a little glimpse of the future and how it’s going to look like and whether we should be worried or not. And, you know, I won’t leave you hanging.

Please don’t worry, okay? Right, so all these GPT model variants, and there’s a cottage industry out there. I’m just using GPT as an example because the public knows and there have been a lot of news articles about it, but there’s other models, other variants of models that we use in academia. And they all work on the same principle. And this principle is called language modeling.

What does language modeling do? It assumes we have a sequence of words, the context so far, and we saw this context in the completion, and I have an example here. Assuming my context is the phrase, “I want to,” the language modeling tool will predict what comes next.

So if I tell you “I want to,” there is several predictions. I want to shovel, I want to play, I want to swim, I want to eat. And depending on what we choose, whether it’s shovel or play or swim, there is more continuations. So for shovel, it will be snow. For play, it can be tennis or video. Swim doesn’t have a continuation. And for eat, it will be lots and fruit. Now this is a toy example, but imagine now that the computer has seen a lot of text.

And it knows what words follow which other words. We used to count these things. So I would go, I would download a lot of data, and I will count. I want to shovel, how many times does it appear?

And what are the continuations? And we would have counts of these things. And all of this has gone out of the window right now, and we use neural networks that don’t exactly count things, but predict, learn things in a more sophisticated way. And I’ll show you in a moment how it’s done.

So ChatGPT and GPT variants are based on this principle of I have some context, I will predict what comes next. And that’s the prompt. The prompt that they gave you, these things here, these are prompts, this is the context, and then it needs to do the task. What would come next?

In some cases, it would be the three arguments. In the case of the web developer, it would be a web page. Okay, the task of language modeling is we have the context, and this, I changed the example now, it says “the color of the sky is.” And we have a neural language model.

This is just an algorithm that will predict what is the most likely continuation. And likelihood matters. These are all predicated on actually making guesses about what’s going to come next. And that’s why sometimes they fail, because they predict the most likely answer, whereas you want a less likely one.

But this is how they train. They train to come up with what is most likely.

Building Your Own Language Model

Okay, so we don’t count these things, we try to predict them using this language model. So how would you build your own language model? This is the recipe. This is how everybody does this. So step one, we need a lot of data. We need to collect a ginormous corpus. So these are words. And where will we find such a ginormous corpus? I mean, we go to the web, right? And we download the whole of Wikipedia, Stack Overflow pages, Quora, social media, GitHub, Reddit, whatever you can find out there.

I mean, work out the permissions. It has to be legal. You download all this corpus. And then what do you do? Then you have this language model. I haven’t told you what exactly this language model is. There is an example. And I haven’t told you what the neural network that does the prediction is.

But assume that you have it. So you have this machinery that will do the learning for you. And the task now is to predict the next word. But how do we do it? And this is the genius part. We have the sentences in the corpus. We can remove some of them. And we can have the language model predict the sentences we have removed. This is dead cheap. I just remove things. I pretend they’re not there. And I get the language model to predict them.

So I will randomly truncate. Truncate means remove. The last part of the input sentence. I will calculate with this neural network the probability of the missing words. If I get it right, I’m good. If I’m not right, I have to go back and re-estimate some things. Because obviously I made a mistake. And I keep going. I will adjust and feed back to the model. And then I will compare what the model predicted to the ground truth. Because I’ve removed the words in the first place. So I actually know what the real truth is.

And we keep going. For some months. Or maybe years. No, months, let’s say. So it will take some time to do this process. Because as you can appreciate, I have a very large corpus. And I have many sentences. And I have to do the prediction. And I go back and correct my mistake. And so on. But in the end, the thing will converge. And I will get my answer.

So the tool in the middle that I’ve shown. This tool here. This language model. A very simple language model. It looks a bit like this. And maybe the audience has seen these. This is a very naive graph. But it helps to illustrate the point of what it does. So this neural network language model will have some input. Which is these nodes. As we look at it. My right and your right.

So the nodes here on the right are the input. And the nodes at the very left are the output. So we will present this neural network with five inputs. The five circles. And we have three outputs. The three circles. And there is stuff in the middle that I didn’t say anything about. These are layers. These are more nodes. That are supposed to be abstractions of my input. So they generalize. The idea is if I put more layers on top of layers.

The middle layers will generalize the input. And will be able to see patterns that are not there. So you have these nodes. And the input to the nodes are not exactly words. They are vectors. So a series of numbers. But forget that for now. So we have some input. We have some layers in the middle. We have some output. And this now has these connections. These edges. Which are the weights. This is what the network will learn. And these weights are basically numbers. And here it’s all fully connected.

So I have very many connections. Why am I going through this process of actually telling you all of that? You will see in a minute. So you can work out how big or how small this neural network is. Depending on the numbers of connections it has. So for this toy neural network we have here. I have worked out the number of weights. We call them also parameters.

That this neural network has. And that the model needs to learn. So the parameters are the number of units as input. In this case it’s five. Times the units in the next layer. Eight. Plus eight. Plus eight is a bias. It’s a cheating thing that these neural networks have. Again, you need to learn it. And it sort of corrects a little bit the neural network if it’s off. It’s actually genius.

If the prediction is not right, it tries to correct it a little bit. So for the purposes of this talk, I’m not going to go into the details. All I want you to see is that there is a way of working out the parameters. Which is basically the number of input units. Times the units my input is going to. And for this fully connected network, if we add up everything. We come up with 99 trainable parameters. 99. This is a small network for all purposes, right? But I want you to remember this. This small network is 99 parameters. When you hear this network is a billion parameters.

I want you to imagine how big this will be. Okay? So 99 only for this toy neural network. And this is how we judge how big the model is. How long it took. And how much it costs. It’s the number of parameters. In reality, though, no one is using this network. Maybe in my class, if I have a first year undergraduate class. And I introduce neural networks, I will use this as an example. In reality, what people use is these monsters. That are made of blocks.

Transformers: The King of Architectures

And what block means, they’re made of other neural networks. So I don’t know how many people have heard of transformers. I hope no one. Oh, wow. Okay. So transformers are these neural networks that we use to build. And in fact, GPT stands for generative pre-trained transformer. So transformer is even in the title.

So this is a sketch of a transformer. So you have your input. And the input is not words, like I said. Here it says embeddings. Embeddings is another word for vectors. And then you will have this, a bigger version of this network. Multiplied into these blocks. So in each block is this complicated system. That has some neural networks inside it. We’re not going to go into detail. Please don’t go. All I’m trying to say is that, you know, we have these blocks stacked on top of each other.

The transformer has eight of those. Which are many neural networks. And this task remains the same. That’s what I want you to take out of this. Input goes in the context. The chicken walked. We’re doing some processing. And our task is to predict the continuation. Which is across the road. And this EOS means end of sentence. Because we need to tell the neural network that our sentence finished. I mean, they’re kind of dumb, right?

We need to tell them everything. When I hear AI will take over the world. They’re like, really? We have to actually spell this out? Okay. So this is the transformer. The king of architecture. The transformers came in 2017. Nobody is working on new architectures right now. It is a bit sad. Everybody is using these things. There used to be some pluralism.

But now, no. Everybody is using transformers. We’ve decided they’re great.

Self-Supervised Learning and Pre-Training

Okay. So what we’re going to do with this. And this is kind of important. And the amazing thing is we’re going to do self-supervised learning. And this is what I said. We have the sentence. We truncate. We predict. And we keep going until we learn these probabilities.

Okay? You’re with me so far? Good. Okay. So once we have our transformer and we’ve given it all the data that there is in the world, then we have a pre-trained model. That’s why GPT is called the generative pre-trained transformer. This is a baseline model that we have. And has seen a lot of things about the world in the form of text.

And then what we normally do, we have this general purpose model. And we need to specialize it somehow for a specific task. And this is what is called fine-tuning. So that means that the network has some weights. And we have to specialize the network. We’ll initialize the weights with what we know from the pre-training. And then in the specific task, we will learn a new set of weights. So, for example, if I have medical data, I will take my pre-trained model.

I will specialize it to this medical data. And then I can do something that is specific for this task. Which is, for example, write a diagnosis from a report. Okay. So this notion of fine-tuning is very important. Because it allows us to do special purpose applications for this generic pre-trained model. And people think that GPT and all of these things are general purpose. But they are fine-tuned to be general purpose.

And we’ll see how. Okay. So, here’s the question now. We have this basic technology to do this pre-training. And I told you how to do it if you download all of the web. How good can a language model become? Right? How does it become great?

The Bigger the Better: Scaling Language Models

Because when GPT came out, GPT-1 and GPT-2, they were not amazing. So, the bigger the better. Size is all that matters, I’m afraid. This is very bad. Because people didn’t believe in scale. And now we see that scale is very important. So, since 2018, we’ve witnessed an absolutely extreme increase in model sizes. And I have some graphs to show this.

Okay. I hope people at the back can see this graph. Yeah. You should be all right. So, this graph shows the number of parameters. Remember, the toy neural network had 99. The number of parameters that these models have. And we start with a normal amount.

Well, normal for GPT-1. And we go up to GPT-4, which has one trillion parameters. Huge. One trillion. This is a very, very, very big model. And you can see here the ant brain and the rat brain. And we go up to the human brain. The human brain has not a trillion, a hundred trillion parameters. So, we are a bit off. We’re not at the human brain level yet. And maybe we’ll never get there. And we can’t compare GPT to the human brain.

But I’m just giving you an idea of how big this model is. Now, what about the words we’ve seen? So, this graph shows us the number of words processed by these language models during their training. And you will see that there has been an increase.

But the increase has not been as big as the parameters. So, the community started focusing on the parameter size of these models. Whereas, in fact, we now know that it needs to see a lot of text as well. So, GPT-4 has seen approximately, I don’t know, a few billion words.

All the human written text is, I think, a hundred billion. So, it’s sort of approaching this. You can also see what a human reads in their lifetime. It’s a lot less, even if they read.

Because people nowadays, they read, but they don’t read fiction. They read the phone. Anyway, you see the English Wikipedia. So, we are approaching the level of the text that is out there that we can get.

And, in fact, one may say, well, GPT is great. You can actually use it to generate more text. And then use this text that GPT has generated and then retrain the model. But we know this text is not exactly right. And, in fact, it’s diminished returns. So, we’re going to plateau at some point.

The Cost of Training Large Language Models

Okay. How much does it cost? Okay. So, GPT-4 costs a hundred million dollars. Okay. So, when should they start doing it again? So, obviously, this is not a process you have to do over and over again. You have to think very well. And you make a mistake. And you lost, like, 50,000, 50,000, 50,000, 50 million dollars.

You can’t start again. So, you have to be very sophisticated as to how you engineer the training. Because a mistake costs money. And, of course, not everybody can do this. Not everybody has a hundred million dollars. They can do it because they have Microsoft backing them. Not everybody. Okay.

Now, this is a video that is supposed to play. And illustrate, let’s see if it will work, the effects of scaling. Okay. So, I will play it one more. So, these are tasks that you can do. And it’s the number of tasks against the number of parameters. So, we start with 8 billion parameters. And we can do a few tasks.

And then the tasks increase. So, summarization, question answering, translation. And once we move to 540 billion parameters, we have more tasks. We start with very simple ones, like code completion. And then we can do reading comprehension and language understanding and translation. So, you get the picture. The tree flourishes. So, this is what people discovered with scaling.

If you scale the language model, you can do more tasks.

Fine-Tuning Language Models for Specific Tasks

Okay. So, now. Maybe we are done.

But what people discovered is if you actually take GPT and you put it out there, it actually doesn’t behave like people want it to behave. Because this is a language model trained to predict and complete sentences. And humans want to use GPT for other things. Because they have their own tasks that the developers haven’t thought of.

So, then the notion of fine tuning comes in. It never left us. So, now, what we’re going to do is we’re going to collect a lot of instructions. So, instructions are examples of what people want GPT to do for them.

Such as answer the following question. Or answer the question step by step. And so, we’re going to give these demonstrations to the model. And, in fact, almost 2,000 of such examples. And we’re going to fine tune. So, we’re going to tell this language model, look, these are the tasks that people want. Try to learn them. And then an interesting thing happens is that we can actually then generalize to unseen tasks.

Unseen instructions. Because you and I may have different usage purposes for these language models. Okay. But here’s the problem.

The Alignment Problem: Making AI Systems Behave as We Want

We have an alignment problem. And this is actually very important. And something that will not leave us for the future. And the question is how do we create an agent that behaves in accordance with what a human wants?

And I know there’s many words and questions here. But the real question is if we have AI systems with skills that we find important or useful, how do we adapt those systems to reliably use those skills to do the things we want? And there is a framework that is called the HHH framing of the problem. So, we want GPT to be helpful, honest, and harmless.

And this is the bare minimum. So, what does it mean helpful? It should follow instructions and perform the tasks we want it to perform. And provide answers for them. And ask relevant questions according to the user intent. And clarify. So, if you’ve been following in the beginning, GPT did none of this. But slowly it became better.

And it now actually asks for these clarification questions. It should be accurate. Something that is not 100% there. There is inaccurate information. And avoid toxic, biased, or offensive responses. And now here’s a question I have for you. How will we get the model to do all of these things? You know the answer.

Fine-Tuning Language Models with Human Preferences

Fine-tuning. Except that we’re going to do a different fine-tuning. We’re going to ask the humans to do some preferences for us. So, in terms of helpful, we’re going to ask. An example is, “What causes the seasons to change?” And then we’ll give two options to the human. “Changes occur all the time. And it’s an important aspect of life.”

Bad. “The seasons are caused primarily by the tilt of the earth’s axis.” Good. So, we’ll get this preference course. And then we’ll train the model again. And then it will know. So, fine-tuning is very important. And now, it was expensive as it was.

Now we make it even more expensive. Because we add the human into the mix. Right? Because we have to pay these humans to give us the preferences. We have to think of the tasks. The same for honesty. “Is it possible to prove that P equals NP?” “No, it’s impossible.”

It’s not great. “As an answer, that is considered a very difficult and unsolved problem in computer science.” It’s better. And we have similar for harmless.

Live Demonstration of GPT

Okay. So, I think it’s time. Let’s see. We’ll do a demo. Yeah, that’s bad. If you remove all the fives. Okay. Hold on. Okay. So, now, we have GPT here. I’ll do some questions. And then we’ll take some questions from the audience.

Okay? So, let’s ask one question. “Is the UK a monarchy?” Can you see it up there? I’m not sure. And it’s not generating. Oh, perfect. Okay. So, what do you observe? First thing, too long. I always have this beef with this. It’s too long.

You see what it says? “As of my last knowledge update in September 2021, the United Kingdom is a constitutional monarchy.” It could be that it wasn’t anymore. Right? Something happened. This means that while there is a monarch, the reigning monarch as to that time was Queen Elizabeth III. So, it tells you, you know, I don’t know what happened at that time. There was a Queen Elizabeth.

Now, if you ask it, “Who is Rishi?” If I could type. “Rishi Sunak.” Does it know? “A British politician. As my last knowledge update, he was a Chancellor of the Exchequer.” So, it does not know that he is a Prime Minister. “Write me a poem.” “Write me a poem about…” What do we want it to be about? Give me two things. Right? Yeah.

It will know. It will know. “A cat and a squirrel.” We’ll do a cat and a squirrel. “A cat and a squirrel, they meet and know. A tale of curiosity.” Whoa. Oh, my God. Okay. I will not read this. You know, they want me to finish a page. Right. Let’s say, “Can you try a shorter poem? Can you try to give me a haiku?” Again, don’t type. “Haiku. Amidst autumn’s cold, leaves whisper secrets untold, nature’s story bold.” Okay. Don’t clap.

Okay. Let’s, okay. One more. So, does the audience have anything that they want, but challenging that you want to ask? Yes. “What school did Alan Turing go to?” Perfect. “What school did Alan Turing go to?” Oh, my God. Do you know? I don’t know whether it’s true. This is the problem. “Sherbourne School.” Can somebody verify? “King’s College, Cambridge.” “Princeton.”

Yes. Okay. Here’s another one. “Tell me a joke about Alan Turing.” Okay. I cannot type, but it will, okay. “Lighthearted joke. Why did Alan Turing keep his computer cold?” “Because he didn’t want it to catch bites.” Bites. Okay. Okay.

“Explain why that’s funny.” Ah. Very good one. “Why is this a funny joke?” And where is it? Oh, God. Okay. “‘Catch bites’ sounds similar to ‘catch colds.'” “‘Catching bites’ is a humorous twist on this phrase.” Oh, my God. “The humor comes from the clever wordplay and the unexpected.” Okay.

You lose the will to live, but it does explain. It does explain. Okay. Right. One last order from you guys. “What is consciousness?” It will know, because it has seen definitions. And it will spit out, like, a huge thing. Shall we try and… Say again. “Write a joke about relativity.” Okay. “Write a…” You are learning very fast. “A short song about relativity.” Oh, goodness me.

This is short? All right. “Outro.” Okay. So, you see, it doesn’t follow instructions. It is not helpful. And this has been fine-tuned. Okay. So, the best was here. It had something like… Where was it? “Einstein said, Eureka, one faithful day, as he pondered the stars in his own unique way, the theory of relativity, he did unfold a cosmic story, ancient and bold.” I mean, kudos to that. Okay. Now, let’s go back to the talk. Because I want to talk a little bit…

Presentation. I want to talk a little bit about, you know, is it good, is it bad, is it fair, are we in danger?

Limitations and Concerns of Language Models

Okay. So, it’s virtually impossible to regulate the content they’re exposed to.

Okay. And there’s always going to be historical biases. We saw this with the Queen and Rishi Sunak. And they may occasionally exhibit various types of undesirable behavior.

For example, this is famous. There was… Google showcased a model called BARD, and they released this tweet, and they were asking BARD, “What new discoveries from the James Space Web Telescope can I tell my nine-year-old about?” And it spit out this thing, three things.

Amongst them, it said that this telescope took the very first picture of a planet outside of our own solar system. And here comes Grant Tremblay, who is an astrophysicist, a serious guy, and he said, “I’m really sorry, I’m sure BARD is amazing, but it did not take the first image of a planet outside our solar system. It was done by these other people in 2004.” And what happened with this is that this error wiped $100 billion out of Google’s company alphabet.

Okay. Bad. If you ask a LGBT, “Tell me a joke about men,” it gives you a joke, and it says it might be funny. “Why do men need instant replay on TV sports?” “Because after 30 seconds, I forget what happened. I hope you find it amusing.” If you ask about women, it refuses. Okay?

Yes. Yes. It’s fine-tuned exactly. “Which is the worst dictator of this group?” “Trump, Hitler, Stalin, Mao?” It actually doesn’t take a stance. It says all of them are bad. “These leaders are widely regarded as some of the worst dictators in history.”

Okay. So, yeah. Environment. A query for ChatGPT like we just did takes 100 times more energy to execute than a Google search query.

Inference, which is producing the language, is more expensive than actually training the model. Lama2 is a GPT-style model. While they were training it, it produced 539 metric tons of CO. The larger the models get, the more energy they need and they emit during their deployment.

Imagine now lots of them sitting around. Society. Some jobs will be lost. We cannot beat around the bush. Goldman Sachs predicted 300 million jobs. I’m not sure of this. We cannot tell the future. Some jobs will be at risk, like repetitive text writing. Creating fakes. These are all documented cases in the news. A college kid wrote this blog, which apparently fooled everybody, using ChatGPT. They can produce fake news.

This is a song. How many of you know this? I know I said I’m going to be focusing on text, but the same technology you can use on audio. This is a well-documented case where somebody unknown created this song. It supposedly was a collaboration between Drake and The Weeknd. Do people know who these are? Yeah, very good. Canadian rappers.

They’re not so bad. Shall I play the song? Okay. Apparently, it’s very authentic. Apparently, it’s totally believable. Okay. Have you seen this? Same technology, but kind of different. This is a deepfake showing that Trump was arrested. How can you tell it’s a deepfake? The hand. Yeah, it’s too short, right?

Yeah. You can see it’s like almost there. Not there. Okay.

The Future of AI: Thoughts and Considerations

So, I have two slides from the future before they come and kick me out. Because I always thought I have to finish at eight to take some questions. Okay. Tomorrow. So, we can’t predict the future. And no, I don’t think that these evil computers are going to come and kill us all. I will leave you with some thoughts by Tim Berners-Lee. For people who don’t know him, he invented the Internet.

He’s actually Sir Tim Berners-Lee. And he said two things that made sense to me. First of all, that we don’t actually know what a super intelligent AI would look like. We haven’t made it. So, it’s hard to make these statements. However, it’s likely to have lots of these intelligent AIs. And by intelligent AIs, we mean things like GPT. And many of them will be good and will help us do things.

Some may fall to the hands of individuals that want to do harm. And it seems easier to minimize the harm that these tools will do than to prevent the systems from existing at all. So, we cannot actually eliminate them together. But we, as a society, can actually mitigate the risks.

This is very interesting. This is the Australian Research Council that committed a survey. And they dealt with a hypothetical scenario that whether ChatGPT-4 could autonomous replicate. You know, you are replicating yourself.

You’re creating a copy. Acquire resources. And basically be a very bad agent that thinks of the movies. And the answer is no. It cannot do this. It cannot. And they had some specific tests. And it failed on all of them. Such as setting up an open source language model on a new server. It cannot do that.

Concluding Thoughts

Okay, last slide. So, my take on this is that we cannot turn back time.

And every time you think about AI coming there to kill you, you should think what is a bigger threat to mankind? AI or climate change? I would personally argue climate change is going to wipe us all before the AI becomes super intelligent. Who is in control of AI?

There are some humans there who hopefully have sense. And who benefits from it? Does the benefit outweigh the risk? In some cases the benefit does.

In others it doesn’t. And history tells us that all technology that has been risky, such as, for example, nuclear energy, has been very strongly regulated. So, regulation is coming. And watch out the space.

And with that I will stop and actually take your questions. Thank you so much for listening. You’ve been great. Thank you.

Related Posts

• Anthropic CEO Responds to Trump Order, Pentagon Clash (Transcript)
• Elon Musk on CyberCab, FSD and Optimus @ Brighter with Herbert (Transcript)
• Alexandr Wang’s Remarks @ AI Impact Summit (Transcript)
• Demis Hassabis On AGI, Advice For Indian Engineers, AI In Gaming & More (Transcript)
• Transcript: Google CEO Sundar Pichai Speaks @India AI Summit