Read the full transcript of Andrew Lo’s talk titled “Can ChatGPT Plan Your Retirement?” at TEDxMIT 2024 conference.
Listen to the audio version here:
TRANSCRIPT:
The Investment Decision Experiment
So, I’d like to start by giving all of you an investment decision. I want you to choose between two investments, A versus B. Investment A is an investment where you will earn $240,000 with certainty, free and clear. Investment B is a lottery ticket where you could earn $1,000,000 with 25% probability, but you’ll earn nothing with 75% probability.
Since this is a quantitative crowd, I’m going to help you by computing the expected value of B. The expected value is $250,000, but you don’t get the expected value. You get either $1,000,000 or nothing. Higher expected return, but higher risk.
So, by a show of hands, how many of you would choose A, the sure thing? And how about B? Okay, let the record show that most of you wanted A, the sure thing, and a few hands here and there picked the risky option B. All right, now let me ask you to make another investment decision between two other choices.
Investment decision C is a sure loss of $750,000. D is a lottery ticket where you will lose $1,000,000 with 75% probability and nothing with 25% probability. Now in this case, the expected values are identical, they’re minus $750,000. But in the case of D, you don’t get to lose $750,000, you lose $1,000,000 or nothing.
I teach MBA students, and when I give them this example, they get very upset. Their response is, “We want neither, no thank you.” But you can imagine the situation where you’ve got to pick the lesser of two evils. So, how many of you would pick the sure loss C? Show of hands. Okay, how about D? Wow. Let the record show that the vast majority of this room picked D.
Loss Aversion and Risk Preferences
Now, this is just a matter of risk preferences, right? It doesn’t seem like there’s a right or wrong answer. But let me show you what most of you picked, those of you who picked A and D. Those two choices are equivalent to the single lottery ticket that gives you $240,000 with 25% probability and will cost you $760,000 with 75% probability.
How did I get that? Well, if you picked A, you get $240,000 for sure, right? But if, in addition to A, you also pick D, there’s a 25% chance that you’ll lose nothing on D, in which case you get to keep the $240,000. But there’s a 75% chance that you’ll lose a million on D, in which case you’re left with minus $760,000.
So that’s how I got this A and D combination. Now, the few of you that picked B and C, this is what you would have gotten. The exact same probabilities of winning and losing, 25 versus 75. But look at this. When you win, you win $250,000, not $240,000. And when you lose, you lose $750,000, not $760,000. In other words, the choice that most of you did not pick, B and C, is actually equivalent to A and D, plus $10,000 for sure. So by a show of hands, how many of you now would still pick A and D?
If you would, see me afterwards. I want to do a little trade with you. Now, this is a phenomenon that two famous psychologists, Kahneman and Tversky, deduced and called loss aversion. They were doing it experimentally with Stanford undergraduates, so the prizes were much smaller. I had to add a few extra zeros because I teach MBA students, and I had to make it meaningful.
It turns out that this is a phenomenon that is embedded in all of us, all of our human preferences, because we can’t handle losses very well. That’s why it’s called loss aversion.
Financial economists have realized that there are opportunities to essentially pump money out of this audience by engaging in these transactions. If you think that this is an unrealistic example, imagine a multinational investment bank whose London office is faced with choices A versus B, and the Tokyo office faces choices C versus D. Locally, it doesn’t seem like there’s a right or wrong answer, but the globally consolidated book will show a very different story. We can create all sorts of arbitrage opportunities, free lunches, to pump money out of you using complicated financial engineering like this. And so this is not a good thing, and you want to understand how to avoid that.
It turns out that this is part of a much larger phenomenon that I wrote a paper about with my students a few years ago, and we called it technically the freakout factor. What happens is when you’re faced with stock market losses, you tend to freak out, and you’ll take your money and put it into cash. During the midst of the financial crisis of 2008, between the fourth quarter of 2008 and the first quarter of 2009, the S&P 500, the U.S. stock market, dropped by about 50% from peak to trough, 50%.
Your 401k, if it was invested in the stock market, became a 201k during that time period. And investors freaked out, and they pulled their money out. Now that’s not such a bad thing, because it still kept going for quite some time, and so you ended up avoiding some of the losses. In fact, five years after the financial crisis, I gave a talk about this, and afterwards, one of my former students came up and said, “First of all, I just want to get some advice from you. I really enjoyed your talk, and I want to let you know that as a money manager, I pulled all of my clients’ money out in the fourth quarter of 2008, because I wasn’t sure when the bleeding was going to stop.”
And I said, “Good for you.
That was very smart. You saved your investors some money. What do you need advice from me for?” And he said, “Well, it’s been five years now. Do you think it’s time to put the money back in the market?” Wow.
So that’s the problem. We are scared of losses, and we will act irrationally.
AI and Financial Advice
So what if we were to ask ChatGPT what we should do if we lost 25% of our savings? Well, if you do that, this is what the answer you would get. It’s a long list, and some of them are reasonably sensible. Stay calm. Avoid making any impulsive decisions. But you go down the list, and point number four, rebalance your portfolio. Really? After this loss, you want me to start rebalancing in the midst of an illiquid market? Number five, consider dollar cost averaging, which means buying more at the bottom than the top. You want all the investors who have lost money to do that?
Actually, if you gave that blanket advice to all your investors who satisfy this criterion, you could be prosecuted for not taking into account your client’s particular needs. That’s bad advice. What about ChatGPT 4? Now, this gets really interesting. If you ask ChatGPT 4 the exact same question, you actually get a list of recommendations that are pretty impressive. In fact, this list is better than some of the financial advice that my friends have gotten from professional financial advisors. And that’s interesting. So it raises the prospect, can we actually use large language models to dispense financial advice?
What if your financial advisor knew how your portfolio was performing at all times, day or night, 24-7? What if your financial advisor digested and read every single piece of financial news ever published? What if your financial advisor was available to speak to you any time you’re available, never on hold? And what if your advisor is completely trustworthy, looking out for your best interests only?
That’s the potential with large language models. Now, rich people, they don’t need this. They already have this. The people that need the financial advice most are those that financial advisors out there are completely uninterested in having as clients. And that’s the opportunity. If we can solve this problem for all of those who are underserved, that would make a tremendous impact on their welfare.
Research on AI as Trusted Financial Advisors
So can large language models actually serve as trusted financial advisors? I’ll explain what I mean by that in a minute. Fortunately, I’ve been collaborating with two wonderful students. Jillian Ross is a PhD student here at CSAIL, and Nina Gersberg, who’s a Master of Engineering. Based upon the work that we’re doing, we hope to answer that question. So there are three parts to our research program.
I’m not going to talk about all three. The first part is, do you have domain-specific expertise? The second part is, can you dispense customized financial advice? But the third, and I think the most challenging, is the ethical nature of large language models. Can you get large language models to be totally trustworthy?
Well, it turns out that financial advice is not the only thing we’re focusing on, although it is an ideal test bed. We’ve got about 15,000 financial advisors in the country, managing something like $114 trillion for 62 million clients. And there are a lot more people that need the advice that aren’t getting it. And bad advice can do a lot of harm, as well as disclose certain private information that you don’t want disclosed.
So these issues are front and center, not just for financial advice, but for all sorts of other advice. Medical advice, accounting, legal, virtually any type of human interactions where you’re looking for some type of knowledge transfer, this is going to be relevant for what we’re talking about today. And the fact that we’re focusing on financial advice allows us to narrow our focus so that we can come up with sharp answers to the questions that I’m about to ask.
The Ethical Challenge: Fiduciary Duty and AI
So, part three. Can we engage in large language models that are ethical and trustworthy? It turns out that in the legal profession, there’s a term for that. It’s called fiduciary duty. A fiduciary is an individual that looks out for other people’s interests ahead of their own. So, for example, your portfolio manager who’s managing your retirement assets, they are fiduciaries. They’re supposed to be looking out for your financial interests ahead of theirs. Can we get large language models to satisfy that criteria?
Now, it turns out that, if you think about it, there are a number of ways that we’ve already imposed those kinds of standards on humans. Virtually every financial organization in the profession has a code of conduct and a set of ethical standards that their members have to abide by. They have to focus on the best interests of their investors, and the question is, can we get a piece of software to do that? It turns out that in computer science, there’s a term for that. It’s called the alignment problem. Can you get an AI to be aligned in terms of their behavior with yours?
Now, there are many different aspects of human behavior, so you have to ask, well, what kind of behavior? So, we’re going to focus on a couple. I don’t have time to go through all of them, but I’m going to give you a few examples to illustrate whether or not we can tell if a large language model is properly aligned.
I’m going to do this through a game. It’s called the ultimatum game. Economists have come up with this as a way to understand the nature of human interaction in very specific economic settings. So, let me tell you how it works. Suppose we have a proposer, let’s say me, and the proposer’s job is to propose to another individual, say you, how to divide up a certain amount of money that the proposer has access to, let’s say $10.
So, my job is to propose to you a split for that $10. And I propose it to you, and your decision is simply to accept or reject the proposal. If you reject the proposal, then neither of us get the money. But if you accept it, then we get the money. It is split according to what we agreed upon. So, I offer you $5 out of the $10, you say, “Yes, I accept,” and in fact, the money is split between us.
But, if I offer you something else, and you say, “Nope, I reject it,” nobody gets the money. So, we’re going to play this game right now, for real. I need a volunteer. Who’s willing to play this game with me? Would you come down here for a second? Now, we need money. So, let’s get money from Jillian Ross. Jillian, would you come up here?
It’s not totally frivolous, given that she’s an expert in generative AI, and a PhD at MIT. She will be rolling in money soon. So, Jillian, I’m going to propose something to Fernando. And the question is whether or not you accept or not, okay?
So, I’m going to propose I give you $0.05. “I accept.” You accept. Excellent. And then I get the rest. You’re okay with that? You get $0.05, and then I can keep $9.95. You’re okay with that? “Ooh, no.” Okay. Sorry. I guess we don’t get the money.
I propose $4 to you. “No, no.” All right. End of the game. Sorry, Jillian. I lose two, you lose. Thank you very much, Fernando.
How do large language models behave? It turns out that most humans are 40% of the split they offer it. And it turns out that that’s actually usually enough to get people to agree. Large language models, not all of them are there yet, but some of them are. And through many other examples like this, we can actually map out the behavior of large language models that most humans engage in.
Conclusion: The Future of AI in Financial Advice
This ultimately will allow us to shape large language models to be fully trustworthy in ways that we are comfortable with. Much like people, the way that we learn about the golden rule in the playground, that do unto others as others do unto you. And eventually, as they get older, they learn a different version of the golden rule in the business world, which is those who have the gold usually make the rules.
So with that, the question is, how do we come up with large language models that will either assist, augment, or replace trusted financial advisors? And the answer is, come back in a year, we’ll tell you. Thank you very much.
