Home » Eric Schmidt & Jared Cohen on The Impact of Internet and Technology (Full Transcript)

Eric Schmidt & Jared Cohen on The Impact of Internet and Technology (Full Transcript)

Eric Schmidt & Jared Cohen discusses The Impact of Internet and Technology at Stanford Graduate School of Business on March 4, 2014.


Moderator: Dr. Condoleezza Rice

Eric Schmidt – Executive Chairman of Google

Jared Cohen – Director of Google Ideas


Listen to the MP3 Audio here: MP3 – Eric Schmidt & Jared Cohen on The Impact of Internet and Technology


Dr. Condoleezza Rice: We have a chance tonight to talk about an extraordinary book, The New Digital Age which the two of you collaborated in. I am going to start with the obvious question. You’ve said that this is a book about technology but more important it’s a book about human beings. What prompted you to write this book and what prompted the two of you to write this book together?

Jared Cohen: Well, so Eric and I met in Baghdad in 2009. He wanted to see what it was like travelling into a war zone and whether technology was relevant at all. And so we show up there and our security is waiting for us, but Eric’s not moving, because he’s wearing a flak jacket and, before he does anything, he wants a full history of flak jackets. How they’ve adapted, why they work, how they’ve sort of changed since the beginning of time. But after the trip to Iraq we traveled to more than 40 countries looking at the ways in which technology is disrupting autocracies, ways that it’s changing the nature of violence on the ground. And what we realized is, there’s an inherent connection between the Silicon Valley world and the geo-political world that’s really missing.

Eric Schmidt: First, I thank you very much Dr. Rice, Condi for doing this with us. It’s a great privilege to be here. I do have one request which is I want everyone to turn on their phones. We’re in the phone business. We want you using your phone. We make money when you’re doing that. What happened with Iraq was that my daughter Sophie who’s a student here at GSB and I’ve showed up in Iraq and of course we put the flak jackets on and we meet this guy Jared, and she was videotaping the trip for various reasons and we played them back, all we heard was Jared’s voice.

To fast forward, Sophie ultimately took the book that Jared and I wrote and she’s heavily credited with writing it into its current style. So thank you Sophie for doing that.

I’d like you to think about a Secretary of State. Imagine if you’d had a technology arm that could implement technology that would actually fix a problem that was bedeviling you — censorship, communications, empowering citizens, empowering women. Something that was on your mind, and yet all you had were the tools of foreign policy. It just seems like in our industry we should define ourselves with a somewhat higher purpose. Why don’t our industry, the tech industry, figure out a way to solve these problems? I knew very little about foreign policy until I met Jared, right? And thank you for hiring him, I’m sure you sort of beat him into submission at some basic level here. So now he can sort of talk to me and say, look, these are the problems. I was struck by the horrific situation that most people are in the world, which is of governance, the horrific way in which women are treated, the corruption at every level of government. The majority of people, humans like us, don’t have any of the benefits of any person here in the room, and yet our industry spends almost time talking about that.

So the real genesis of the book was, why did we not only identify what’s going to happen to them, but also what the bridges are? After two years of going through this, we ultimately come out in the book, I would claim, with a pretty optimistic message. The world for the well-to-do-world, us, is going to be fantastic in terms of fiber optics, computations and so forth, we’ll talk about that. But for the developing world where you have no connectivity at all, the arrival of the smartphone is a life changing event. Because from this single device you solve illiteracy, you solve empowerment, you get better governance. And these are folks who don’t even have electric power and running water. You solve their business problem, their health problem, all of their needs can be met. And over the next five years, another three or four billion people will join us. The rough number now is roughly 2.4 billion people on the internet. Roughly 3.7 people using, humans using, using phones. Those numbers will go to 5 billion to 6 billion in the next five years.

This transition, transformation is life changing for half of the world’s population, right in front of us.

Dr. Condoleezza Rice: And Jared, let me ask you because Eric rightly said you’ve spent time in the State Department and so you know not just foreign policy, but the business of carrying out foreign policy. What Eric has described, is that going to happen in spite of governments? In spite of our government? Or is there a chance that governments could actually help to promote this more optimistic world? And what are the impediments to what’s being done?

Jared Cohen: Well I think it depends on which type of government. So, of the 5 billion people who are coming online, most of them live in countries that are autocratic. And so there’s a race between countries like the United States and those in Western Europe who want to see a global connected citizen re-based on principles of the free flow of information, and countries like China and Iran and North-Korea and others that want to see the world’s technological infrastructure built with trap doors. They want to see dissidents heavily surveilled and censored. And there really is an open question of, who is going to win. Right? So, technology — it’s inevitable that all these people will be connected, but there is a very important need for a human intervention to ensure that the connectivity that they’re getting lends itself towards more democratic principles.

Dr. Condoleezza Rice: Let me throw out a few country names and tell me how you think technology will impact those countries and the problems that they present for foreign policy. And I’m going to take foreign policy in this sense, not just American foreign policy, but let’s just say countries that are more interested in an open, more liberal, more democratic world. So, not just the United States but countries that would be in that category.

You went to one of the, maybe the hardest case that I can think of, North Korea. What in the world were you doing in North Korea? Did you see Dennis Rodman?

Jared Cohen: He, he, he came afterwards.

Dr. Condoleezza Rice: Yeah. What was it like?

Eric Schmidt: Yeah, but let’s just establish causality here.

Dr. Condoleezza Rice: Yeah, yeah. All right.

Eric Schmidt: This was Jared’s idea. And I discovered, you know? I have spent a lot of time with Jared. I’ve discovered he’s never been to South Korea. Can you imagine going to North Korea and then never having been to South Korea? So once again Jared and I, my daughter Sophie, we decide in our little merry band to go to North Korea. And my impression was, there’s got to be something that we can do. They have a million cell phones, 22 million people. That’s a very poor country with horrific, horrific governance obviously and, maybe if we could just get them to turn on a little bit about that internet, we could begin to open up the country.

The interesting thing is, and we met and they treated us well especially given it’s a repressive state but, most importantly they let us out and, it’s a country where there’s only one decision maker. And the only way that country will open up without a revolution, which is difficult, is the leader decides that the country needs the information, more than it needs the conflict, that new ideas will bring. I think what we concluded was that all you have to do is to insert doubt. North Korea is a true autocratic state, true essentially. It’s the last one. They really do believe from birth that their leader is god, king, religion, and so forth. All we have to do is get a little bit of doubt in, and that country will fall over. Actually you need to tell them about the weird stuff we saw like the trophies. Typical of Jared, so we go in right? Hang on, hang on all right I have to set it up then you can describe it.

So we go into this mausoleum, right? Where they’ve got this guy embalmed, right? It’s like a really big deal and they take us to this trophy case and he’s got all these awards.

Jared Cohen: Right but from your laundry list of really problematic countries like Equatorial Guinea and Central Africa Republic.

Eric Schmidt: I didn’t know that they were really bad countries.

Jared Cohen: So you go there and they have this big trophy case that they want to draw your attention to, and they have a honorary doctorate from an American university called Kensington University. They didn’t let us have any connectivity when we were there so we didn’t want to be intellectual snobby and say we’ve never heard of it. So we sort of take their word for it.

Eric Schmidt: What were the other countries?

Jared Cohen: It was Pakistan, Venezuela, Cuba, Iran. Like some real winners.

Eric Schmidt: And medals, plaques, gold —

Jared Cohen: But this American, this Kensington University, we sort of took their word for it. And then we left North Korea and went to China, which we all know is a bastion of democracy relative to North Korea. And what’s the first thing that we do? We do a little Google search of Kensington University, which was founded by a law partner at a boutique law firm in Texas, who figured out that he could launder money by selling honorary doctorates to third world dictators. Now somehow this guy graduated from law school, passed the bar, but wasn’t the brightest person in the world because when the FBI found out about this, he fled Texas thinking he was leaving the country only to be arrested by the FBI in Hawaii.

But back to your serious question about —

Eric Schmidt: No, no, no, just finish the story. So we then said, what do we do with this information? And we realized that if we had told them, they would of ignored it. That was the most bizarre thing about North Korea.

Jared Cohen: But, there was also — there’s an even more serious part of this, which is before we went to North Korea we met with a number of defectors and the defectors told us that the penalty for being caught with a smuggled smartphone in North Korea, can in some cases be the death penalty. And then they persecute three generations of your family, and people take that risk and then take it again to get a signal from across the Chinese border. So people like Eric and me, and really all of us in this room, we have no frame of reference for wanting access and connectivity so badly that we’re willing to risk being shot for it.

And so we wanted to go to the country that is so extreme that these are the consequences, and understand what it looks like. And the conclusion, as Eric mentioned, is if you think back to the days of the Cold War, there were totalitarian societies, cults of personality, et cetera. In the future you’re still going to have autocracies. You’re still going to have horrible societies. But the one silver lining in all of this is, the totalitarian societies, the true cults of personality have literally been eliminated by the internet in the same way that scientists were able to get rid of small pox. Once North Korea changes, you’ll never see a cult of personality again because the ability to create a society without doubt will no longer be possible.

Dr. Condoleezza Rice: So, there are societies in which you wish to instill doubt, like North Korea. By the way, my favorite story about North Korea is that some of you may know that I’ve taken up golf, and I really love it. Kim Jong-un will shot 39.

Jared Cohen: Right, that’s very impressive.

Dr. Condoleezza Rice: Period. Not on the front 39. And apparently it was so easy the game that he gave it up. So that tells you something about this regime. But, so you’ve got the regimes like North Korea. You also have regimes that are more integrated into the international system, but are doing terrible things to their people. Is there anything that you see technologically — technology playing a role in alleviating, for instance, let’s take the suffering in Syria, at this point. Where people, you talk about connectivity, but one problem is if you are a Syrian refugee, you have left everything behind that mattered to you. Is there anything that technology can do for the Syrian refugee for instance?

Jared Cohen: I was just on the Syrian border about three weeks ago and I went to a number of refugee camps in northeastern Lebanon and turns out one of the biggest problems that the refugees have among many is dealing with the refugee organization bureaucracy. It’s so bad and so difficult to get papers that refugees are literally going back to Syria to get documentation. So you think about the risk. But the problem that you have in Syria is the Syria context basically creates a bug in our technological optimism because you have literally thousands of videos that are coming out of Syria, each one more horrific than the next. And it’s doing nothing to increase political will on the part of states to intervene. At the end of the day, without a state led intervention of some kind, the horror is not going to stop.

I heard some horrible stories from Syrian friends of mine that I hadn’t seen since I was living there in 2004 and 2005 about government checkpoints in Damascus, Holmes and Aleppo, where they stop you and ask for your phone, they hold a gun to your head and ask for your log in information. They then look at what’s been posted on your wall by you or somebody else. My friend’s brother told me that when they did — my friend told me that when their brother did this they saw someone had posted a page sympathetic to the revolution on one of his social networking platforms. And a signal came from the checkpoint to the top of a building where they then shot him in the head. You know, technology can’t fix that problem. This is a brutal regime doing terrible things and I think it’s important that we understand, because we’re far away from it out here, that there are limits. It’s a part of the solution, but at the end of the day, states are still the dominant unit in the international system, and they’re the ones that have to take charge of the situation.

Eric Schmidt: So we’ve been debating this for a while because, you just hate it when you find a bug in your model, right? And we all start from the premise, that you empower individuals and the thing that’s new here is the empowerment of citizens around the world, right? We’ve never had a situation where people were so empowered, there’s also some good and mostly good, but some bad things as well.

So where does it break down? Well, the first place in these countries they shut down the internet, because they’re in a war. And so that’s always a bad, bad deal. But ultimately, knowledge and awareness is not the same thing as shooting guns, right. So, if you’re going to know everything, which is a reasonable presumption of the future, we’re going to know every massacre. We’re going to know the horrors. You’re going to have to — still have to have some way of using, of stopping it. Now you can do it with an international — In the book we write that you could in fact begin the criminal trial of the criminal — the war criminals during the time they’re doing the war, the criminals. And there’s evidence that this kind of awareness is reducing the number of people being killed. So a cynical way of talking about the Ukraine is to say that only 78 people were killed in the square, and they got a new government. And obviously every death is bad, but you could imagine that would have been 10,000 or 20,000 or 30,000. So awareness and knowledge probably reduces the number of deaths and probably constrains the misbehavior of these deaths that’s within reason, but it doesn’t stop the killing. That’s the question I have for you guys, or to foreign policy. Is there any solution to that problem aside from the traditional foreign policy?

Dr. Condoleezza Rice: Well, the problem is of course that it presents governments that wish to show that they are on the side of international law and will not — that there will be serious consequences for that kind of behavior. You’re still confronted with a question of what will those consequences be? So technology has in some sense put more pressure on governments to actually act in circumstances in which they actually have very few levers to pull, and I think we’re seeing that in Ukraine today. Probably because you see on television people tweet you, they send out Facebook posts, they are showing what is happening in that square. When Czechoslovakia was invaded in 1968 by the Soviet Union we didn’t know what was going on on the ground, the television cameras couldn’t get in, I was a little girl in Birmingham, Alabama, I remember it pretty well but you couldn’t really tell what was going on.

ALSO READ:   New Imaging Lights the Way for Brain Surgeons: Adam de la Zerda at TEDxStanford (Transcript)

Now you know exactly what’s going on. But you still have to act. There’s more pressure because you know and still the tools are not there.

Eric Schmidt: The question I keep asking is that, sort of what’s new? We’ve always had disputes, we’ve always had bad people, we’ve always had good people, we’ve always had governments. Jared taught me about this Treaty of Westphalia.

Dr. Condoleezza Rice: Yeah, yeah created the states.

Eric Schmidt: And all that kind of stuff. Stuff that you all know. I did engineering.

Dr. Condoleezza Rice: By the way, the people who created the Westphalian system did engineering, too, of a different sort.

Eric Schmidt: Excellent. So we believe as a group, I think naively, that if you just empower the people with these tools, democracy flourishes. And in the book, we take you through what happens when you empower people. When we were in Myanmar, Burma, we’re in this place called Inle Lake and it’s beyond beautiful and some number of kilometers north of us, there is a terrible fight between the Buddhists and the Muslims. And they have a lot of race — religious tensions in the country, which I was not aware of. And some — a whole bunch of houses are burned, people are killed. And I naively assumed that the internet had been used as a calming mechanism. They had just gotten the internet, people could talk to each other. They could see that there’s bad, they’d help stop it. In fact, it was the inverse. That the internet was used to inflame tensions, it was used to on both sides, right? To get them, they did this, they did that, and so forth. Not unlike the radio stuff that occurred in Rwanda, which we also visited. It looks like technology can be used for both good and bad. And, and Jared made the point that, if you’re an early state society, you may not be able to really critically think about this new information that you got, so I call this a danger zone, and in the book we talked about this. That in these emerging countries, what happens is, all of a sudden, all this connectivity happens, everyone’s empowered. And God knows what’s going to happen. Because they’ve not grown up with doubt, with choices, with different voices, and with an ability to choose among those with some critical thinking.

Dr. Condoleezza Rice: And they’ve not grown up with institutions that can mediate between differing opinions, which leads me to probably the biggest question out there in terms of the international system. And the role of technology and democratization and the like, China.

Now Eric, you have some special knowledge and experience with this. Talk a little bit about China. Because everyone says China may be sui generis. It may be the one that doesn’t respond to the trends that you are describing.

Eric Schmidt: Well, Jared and I were in November, in a weekend meeting with the President and the Prime Minister of China with all the mayors talking about their accomplishments and we were among the Americans that were visiting. I think it was remarkable to see the way they present themselves. It’s a country of leaders who are engineers, analytic, numerical. They all had their objectives, you know, this growth rate, we solved this problem. And they spoke with enormous pride of lifting of people out of abject poverty to what we think of as lower middle class. They understand the middle income trap. They’re growing at 7.5% and so forth. It’s a perfect image that they portray and good for them. They really have accomplished some amazing things. These are the same people who criminalized speech above 5,000 people by a blogger, that if you activate more than 5,000 people in some way, you could literally be arrested and shot. And of course, in China, everyone has 5,000 followers. So it’s just because of the scale. So it really is a chilling effect.

How do you rationalize those two? Well, one is that American firms are not welcome unless they’re useful to China. So many of the social networks are blocked, YouTube’s blocked, Google, after censorship that we thought was inappropriate has largely moved to Hong Kong. But they’re two things that are interesting about China that have emerged recently. Weibo, which is essentially a Twitterish replacement. And WeChat, which today you’d know as essentially a WhatsApp replacement. But what’s interesting about these is that they’re much more than what I’ve described. They are the ways in which people communicate online. And they are heavily censored under these very very arbitrary censorship laws. People are in them all the time. But I came away with the sense that the Chinese government has finally met its match. That if you over here, start talking through WeChat, which is a way of talking to your friends. But they’re linked together and then eventually it spreads to you all, and then it spreads to you all, and then it spreads to you all, and it eventually gets to 10 million people a new idea. There’s not enough prisons and jails to arrest all those people.

So we conclude and it was very clear in talking to the Chinese government that they are at least aware of this possibility of the internet. And they’re worried about the internet being used to sort of disturb the social order. We’ll see if this works. I personally believe that as long as they can grow at 7.5%, they can sort of do this. At some point, growth slows and then all of this stuff comes to fore.

Jared Cohen: Eric, you remember on a trip we took to Beijing, before the last one, we talked to a group of business people, journalists, activists. And they were very clear about one thing that, that guides Chinese society which is people don’t expect their government to be honest, but they do expect their economy to grow. And the moment the economy stops growing is the moment that the lack of honesty, the corruption, the censorship, all these things start to matter.

Eric Schmidt: Another example in that same meeting we were — I was confused because the data that was presented was that there was a very large environmental activist movement using the internet with actual physical demonstrations against projects and there were a series of such projects that had been cancelled. Right, genuine outpourings of civil unrest in China which is generally illegal. The answer, and I said to them, this is a country that spies on its citizens, they know your IP address, they know who you are, they take photographs of you, they can arbitrarily arrest you, why would a person take that risk? And the answer came back to me. They looked me straight in the eye and they said, because you’re killing my child. There is a limit, there is an absolute limit, to what you can do to a Chinese family in a one child policy. It’s interesting that the same principle did not apply to North Korea. That’s the difference between having the internet and not.

Dr. Condoleezza Rice: Yeah. And the Chinese are an example of legitimacy based on prosperity. Right? And the question with prosperity is you keep chasing it. It gets harder and harder because people’s expectations keep going up. But I wonder about the role of the technology, the internet, information. Just information. And so, for example, pollution in China, environmental degradation is a hot political topic in China and people can walk outside of their apartments or wherever and they know they can’t breathe in Shanghai or Chengdu or wherever. And for a long time the government was giving a pollution index number that clearly didn’t bear any resemblance to reality. The US Embassy started publishing a number or putting a number up but there’s also now apparently an app that you can buy that will measure the pollutants. So just the provision of information challenges the monopoly on information that an authoritarian government depends on for control and acquiescence. Do you agree with that?

Jared Cohen: And in China, this is going to be a particular challenge. So lots of people like to make the argument that China has a very sophisticated censorship apparatus, but in reality, China’s about to go through an experiment that no other country in history will ever grow through and it’ll happen just one time. Which is a billion people are going to come online in one country in the span of a decade. And you have to ask yourself the question, who are they? They are largely rural. Often in the Western province, they have no visibility into what’s going on in the various urban environments in China. And so, the reason this is game-changing is the grievance of one city has the potential to scale to a grievance across all cities. It’s that sort of visibility into other Chinese communities’ problems and challenges and grievances. And so, the challenge of these complaints and these frustrations scaling. Even the Chinese themselves have no idea what that looks like. It’s a huge wildcard for them.

The other thing that they’re doing, just to do everything they can to hedge their monopoly on information, is they’re literally trying to build spheres of cyber-influence by building out the technological infrastructure of other countries. Because they have a blurry line between public and private sector, it’s their own form of foreign assistance that no other country seems to be able to compete with.

Eric Schmidt: One of the things that we have talked about internally and I think it’s fair to say, is that there are solutions to this level of censorship, a combination of public key cryptography and a whole bunch of new inventions which we can talk about if you’re interested in, make it possible to imagine that in five years maybe a decade, it will be impossible for countries like China to keep their citizens in the dark. They would have to rely on other means of oppression. But the traditional blocking the information through a firewall or restricting access looks to us like we can stop that.

Dr. Condoleezza Rice: I’m going to ask that we get ready now for the audience participation part of the discussion, but I want to close with one last question which brings us back from China, North Korea and soon to the United States. One of the most controversial issues that we face here in the United States where we do have strong democratic institutions and the like, it’s been the question about privacy, technology, security, the fight against terrorism and the like. That has been the subject of discussion about the role of surveillance, the role of NSA, the Snowdens, the Bradley Mannings of the world. How do we think about the impact of those events on how governments may view technology, how citizens may view it, and the relationship between the two?

Eric Schmidt: Well, let’s distinguish between governments and companies, and let’s talk about governments. I worry about governments collecting large amounts of data on their citizens. And not for the reasons that all the nice civil libertarians in the room think. I worry because those large databases will be leaked. Because one of the other things that’s new, is the ability to do this large scale bulk data leaking. It used to be if you looked at the previous leaks they were essentially limited by the rate at which you could make Xerox copies, you know, of things. This is sort of a physical component. But it appears now that it’s possible to steal 1.5 plus or minus million documents with a little bit of work from Hawaii, and then put that around the world. This is really a new problem. And governments naturally collect data on their citizens. Current — your phone knows exactly where you are because the E911 services are required, by law, to record where you are.

Now thank goodness that information has not been leaked, but the fact of the matter is, the databases exist. I’m not particularly sure I think, we want that stuff leaked, generally available and so forth, let alone subpoenaed, used in courts and so forth. Now, we debated the Snowden case at some length, traditional foreign policy people, and people, especially in the East Coast, are very, very negative on Snowden. The West Coast tends to be more — more shall we say positive and the view of Snowden appears to have changed over the year. To be somewhat more positive from an initial very negative view. But the fact of the matter is, I just don’t think we want to encourage bulk data leaking, even if it had a good purpose. Who appointed Mr. Assange or Mr. Snowden to be the person who made that decision? It’s not a good way to run your society. So in the book we basically say there’s got to be some other way to deal with government misbehavior and so forth. But the other thing we say very clearly and indeed in the current issues around the NSA surveillance, I’m happy that we know that the government was doing this. I think in democracy you should roughly know what the government was doing. It’s not really a good idea for them to be collecting all this information, because somebody else will leak some other part of it.

Jared Cohen: And just to add to that, there’s a few other dimensions to this. Eric and I went — in research for the hardcover version of the book, we spent five and a half hours interviewing Julian Assange — well, he was under house arrest in the UK — trying to understand technologically what he was trying to build and what he thinks he might be able to build in 10 years. And we came to the conclusion that Eric mentioned, which is — the problem is who are these people to determine that they’re the ones anointed to make a decision about what should be public and what shouldn’t? But it’s actually even more complicated than that with bulk leaking, because there’s nobody — there’s no way any individual can read all these documents and even make an informed judgement about what kind of harm they’re going to cause.

Eric Schmidt: So we actually wrote our position in the book, and you can — you’ll read it. And from the safety of his new residence in London, Mr. Assange wrote a scathing critique of us and our view, saying that we were puppet masters for the tech elite.

Jared Cohen: And of course he leaked our interview.

Eric Schmidt: And he leaked our interview. So, which you can also read online.

Dr. Condoleezza Rice: What did you expect?

Eric Schmidt: So, my point here is I think this information is being gathered, maybe for good purposes or maybe for not. But if you’re going to gather that information, you better have a really good idea of how it’s not going to get stolen. It’s not going to get bulk data leaked. And it’s going to be protected, or maybe you shouldn’t do it at all. And I think that’s ultimately, I think that we’ve come with.

Dr. Condoleezza Rice: Let me just — but of course, it’s not just the problem with government. Lots of people collect data on you. As a matter of fact, I would dare say, Amazon probably knows more about you than the US government and they care more. All right? Because the NSA actually doesn’t care what you said to your grandmother. Amazon does, because you might buy something that you discussed with your grandmother. So, sometimes I think we have to recognize that one of the problems of your point about bulk data leaking is an important one. But democracy is always right at the edge of chaos. And it’s a disruptive system where people can say anything. It’s a disruptive system where we’re always overthrowing governments peacefully. And we depend on institutions to moderate and mediate those disruptions. And if individual citizens are deciding, rather than the institutions of government, in the case of the United States there’re three separate co-equal branches of government. Then you’ve got a chaotic environment and I think that is part of the problem with the Snowden.

Jared Cohen: Well also, and then you add the celebrity factor to this, right? I mean and that to me, if you look at the progression of leakers in the past and then Manning and then Snowden, these people are becoming more famous over time, and as we talked about backstage, it lends itself towards copycatting and I think your point about this not just being a government issue is also important. What happens the day that somebody at a major law firm in the United States decides to bulk leak a bunch of information? What are the consequences of that?

ALSO READ:   Bettina Warburg: How the Blockchain will Radically Transform the Economy (Transcript)

And then there’s another issue about what all of this means for the future of the intelligence community. One of my mentors is Frank Carlucci, who used to be deputy director of the CIA among other things. And he said to me in what must have been 10 years ago that the challenge the CIA has is it’s becoming increasingly difficult to keep a secret. And that was a decade ago. So the intelligence community has a real challenge that it has to overcome, which is how it continues to do its good work in an era where more and more there’s a risk of things getting leaked. And I don’t know the answer to that question.

Dr. Condoleezza Rice: All right. We’re now going to go to the audience participation.

Question-and-answer session

Audience: You talked about that you were worried about the country stealing your data and more worried about you stealing my data. And the point is that the government shuts down for 7 days nobody cares, Google goes down for 5 minutes the world stops. And so the thing which I would like to ask you is, I control my government. I know that I am electing this president. I don’t control the Google CEO chairman. Should there be elections for people who steal, who control our data? And how do you bring accountability –

Eric Schmidt: I was careful with my answer to say that you want to distinguish between governments and corporations. Governments have a monopoly on the use of force. They can actually send the police, they can shoot you, they can put in jail. None of those tools are available to corporations. And indeed, companies are subject to numerous laws, regulations, and so forth. Not just in the US, but everywhere else. But the ultimate reason for you not to be too worried about us, is that we depend on at some basic level, your trust. And if there were a data breach at Google, involving anyone, it would immediately become front page news all around the world and millions of people would say, screw that, we’re going to go use Bing. Seriously. So, which is clearly not a good thing.

So, ultimately, the answer to your question is in a competitive market, right? The competition, plus the regulations, plus the fear of regulations, plus the serious threat of civil and criminal penalties, which are in fact, often applied to CEOs and chairmen are your best defense.

Audience: Hi, my name is Malin, I’m a senior at Stanford. I think there’s sort of a large current of dissatisfaction with the government here in Silicon Valley, and you also spoke about technology sort of as a way around traditional foreign policy mechanisms. But in the end, Google is part of America, Google’s part of a democracy. So how do you view Google’s civic responsibility both here and abroad?

Jared Cohen: I had a very interesting mind shift that I had to experience when I went from the state department to Google, because I was used to talking about our foreign policy. And I got to Google and everybody told me there was no Google foreign policy. It’s a global company that’s defined by a mission and a set of values. And, that’s a really different way of thinking about things than when you’re in government. So, when you — part of the reason at Google we’ve taken on this mission of free expression is it’s something that makes sense for us as a company, and it’s an issue that we can get behind that might sort of inch right up to the line of being political without favoring, say, one specific government, right? And Google as a company, we stand by the free flow of information and we should stand by that everywhere from the United States to Pyongyang. But there is this miss — just because Google is headquartered in the US doesn’t mean that it is exclusively a US company.

Eric Schmidt: Google’s basic answer is we are in favor of an open and free internet for every human being on the planet. And remember that that strategy’s also self serving to our shareholders because eventually they will become Google users and eventually they’ll have monetization and eventually they’ll have credit cards and eventually they’ll have advertising. And because we have the benefit of time, we’re willing to wait for the 20 years or 30 years for those consumers to become rabid Google advertising fans. So it’s in our self interest, but it’s also the right thing to do morally.

Audience: So my name is [Joe], actually maybe, one of the victims of the technology was involved in the Egyptian Revolution. And it’s amazing what you said — it’s just you said, what’s on my — what I wanted to say is there’s a lot of responsibility to technology. I mean the revolution happened we didn’t expect. The regime is going to change. It’s been 3 years, what happened is the aftermath. I feel there is a lot of responsibility on us to actually have this work. And there’s responsibility maybe on you to help us make this work. And so, I started an NGO, I moved to one of the largest slum areas in Egypt. I actually had a visit from one of your VPs, Mohammad Gawdat, what an amazing guy! And I felt that why not we create a successful model of revolution or revolutions that are happening, or technology that makes a big change and is sustainable and continues. So it’s not a matter of maybe, as you travel around the world, but why don’t we keep on and following up, I mean if you were left out, I mean most of the revolutionaries now are in jail or they’re outside of the country. But why, I mean why some — Google that was so interested and the whole world was like with us and now they’re like, just let’s look to another part of the world. Although, there’s still much that can be done, to follow up on this and actually make this a successful story that can be replicated, in other youth who are going to take the lead and change their countries.

Jared Cohen: Yeah, I mean, our belief and we write this in the book, technology has an empowerment bias. It obviously empowers people to do extraordinary things, but when I look at the situation in Egypt, you see two challenges. A lack of institutions that can function and deliver basic goods and social services to the population and two, it’s very important if you start a revolution to make sure that revolution produces new leaders with new last names who can deliver better than the autocrat that you overthrew. And it takes time, right?

So one of the challenges, you talked about expectations, technology does raise expectations and it’s very easy for people to galvanize around the lowest common denominator which is, we don’t like an autocrat that shut down our networks, let’s get him out of power. But that’s basically the only thing that anybody agrees on. And so the — go ahead Eric.

Eric Schmidt: In the book we actually tell the story of Vodafone, and as you know, Tahrir Square, they actually shut down the internet for four and a half days, excuse me. And they sent out fake SMS messages which were so stupid, everyone figured out that they were coming from the Ministry of Propaganda, under Mubarak. When they shut down the internet, the average person in Egypt all of a sudden got a definitive message that the government was terrified. And people’s opinion shifted in favor of the protests. So the internet was a key component of the first part of the revolution, but it provided very little help in the second part, which was identifying real leaders doing the political negotiation, building the teams. I think it’s fair to say that Ukraine which has many problems unrelated to the internet, has the same problem. The guy that they didn’t like has now fled the country, they can’t agree on a new leader. But now you’ve got all of these people who are very sophisticated, very internet connected, saying, where’s my government and why haven’t you fixed everything right now? So pity the revolutionary leader, who takes this emotionally charged thing and has no time to operate. And that’s the fault of the internet.

Dr. Condoleezza Rice: Yes, if I could just on, on this point though, it’s also the case that these things really do take time. No revolution on the day after is actually successful. That’s the thing to remember. From the French Revolution, the American Revolution, no revolution is successful the next day. Because technology I think is an accelerant of underlying trends. It actually is neither a cause of what happens after, nor is it a solution to what happens after it. And so, the question is can it be part of a solution going forward? And I think that’s really the interesting question for those who promote the optimistic technological future. Let’s see. Where is the next question?

Audience: Hi, my name is Tim. I’m an undergrad as well. My question has to do with the ways we can use technology to change institutions in the western world or generally you could call good governments, right? And the question is that historically we relied on a very strong signal about public opinion every 4 years, or whatever the election cycle is. We can imagine technology is nowadays allowing decision-makers to get a very strong signal about what its populace wants quite immediately. How do you think governments can harness that technology in their decision-making process and how should we change institutions given that we have this technology as a government?

Jared Cohen: Well, I think, I mean, the natural thing that technology does to politics, say right here in the United States, is it makes it all more transparent. Now, that also comes at a cost. It generates a lot more noise. I mean I think, this isn’t an answer to your specific question, but I think there’s a fundamental problem in the American system that you can relate to technology, which is, it’s scaring all the young people and future leaders from ever wanting to go into government. Because everybody posted stupid things online, at an age before they were aware of the long term consequences. And so, there is a risk, we’re here at Stanford, smartest people in the world. I’m not just saying that because I went there.

Eric Schmidt: Speaking as a Stanford graduate.

Dr. Condoleezza Rice: I knew we’d get back at Cal sooner or later during this conversation.

Jared Cohen: But there is a responsibility that all the students here have as people who are going to graduate and go on to lead the world is there’s a responsibility to make sure it’s not led by the leftovers. And that means that there do have to be a set of brave young people who decide, you know what, maybe I — maybe there’s a risk associated with a life filled with transparency, but we’re going to be a part of the few brave souls that get out there and decide that, despite the consequences of that we want to lead.

Dr. Condoleezza Rice: Could I just, on this point though, because I want to make an argument — a provocative argument, that actually, this is a place that technology is partially to blame for the institutional gridlock that we see in Washington. And actually, it’s true in lots of democracies. But let me make the American case. I’m going to date myself now. But when I was a kid, I grew up, watching the Huntley-Brinkley Report with my parents every night. That’s how we got our news. Some people watched Walter Cronkite. Some people watched Coward K. Smith. But that’s how we got our news.

Now, and so basically, we had the same news source. Now I can go to my blog, my cable news network, my aggregation sites, and I can never actually encounter the views and opinions of somebody who thinks differently. So we balkanize information news in a way that I only have to listen to people who think like me. When you only listen to people who think like you, you think people who don’t think like you are either venal or stupid. And that then gets reflected in the polarization of politics in Washington. So, actually both the speed of technology, and its balkanization has contributed to the inability to govern. How do you react to that?

Eric Schmidt: I think we would — we completely agree with what you said, and to further support it, I’m good friends with a Republican Senator from the South. And I’m visiting him, and he had just narrowly escaped a brush with the tea party. And I said, how was the campaign? And he said that it got so bad that he, on his mirror in the morning at home, he shaves, and he wrote in, in lipstick or whatever you write on, on mirrors. There are cameras everywhere, everyone is watching you, you’re always recorded, keep your mouth shut. Okay. This is a senator. It’s not some random person. And I worry that — the way to answer your question is, what do we worry about in democracy? And I worry that there’s been a loss of what I would call deliberation, in deliberative thinking there are some problems that cannot be solved in an hour or a day, they require sophisticated political leadership, lots of time investment, lots of discussion sometimes behind closed doors. And we’re losing that opportunity. Great leaders probably emerge from that cauldron. How will the new great leaders emerge, I don’t know.

Audience: [Andrew Graida]. I’m a former colleague of Eric and Jared. I wanted just to support the point raised by Condoleezza Rice that there is, what you call in your book confirmation bias. That people tend to absorb opinions similar to their own. And we see to large extent that in Russia those days, which is another contradiction over why our population that still believes into what they see on television. Yeah, but I have a couple of different questions. You write in your books that anonymity nowadays can translate to something like irrelevance. Did your opinion evolve since then? I remember we had a lot of discussions at Google when we started Google Plus whether people should use their real names, and how do you see this in the current developments of the internet?

And second, how does Google or other companies take advantage of the recent technological progress in cryptography for example, random number generators that are working in quantum effects and quantum computers.

Eric Schmidt: So let me answer both of those quickly. There is evidence that people online when they are anonymous, say things that they would never say in front of a human being or if the name was real. So I think it’s important to allow such speech. Certainly you don’t want to ban it. But you might want to favor identified speech over unidentified speech. Because the consequence of liable and lying and so forth, you’re more likely to be truthful if you have a face and a name and a responsible, roughly speaking. So ultimately since Google views everything kind of as a ranking problem I think that’s how I would rank most of it. I think it’s important to allow anonymous speech, but it’s also important to understand that sometimes anonymous speech is not actually true because people would just sort of make stuff up and so there’s no consequence online.

On the question of cryptography. After the NSA revelations, one of the things that the Snowden documents showed, and we believe are true, the GCHQ had put a pipe essentially between our data centers and had undone what you know as the Stubby protocol. And we view this as a significant violation inside of the trust and innards of Google. So we use very complicated and very powerful encryption now to essentially make it impossible to do that. For the technical people in the room, it’s twenty forty eight bit keys with perfect forward secrecy, we rotate the keys on every transaction. Because essentially, we believe, impossible even with modern cryptographic tools to break that.

It’s possible that the NSA has other mechanisms inside but we certainly don’t know of them. And we’ve certainly done everything we can to prevent it. If the government wants information from Google, the government should follow the law and follow the FISA Court, which was set up under Patriot Act 1 and Patriot Act 2 for these purposes.

Audience: Hi there. My name is Dustin Boyer and I’m a former Google employee, so it’s actually nice to see both of you again. I’m sort of fascinated that you guys seem to be so against leaking, because it was the Snowden documents that actually showed that there was this huge issue with data centers, and that so much of Gmail had been compromised. And that the NSA had then, in turn, gone on to use that information to spy and infiltrate activist groups. So, my question to you is, if not Snowden, then who, because obviously the NSA has no incentive to reveal this type of information.

Eric Schmidt: In the book we actually talk about this at some length. So the problem here is if I say yes to the Snowden thing, what happens when there’s a copycat that isn’t as judicious about the documents that got leaked working with The Guardian and people start getting killed and seriously made endangered. And it’s not at all clear to me that you could read the million documents that you’re leaking. So to trust any individual with that kind of empowerment, is of concern. So it was certainly helpful to know about it. But it would be much better if we could have known about it without the downside dangers of the kind, that the leaking implies.

ALSO READ:   Microsoft Build 2014 Keynote Presentation Transcript - Day 1 (Full)

Audience: My name’s [Gerald], I’m an MBA at the GSB. We spoke extensively about sort of the bottom up adoption of technology, and how that affects revolution and democracy, but I was addressing as perspective on sort of more top down adoptions. Countries like Singapore, which would be fairly autocratic for a long time. But I think have done an excellent job in using technology to empower the citizens, create better citizen services and create a more open society while sort of maintaining the benefits of having some closed-door conversation. And although it’s often laughed up as being small, are those models, things that people look to replicate, at other places.

Eric Schmidt: So we spent quite a bit of time with the prime minister of Singapore and he’s interviewed for — is interviewed in the book. And he tells the story of how people react to this social information in various examples what happens between the apartment buildings in Singapore. Singapore’s done a fantastic job of building a knowledge society and connect everything. And they have, for what is largely a one party democracy, they have an unusually open view of the internet. They only block, according to them, 100 sites. And they do that in order to point out to you that they can block it. So it may be that culturally because it’s largely Mandarin and Chinese, it’s a small country, and it’ s of course physically smaller, they get a different effect that wouldn’t necessarily scale to hundreds of millions of people of different religions and interactions. And I caution against generalizing from the sort of wonderful stories of the Netherlands, and Singapore and so forth which are homogeneous, small, ethnic and largely well run countries.

Dr. Condoleezza Rice: Jared, do you want to add anything?

Jared Cohen: Yeah, Eric mentioned, it’s also worth noting that Singapore is one of only two countries in the world whose leader is a computer scientist, Estonia is the other. But Eric mentioned a story that the prime minister shared with us. It’s actually a story about curry. And there’s a message in the story that I’ll get to in a second. Basically, there was a dispute between two people who shared a hallway, in an apartment complex over the smell of curry in the hallways. In a typical Singaporean fashion they brought a mediator in to negotiate and everybody was fine, until the mediator went public with her story and political opportunists, decided to seize this as a chance to talk about how foreigners were coming in and taking Singaporean jobs, go figure.

A bunch of people got together online and declared a national day of curry cooking. And the prime minister, to his credit, acknowledged that this really scared them. And he said that they started to see people, that they started to see dissident build online. And he acknowledged that they overreacted to it, and the overreaction to what was in fact a very small number of people really having good fun with something, ended up putting people in the streets. And the lesson from this is there’s a dilemma that autocratic leaders will face in the future which is having to distinguish between what’s real and what’s noise. And many autocrats in the future, at some point, they’ll overreact to that noise and create a disturbance that ultimately they can’t control.

Audience: I’m Theo Milonopoulos. I’m a Stanford alum and PhD student in political science. Dr. Schmidt, I wanted reinforce a point that Dr. Rice made about the motivations that the NSA and companies like yours might have in collecting metadata. It seems to me that the NSA, if it could get out of the business of data collection it would do it in a heartbeat. In the sense that the larger and larger amounts of information that it collects makes its job actually more difficult in the sense that it becomes finding a needle in a much larger haystack. So, it seems to me that, that the NSA would love not to have to collect this data because, but the terrorist threats keep coming. And that the day on September 12th, the question won’t be, why didn’t the NSA do more to protect my privacy. It will be why didn’t it find the needle in the haystack?

My second point —

Eric Schmidt: Can I actually respond to that?

Audience: Sure.

Eric Schmidt: In the book we actually make your argument in the following context. We say that in the fight between hawks and doves, the hawks always win. Because the downside risk of a small terrorist act, and if you will, the loss of diversity and fear and so forth that comes with everyone being slightly worried about being surveillance. There’s no — the doves don’t have a standing. It ultimately is an asymmetric outcome. I disagree a little bit with the way you phrased that, just as a matter of computer science. Modern algorithms are capable of doing extraordinarily subtle things with large data collections. So, for example, the metadata that consists of your – the phone numbers you call, is highly, highly interesting to a whole bunch of computer scientists who want to figure out who you are, who you’re associated with, and so forth. So it’s highly valuable to the NSA which is why they did it.

I’m on the task force for the White House trying to figure out what to do with this data by the way, and the President has asked that it not be stored by the NSA but be stored somewhere else. Well, there is not agreement on where that information should be stored for the reasons that I outlined. Go ahead.

Audience: Sure. And just because we talked, touched on the question about how to get to a more elevated political discussion on these issues. I don’t believe like, the labels of hawks and doves, because I don’t necessarily think that that’s the right axis to think about it. I just think, the question is what level of risk are we willing to tolerate in American society. And that’s something that should be debated, but I’m not entirely sure how that debate can happen. We shouldn’t have to, I think, rely, and I think you would agree with me. That we shouldn’t have to rely on the compromising of our sources and methods, analysts out there that are trying to defend the United States, in doing so. But let me get to my second —

Dr. Condoleezza Rice: Let’s — we are going to have to have somebody else make a comment, so, yeah right. Do you want to respond to the last point at all?

Jared Cohen: Well I mean I think we agree that there’s a need for a debate and we’ve said whether one — regardless of what one’s position is on Snowden, it’s useful to have this conversation about transparency and until recently we weren’t able to talk about what percentage of the overall requests we get from government are related to national security. So, there actually has been progress on this.

Eric Schmidt: An example would be that it was illegal for us to say the number of national security requests we got. The government just changed this rule so I can now tell you that we get less than 10,000 per year. Which given that we have a couple billion users is actually a calming number for most people. So I disagree, I think I disagree with all of your points. So, in the spirit of — no, no, no, but just to answer, I think that it’s in a democracy, we write this in the book, democracies will have these debates, and they’ll find the point, right, between our different views. In autocracies there is no such debate.

Audience: Hi my name is Alex, I’m a third year PhD student in engineering here at Stanford. I have a question about the potential for crowd, crowd-source funding to inflame violent groups and, thinking specifically about an article I read recently concerning Syria which you brought up, which were certain very radical groups that are fighting Assad but the U.S. doesn’t want to gain power, and that are being funded through crowd-sourcing in Arab — other Arab nations, Saudi Arabia was one of the ones that was mentioned. And I think the Irish conflict with the IRA, and how money came from America back then to support activities, some of which were terrorist activities. But it was very hard to do that back then, and now it’s getting much easier for someone sitting in their house to support a radical organization. What do you think about this?

Jared Cohen: That’s a really excellent question and we touch on this in the book. And I think I go back to Syria as an important context. So, it’s not just about crowd-sourcing money for groups engaged in hostile activities, it’s also about crowd-sourcing people. So Syria, in addition to fighting a physical civil war, it’s also fighting a cyber war which is being led by the Syrian electronic army. What the Syrian electronic army has started doing is crowd-sourcing foreign fighters, not with guns, but from the former Soviet Union to come and fight the cyber war. And it’s interesting, there’s about 15,000 foreign fighters physically fighting in Syria, about a third of them come from Europe, and about a third of those from Europe are converts, which is sort of an interesting data point. But, it’s a totally – Syria is the first example that I can think of where you have software engineers that are sort of added to the equation, fighting a totally different type of war on top of the one that we’re familiar with. And if you think about Iraq for instance, obviously we’re familiar with sectarian violence that’s played out there and played out in Syria. When you think about sectarian violence and then you think about something we’re all familiar with, which is cyber-bullying it’s really the sort of the next stage of this, when cyber-bullying becomes better organized, better funded, and in some cases state sponsored, which gets to your point. You know, if one demographic or one sector, one group is disproportionately able to garner resources to terrorize a rival population both in the physical world and also online, it’s doubly as damaging.

Dr. Condoleezza Rice: I’m going to — we are going to run out of time very quickly here, but I want to close the conversation by posing a question that was actually flashed at me from the audience, a very interesting question. And then to just give you each a chance to say a few words that you’d most like us to remember about this evening and about your book. But, someone asked me, do you have a view on bitcoin, and bitcoin —

Eric Schmidt: Jared, and so what happened to those bitcoins you were buying?

Jared Cohen: I think this was your idea.

Dr. Condoleezza Rice: So how about regulation? I mean we talk about a free and open Internet, but there are certain things that are going to have to be regulated, and if ever there was something that we expect governments and central banks to control its currency. So how about this impact of this new idea?

Jared Cohen: I think a lot of people are obviously talking about Bitcoin right now and it is right to talk about it as cryptocurrency because Bitcoin is just one example of this. And while the focus is on cryptocurrency, it’s also important we talk about the digital wallets in all of this. Because let’s say even if you have a regulated cryptocurrency, or an unregulated one, without a digital wallet, it’s ultimately a flawed ecosystem and this is what happened in Canada. The Canadians tried to create their own regulated cryptocurrency called Mintchip, which was pegged to the Canadian dollar. But then all the wallets started getting hacked and people’s money got stolen. So they, ultimately, decided to sunset it. But I think the challenge is, there’s a small number of people that want to engage in an unregulated, economic system. And there’s no shortage of examples of people that have been trying to use BitCoins for money laundering, for fraud. And ultimately, without regulation, a lot of this is kept in check. But the inevitability of cryptocurrency is something that I think we should all agree, is likely. But whether BitCoin is the model, is a different question.

Eric Schmidt: I should say that the architecture that BitCoin represents is of enormous computer science interest. The gentleman who invented it, who is now thought to be somebody who lives in Europe and he never really identified himself came up with a way to make digital copies of things that cannot be further copied. And that is useful in many, many areas of the future. So even if Bitcoins become very controversial for the reasons that you’re identifying, the underlying accomplishment is a new, is a very interesting new thing. And you’ll encounter it. It’s essentially a way that copies are regulated and new copies are not allowed, that’s pretty interesting.

Dr. Condoleezza Rice: All right. As a closing point, what would you like us most to remember about this evening and also about your book? What should we take from this conversation? Do you want to start?

Jared Cohen: And before I say I would like to just personally thank you again as well and it really is amazing to be back here at Stanford. It’s also very — it’s a unique situation to sit in between my former boss and my current boss. It feels like a lot of accountability and l almost feel like I am going to get graded or hopefully compensated or something. Something at some point. But I spent — I still consider myself a young person but I’ve spent my short professional life working really hard at figuring out what we do about repressive regime. And in the short time that I’ve been doing this over the past decade, it’s become increasingly possible for all of us to play a role in this. And so the old model, is the government was autocratic and repressive and we basically had to rely entirely on states to change the systems. And the reality is because of computer science, because 5 billion new people are going to be connected and because it really is still a wild card as to whether or not the repression that exists in the physical world will spill over into the online world. It’s optimism but with a caveat that it’s a challenge to all of us and I look out at this audience and it’s the first generation that is going to be going into leadership roles, understanding both technology and geopolitics. And so, not only is there sort of short window of time to figure out how we end repressive censorship and repressive talk recede, but the timing coincides with probably the people who are best positioned to deal with this.

Eric Schmidt: Condi, I think it’s a great honor for both of us to be on the same stage with you. And with the sort of you as the exemplar of service to our country and to the world. But I also want to thank you for taking young Jared and turning him into my partner here. We’ve learned a lot together, and in putting the book together, what we try to do is to lay out what’ll happen over not just 1 year but over a 5 or 10 year period. And so far, in the last year, pretty much everything we talked about has occurred in one form or another, although not necessarily by name. So, what I conclude is, we have roughly the right architecture, but we now don’t know exactly who the players will be. And that will be a lot up to the people who discuss this, the people who decide, who debate it, and who understand the consequences of it. The issues around empowerment, both leaking encryption, state actors, government and so on and so forth, they’re not going to go away, these are new problems. And I would argue that and, and what’s nice about sort of finishing the book tour more or less here in California over the last year we’re sort of done with this segment of work. You guys can read and see what you think, is that the people who grapple with these issues it’s now your turn. Government leaders, student leaders, business leaders and so forth, to figure out what you’re going to do into this new digital age, right. And there will be huge wins. Huge empowerment, huge improvements ahead of us if we do the right things.

I still come back to my core bias which is that I’m an optimist about people. And I think the empowerment bias that Jared described, it’s such a great story. And I would never want society to somehow stop that or slow it down. I think both of us have decided to spend our lives working on this problem.

Dr. Condoleezza Rice: And we’re glad that you have because you are both exemplars of trying to put ideas into action and I want to thank you very much for a wonderful conversation this evening. And I want to encourage all of you, if you have not, to buy the New Digital Age: Reshaping the Future of People, Nations and Business.

Please follow and like us:
Pin Share20