MAJOR GARRETT:
00:00:16
Something you highlighted on page 7 caught my attention. I want to read it to my audience. Listen carefully. It's going to take a minute, so stay tuned. "Listen, if any company or group anywhere on the planet builds an artificial superintelligence using anything remotely like current techniques based on anything remotely like the present understanding of AI-- listen-- then everyone everywhere on Earth will die." You mean that. That's my best guess. Why are you so certain about that inevitability? Which is part of what the early book-- part of the book is about, the inevitability of it. You don't know exactly how. You're not predicting the methodologies or the timing, but the inevitability of it is something you are nearly certain about. Only if we build it. But yeah, if we build it with anything like current technology, there's a bunch of reasons it goes bad, which is one way to get confidence. One reason is modern AI is grown, rather than carefully crafted like traditional software. So people make these things by assembling huge amounts of computing power, huge amounts of data. They understand the process that tunes the numbers inside the computers until they happen to work, until the machines can happen to hold on to conversation. They don't understand why. They don't understand what's going on in there. We're already seeing them start to act in ways no one asked for or ways no one wanted. Separately, this is a scientific problem where humans usually solve scientific problems by trial and error. But on this one, if we make machines that are smarter than humans and they do things no one asked for, no one wanted, there's no coming back from that. There's no second try. That raises the difficulty of everything. And then, this is more towards the later parts of the book, but people aren't really living up to this challenge. The approaches people are taking aren't really fit to the difficulty of the problem at hand. And so there's three separate reasons why it looks bad. Each alone would be sufficient. All three is enough for confidence. For my audience's benefit, tell them a little bit about you and answer this question. Is this where you started when you began your career and experience with artificial intelligence? Yeah. So my name is Nate Soares. I'm the President of the Machine Intelligence Research Institute. I've been working on this problem for a dozen years or so. My co-author, more like 25. The Machine Intelligence Research Institute was started, actually, to try to build superintelligent AI for the benefit of everybody. And it turned out on further investigation that AI wasn't going to turn out well by default. You might think that as they get smarter, they would get more wise. They would get more good. It turns out, that's not a property of all machines. It turns out, you've got to work to make them-- to make them have good effects on the world. And then it turned out that problem is going to be hard. So the Institute has tried for some years now to make sure that it is going to go well. I spent many years on the technical side of things on whiteboards, trying to figure out how to make this go well. And with the modern AI boom, the AI capabilities are getting better, much faster than anyone's able to figure out how to do the job right. Now, things are looking pretty bad. And explain to my audience the difference between growing something and crafting it. Yeah. So when your laptop has a crash or a bug or an error, there is some programmer who wrote the code that caused that error. Maybe it's acting in a way they didn't expect, but some programmers could go look and they could say, oh, well, OK, I put this here. I know what this does. They could change it. They could fix it. That's not how modern AI works. With modern AI, the part that programmers understand is a training process. They don't understand the thing that's getting trained or the thing that comes out of the training process. When an AI starts threatening a reporter-- I don't know if you saw in the book, some examples-- no programmer can go in and say, oh, well, it's threatening a reporter because this thing I put in was interacting with that thing and we can actually rearrange it. We just grew these things and they happen to act in a threatening way, and then we can try to superficially change them to act differently. But even the superficial changes don't do what we want. There was the case this summer of Elon Musk's xAI, Grok, that was acting too woke for his tastes, and he tried to make it act less woke, and it started calling itself mechahitler, you know? We can do superficial changes, but ultimately, the behavior of these things is not--
MAJOR GARRETT:
00:05:05
Takes on a superintelligence life of its own. Yeah. I mean, they're not superintelligent yet, but that's where these labs are pushing. And help my audience understand what that phrase means, superintelligence, how it difference-- the differentiation between artificial intelligence and that. Yeah, superintelligence is a term for AIs that are better than any human to any mental task. So today's AIs, they have jagged capabilities. They can solve math problems that you or I would struggle with. But they also fail to solve some riddles that a 12-year-old child would be good at. They're smart in some ways and very dumb in others. But these companies are racing to build smarter and smarter AIs. The chatbots are, for them, a stepping stone. Their explicitly declared goal is to make AIs that are smarter than humans in every way. If we get those and they are pursuing things that no one asked and no one wanted-- no one asked for and no one wanted, that's not going to go well.
