QA:1

Carlos Espinal: Hi everyone, welcome to ‘This Much I Know,’ the Seedcamp podcast with me, your host Carlos Espinal bringing you the inside story from founders, investors, and leading tech voices.

Tune in to hear from the people who built businesses and products scaled globally, failed fantastically, and learned massively.

Welcome, everyone!

On today’s podcast, we have Byron Reese, the author of a new book called The Fourth Age: Smart Robots, Conscious Computers, and the Future of Humanity.

Not only is Byron an author, but he’s also the CEO of publisher GigaOm, and he’s also been a founder of several high-tech companies, but I won’t steal his thunder by saying every great thing he’s done.

I want to hear from the man himself. So welcome, Byron.

Byron Reese: Thank you so much for having me. I’m so glad to be here.

QA:2

Excellent. Well, I think I mentioned this before: one of the key things that we like to do in this podcast is getting to the origins of the person; in this case, the origins of the author.

Where did you start your career and what did you study in college?

I grew up on a farm in east Texas, a small farm. And when I left high school I went to Rice University, which is in Houston. And I studied Economics and Business, a pretty standard general thing to study. When I graduated, I realized that it seemed to me that like every generation had… something that was ‘it’ at that time, the Zeitgeist of that time, and I knew I wanted to get into technology. I’d always been a tinkerer, I built my first computer, blah, blah, blah, all of that normal kind of nerdy stuff that I did.

But I knew I wanted to get into technology. So, I ended up moving out to the Bay Area and that was in the early 90s, and I worked for a technology company and that one was successful, and we sold it and it was good. And I worked for another technology company, got an idea and spun out a company, and raised the financing for that. And we sold that company. And then I started another one and after 7 hard years, we sold that one to a company and it went public and so forth. So, from my mother’s perspective, I can’t seem to hold a job; but from another view, it’s kind of like the thing of our time instead. We’re in this industry that changes so rapidly. There are more opportunities, which always come along and I find that whole feeling intoxicating.

QA:3

That’s great. That’s a very illustrious career with that many companies having been built and sold. And now you’re running GigaOm. 

Do you want to share a little bit for people who may not be as familiar with GigaOm and what it is and what you do?

Certainly. And I hasten to add that I’ve been fortunate that I’ve never had a failure in any of my companies, but they’ve always had harder times. They’ve always had these great periods of like, ‘Boy, I don’t know how we’re going to pull this through,’ and they always end up [okay]. I think tenacity is a great trait in the startup world because they’re all very hard. And I don’t feel like I figured it all out or anything. Everyone is a struggle.

GigaOm is a technology research company. So, if you’re familiar with companies like Forrester or Gartner or those kinds of companies, what we are is a company that tries to help enterprises, help businesses deal with all of the rapidly changing technology that happens. So, you can imagine if you’re a CIO of a large company and there are so many technologies, and it all moves so quickly and how does anybody keep up with all of that? And so, what we have are a bunch of analysts who are each subject matter experts in some area, and we produce reports that try to orient somebody in this world we’re in and say ‘These kinds of solutions work here, and these work there’ and so forth.

And that’s GigaOm’s mission. It’s a big, big challenge because you can never rest. Big new companies I find almost every day that I’ve never even heard of and I think, ‘How did I miss this?’ and you have to dive into that, and so it’s a relentless, nonstop effort to stay current on these technologies.

QA:4

On that note, one of the things that describe you on your LinkedIn page is the word ‘futurist.’

Do you want to walk us through what that means in the context of a label and how does the futurist really look at industries and how they change?

Well, it’s a lower case ‘f’ futurist, so anybody who seriously thinks about how the future might unfold, is to one degree or another, a futurist. I think what makes it into a discipline is to try to understand how change itself happens, how does technology drives changes, and to do that, you almost by definition, have to be a historian as well. And so, I think to be a futurist is to be deliberate and reflective on how it is that we came from where we were, in savagery and low tech and all of that, to this world we are in today and can you in fact look forward.

The interesting thing about the future is it always progresses very neatly and linearly until it doesn’t until something comes along so profound that it changes it. And that’s why you hear all of these things about one prediction in the 19th Century was that, by some year in the future, London would be unnavigable because of all the horse manure or the number of horses that would be needed to support the population, and that maybe would have happened, except you had the car, and like that. So, everything’s a straight-line, until one day it isn’t. And I think the challenge of the futurist is to figure out ‘When does it [move in] a line and when is it a hockey stick?’

QA:5

So, on that definition of line versus hockey stick, your background as having been CEO of various companies, a couple of which were media-centric, what is it that drew you to artificial intelligence specifically to futurize on?

Well, that is a fantastic question. Artificial intelligence is, first of all, a technology that people widely differ on its impact, and that’s usually like a marker that something may be going on there.

There are people who think it’s just oversold hype.

It’s just data mining, big data renamed.

It’s just the tool for raising money better.

Then there are people who say this is going to be the end of humanity, as we know it.

And philosophically the idea that a machine can think, maybe, is a fantastically interesting one, because we know that when you can teach a machine to do something, you can usually double and double and double and double and double its ability to do that over time.

And if you could ever get it to reason, and then it could double and double and double and double, well that could potentially be very interesting.

Humans only evolve, computers are able to evolve kind of at the speed of light, they get better and humans evolve at the speed of life.  It takes generations.  And so, if a machine can think, a question famously posed by Alan Turing, if a machine could think then that could potentially be a game-changer. Likewise, I have a similar fascination for robots because it’s a machine that can act, that can move and can interact physically in the world. And I got to thinking what would happen, what is it a human in a world where machines can think better and act better, then what are we? What is uniquely human at that point?

And so, when you start asking those kinds of questions about technology, that gets very interesting. You can take something like air conditioning and you can say, wow, air conditioning. Think of the impact that had. It meant that in the evening’s people wouldn’t… in warm areas, people don’t go out on their front porch anymore. They close the house up and air condition it, and therefore they have less interaction with their neighbors. And you can take some technology as simple as that and say that had all these ripples throughout the world.

The discovery of the new world ended the Italian Renaissance effectively because it changed the focus of Europe in a whole different direction. So, when those sorts of things had those kinds of ripples through history, you can only imagine what if the machine could think like that’s a big deal. Twenty-five years ago, we made the first browser, the Mosaic browser, and if you had an enormous amount of foresight and somebody said to you, in 25 years, 2 billion people are going to be using this, what do your think’s going to happen?

If you had an enormous amount of foresight, you might’ve said, well, the Yellow Pages are going to have it rough and the newspapers are, and travel agents are, and stockbrokers are going to have a hard time, and you would have been right about everything, but nobody would have guessed there would be Google, or eBay, or Etsy, or Airbnb, or Amazon, or $25 trillion worth of a million new companies.  And all that was, was computers being able to talk to each other. Imagine if they could think. That is a big question.

QA:6

You’re right and I think that there is…I was joking and I said ‘Tinder’ in the background just because that’s a social transformation. Not even like a utility, but rather the social expectation of where certain things happen that was brought about that. So, you’re right… and we’re going to get into some of those [new platforms] as we review your book. In order to do that, let’s go through the table of contents. So, for those of you that don’t have the book yet, because hopefully, you will after this chat, the book is broken up into five parts and in some ways, these parts are arguably chronological in their stage of development.

The first one I would label as the historical, and it’s broken out into the fourth ages that we’ve had as humans, the first age being language and fire, the second one being agriculture and cities, the third one being writing and wheels, and the fourth one being the one that we’re currently in, which is robots and AI. And we’re left with three questions, which are: what is the composition of the universe, what are we, and what is yourself? And those are big, deep philosophical ones that will manifest themselves in the book a little bit later as we get into consciousness.

Part two of the book is about narrow AI and robots. Arguably I would say this is where we are today, and Seedcamp as an investor in AI companies has broadly invested in narrow AI through different companies. And this is I think the cutting edge of AI, as far as we understand it. Part three in the book covers artificial general intelligence, which is everything we’ve always wanted to see, where science fiction represents quite well, everything from that movie AI, with the little robot boy, to Bicentennial Man with Robin Williams, and sort of the ethical implications of that.

Then part four of the book is computer consciousness, which is a huge debate, because as Byron articulates in the book, there’s a whole debate on what is consciousness and there’s a distinction between a monist and a dualist and how they experience consciousness and how they define it. And hopefully, Byron will walk us through that in more detail. And lastly, the road from here, it is the future, as far as we can see it in the futurist portion of the book, I mean part three, four and five are all futurist portions of the book, but this one is where I think, Byron, you go to the ‘nth’ degree possible with a few exceptions. So maybe we can kick off with your commentary on why you have broken up the book into these five parts.

Well you’re right that they’re chronological, and you may have noticed each one opens with what you could call a parable, and the parables themselves are chronological as well. The first one is about Prometheus and it’s about technology, and about how the technology changed and all the rest. And like you said, that’s where you want to kind of lay the groundwork of the last 100,000 years and that’s why it’s named something like ‘the road to here,’ it’s like how we got to where we are today.

And then I think there are three big questions that everywhere I go I hear one variant of them or another. The first one is around narrow AI and like you said, it’s a real technology that’s going to impact us, what’s it going to do with jobs, what’s this going to do in warfare, what will it do with income? All of these things we are certainly going to deal with. And then we’re unfortunate with the term ‘artificial intelligence,’ because it can mean many different things, but one is that it can be narrow AI, it can be a Nest thermometer that can adjust the temperature, but it can also be Commander Data of Star Trek. It can be C-3PO out of Star Wars. It can be something as versatile as a human and fortunately those two things share the same name, but they’re different technologies, so it has to kind of be drawn out on its own, and to say, “Is this very different thing that shares the same name likely? possible? What are its implications and whatnot?”

Interestingly, of the people who believe we’re going to build [an AGI] very immensely and when, some say as soon as five years, and some say as long away as five hundred. And that’s very telling that these people had such wide viewpoints on when we’ll get it. And then to people who believe we’re going to build one, the question then becomes, ‘well is it alive? Can it feel pain?  Does it experience the world? And therefore, by that basis does it have rights?’ And if it does, does that mean you can no longer order it to plunge your toilet when it gets stopped up, because all you’ve made is a sentient being that you can control, and is that possible?

And why is it that we don’t even know this? The only real thing any of us know is our own consciousness and we don’t even know where that comes about. And then finally the book starts 100,000 years ago. I wanted to look 100,000 years out or something like that. I wanted to start thinking about, no matter how these other issues shake out, what is the long trajectory of the human race? Like how did we get here and what does that tell us about where we’re going? Is human history a story of things getting better or things getting worse, and how do they get better or worse and all of the rest. So that was a structure that I made for the book before I wrote a single word.

QA:7

Yeah, and it makes sense. Maybe for the sake of not stealing the thunder of those that want to read it, we’ll skip a few of those, but before we go straight into questions about the book itself, maybe you can explain who you want this book to be read by. Who is the customer?

There are two customers for the book. The first is people who are in the orbit of technology one way or the other like it’s their job or their day-to-day, and these questions are things they deal with and think about constantly. The value of the book, the value prop of the book is that it never actually tells you what I think on any of these issues. Now, let me clarify that ever so slightly because the book isn’t just another guy with another opinion telling you what I think is going to happen. That isn’t what I was writing it for at all.

What I was really intrigued by is how people have so many different views on what’s going to happen. Like with the jobs question, which I’m sure we’ll come to. Are we going to have universal unemployment or are we going to have too few humans? These are very different outcomes all by very technical-minded informed people. So, what I’ve written or tried to write is a guidebook that says I will help you get to the bottom of all the assumptions underlying these opinions and do so in a way that you can take your own values, your own beliefs, and project them onto these issues and have a lot of clarity. So, it’s a book about how to get organized and understand why the debate exists about these things.

And then the second group are people who, just see headlines every now and then where Elon Musk says, “Hey, I hope we’re not just the boot loaders for the AI, but it seems to be the case,” or “There’s very little chance we’re going to survive this.” And Stephen Hawking would say, “This may be the last invention we’re permitted to make.” Bill Gates says he’s worried about AI as well. And the people who see these headlines, they’re bound to think, “Wow, if Bill Gates and Elon Musk and Stephen Hawking are worried about this, then I guess I should be worried as well.” Just on the basis of that, there’s a lot of fear and angst about these technologies.

The book actually isn’t about technology. It’s about how much you believe and what that means for your beliefs about technology. And so, I think after reading the book, you may still be afraid of AI, you may not, but you will be able to say, ‘I know that why Elon Musk, or whoever thinks what they think. It isn’t that they know something I don’t know, they don’t have some special knowledge I don’t have, it’s that they believe something. They believe something very specific about what people are, what the brain is.  They have a certain view of the world as completely mechanistic and all these other things.’ You may agree with them, you may not, but I tried to get at all of the assumptions that live underneath those headlines you see. And so why would Stephen Hawking say that, why would he? Well, there are certain assumptions that you would have to believe to come to that same conclusion.

QA:8

Do you believe that’s the main reason that very intelligent people will disagree on with respect to how optimistic they are about what artificial intelligence will do? You mentioned Elon Musk who is pretty pessimistic about what AI might do, whereas there are others like Mark Zuckerberg from Facebook, who is pretty optimistic, comparatively speaking. Do you think it’s this different account of what we are, that’s explaining the difference?

Absolutely. The basic rules that govern the universe and what our self is, what is that voice you hear in your head?

QA:9

The three big questions.

Exactly.  I think the answer to all these questions boil down to those three questions, which as I pointed out, are very old questions. They go back as far as we have writing, and presumably therefore they go back before that, way beyond that.

QA:10

So we’ll try to answer some of those questions and maybe I can prod you. I know that you’ve mentioned in the past that you’re not necessarily expressing your specific views, you’re just laying out the groundwork for people to have a debate, but maybe we can tease some of your opinions.

I make no effort to hide them. I have beliefs about all those questions as well, and I’m happy to share them, but the reason they don’t have a place in the book is: it doesn’t matter whether I think I’m a machine or not. Who cares whether I think I’m a machine? The reader already has an opinion of whether a human being is a machine. The fact that I’m just one more person who says ‘yay’ or ‘nay,’ that doesn’t have any bearing on the book.

QA:11

True. Although, in all fairness, you are a highly qualified person to give an opinion.

I know, but to your point, if Elon Musk says one thing and Mark Zuckerberg says another, and they’re diametrically opposed, they are both eminently qualified to have an opinion and so these people who are eminently qualified to have opinions have no consensus, and that means something.

QA:12

That does mean something. So, one thing I would like to comment about the general spirit of your book, is that I generally felt like the book was built from a position of optimism. Even towards the very end of the book, towards the 100,000 years in the future, there was always this underlying tone of, we will be better off because of this entire revolution, no matter how it plays out versus not. And I think that maybe I can tease out of you that fact that you are telegraphing your view on ‘what are we?’ Effectively, are we a benevolent race in a benevolent existence, or are we something that’s more destructive in nature? So, I don’t know if you would agree with that statement about the spirit of the book or whether…

Absolutely. I am unequivocally, undeniably optimistic about the future, for a very simple reason, which is, there was a time in the past, maybe 70,000 years ago, that humans were down to something like maybe 1000 breeding pairs. We were an endangered species and we were one epidemic, one famine, one away from total annihilation and somehow, we got past that. And then 10,000 years ago, we got agriculture and we learned to regularly produce food, but it took us 90 percent of our people for 10,000 years to make our food.

But then we learned a trick and the trick is technology because what technology does is it multiplies what you are able to do. And what we saw is that all of a sudden, it didn’t take 90 percent, 80 percent, 70, 60, all the way down, in the West to 2 percent. And furthermore, we learned all of these other tricks we could do with technology. It’s almost magic that what it does is it multiplies human ability. And we know of no upward limit of what technology can do and therefore, there is no end to how it can multiply what we can do.

And so, one has to ask the question, “Are we on balance going to use that for good or ill?” And the answer obviously is for good. I know maybe it doesn’t seem obvious if you caught the news this morning, but the simple fact of the matter is by any standard you choose today, life is better than it was in the past, by that same standard anywhere in the world. And so, we have an unending story of 10,000 years of human progress.

And what has marred humanity for the longest time is the concept of scarcity.  There was never enough good stuff for everybody, not enough food, not enough medicine, not enough education, not enough leisure, and technology lets us overcome scarcity. And so, I think if you keep that at the core, that on balance, there have been more people who wanted to build than destroy, we know that, because we have been building for 10,000 years. That on balance, on the net, we use technology for good on the net, always, without fail.

QA:13

I’d be interested to know the limits to your optimism there. Is your optimism probabilistic? Do you assign, say a 90 percent chance to the idea that technology and AI will be on balance, good for humans? Or do you think it’s pretty precarious, there’s maybe a 10 percent chance, 20 percent chance that that might be a point where if we fail to institute the right sort of arrangements, it might be bad? How would you sort of describe your optimism in that sense?

I find it hard to find historic cases where technology came along that magnified what people were able to do and that was bad for us. If in fact, artificial intelligence makes everybody effectively smarter, it’s really hard to spin that into a bad thing. If you think that’s a bad thing, then one would advocate that maybe it would be great if tomorrow everybody woke up with 10 fewer IQ points. I can’t construct that in my mind.

And what artificial intelligence is, is it’s a collective memory of the planet. We take data from all these people’s life experiences and we learn from that data, and so to somehow say that’s going to end up badly, is to say ignorance is better than knowledge. It’s to say that, yeah, now that we have a collective memory of the planet, things are going to get worse. If you believe that, then it would be great if everybody forgot everything they know tomorrow. And so, to me, the antithetical position that somehow making everybody smarter, remembering our mistakes better, all of these other things can somehow lead to a bad result…I think is…I shall politely say, unproven in the extreme.

You see, I believe that people are inherently…we have evolved to be by default, extremely cautious. Somebody said it’s much better to mistake a rock for a bear and to run away from it, than it is to mistake a bear for rock and just stand there. So, we are a skittish people and our skittishness has served us well. But what happens is it means anytime you’re born with some bias, some cognitive bias, and we’re born I think with one of fear, it does one well to be aware of that and to say, “I know I’m born this way. I know that for 10,000 years things have gotten better, but tomorrow they might just be worse.” We come by that honestly, it served us well in the past, but that doesn’t mean it’s not wrong.

QA:14

All right, well if we take that and use that as a sort of a veneer for the rest of the conversation, let’s move into the narrow AI portion of your book. We can go into the whole variance of whether robots are going to take all of our jobs, some of our jobs, or none of our jobs and we can kind of explore that.

I know that you’ve covered that in other interviews, and one of the things that maybe we also should cover is how we train our AI systems in this narrow era. How we can inadvertently create issues for ourselves by having old data sets that represent social norms that have changed and therefore skew things in the wrong way, and inherently create momentum for machines to believe and make wrong conclusions of us, even though we as humans might be able to derive that out of contextual relevance at some point, but is no longer. Maybe you can just kick off that whole section with commentary on that.

So, that is certainly a real problem. You see when you take a data set and let’s say the data is 100 percent accurate and you come up with some conclusion about it, it takes on a halo of, ‘well that’s just the facts, that’s just how things are, that’s just the truth.’ And in a sense, it is just the truth, and AI is only going to come to conclusions based on like you said, the data that it’s trained on. You see, the interesting thing about artificial intelligence is it has a philosophical assumption behind it, and it is that the future is like the past, and for many things that is true. A cat tomorrow looks like a cat today and so you can take a bunch of cats from yesterday, or a week ago, or a month or a year and you can train it and it’s going to be correct. A cell phone tomorrow doesn’t look like a cell phone ten years ago though, and so if you took a bunch of photos of cell phones from 10 years ago, trained an AI, it’s going to be fabulously wrong.  And so, you hit the nail on the head.

The onus is on us to make sure that whatever we are teaching is a truth that will be true tomorrow, and that is a real concern. There is no machine that can kind of ‘sanity check’ that for you, that you tell the machine, “This is the truth, now, tell me about tomorrow,” but people have to get very good at that. Luckily there’s a lot of awareness around this issue, like people who assemble large datasets, are aware of data has a ‘best-by date that varies widely. For how to play a game of chess, it’s hundreds of years. That hasn’t changed.  If it’s what a cell phone looks like, it’s a year. So the trick is to just be very cognizant of the data you’re using.

I find the people who are in this industry are very reflective about these kinds of things, and this gives me a lot of encouragement. There have been times in the past where people associated with new technology had a keen sense that it was something very serious, like the Manhattan project in the United States in World War II, or the computers that were built in the United Kingdom in that same period.  They realized they were doing something of import, and they were very reflective about it, even in that time. And I find that to be the case with people in AI today.

QA:15

I think that generally speaking, a lot of the companies that we’ve invested in this sector and in this stage of effectively narrow-based AI, as you said, are going through and thinking through it. But what’s interesting is that I’ve noticed that there is a limit to what we can teach as metadata to data for machine learning algorithms to learn and evolve by themselves. So, the age-old argument is that you can’t build artificial general intelligence. You have to grow it. You have to nurture it. And it’s done over time. And part of the challenge of nurturing or growing something is knowing what pieces of input to give it. 

Now, if you use children as the best approximation of what we do, there’s a lot of built-in features, including curiosity and a desire to self-preserve and all these things that then enable the acquisition of metadata, which then justifies and rewrites existing data as either valid or invalid, to use your cell phone example. How do you see us being able to tackle that when we’re inherently flawed in our ability to add metadata to existing data? Are we going to effectively never be able to make it to artificial general intelligence because of our inability to add that add color to data so that it isn’t effectively a very tarnished and limited utility?

Well, yes, it could very easily be the case, and by the way, that’s an extreme minority view among people in AI. I will just say that upfront. I’m not representing a majority of people in AI, but I think that could very well be the case. Let me just dive into that a little bit about how people know what we know. How is it that we are generally intelligent, have general intelligence? If I asked, “Does it hurt your thumb when you hit it with a hammer?” You would say “yes,” and then I would say, “Have you ever done it?” “Yes.” And then I would say, “Well, when?” And you likely can’t remember, and so you’re right, we have data that we take somehow learning from, and we store it and we don’t know how we store it. There’s no place in your brain which is ‘hitting your thumb with a hammer hurts,’ and then if I somehow could cut that out, you no longer know that.  It doesn’t exist. We don’t know how we’d do that.

Then we do something really clever. We know how to take data we know in one area and apply it to another area.  I could draw a picture of a completely made-up alien that is weird beyond imagination. And I could show that picture to you and then I could give you a bunch of photographs and say find that alien in these. And if the alien is upside down or underwater or covered in peanut butter, or half behind a tree or whatever, you’re like, “There it is. There it is. There it is. There it is.” We don’t know how we do that. So, we don’t know how to make computers do it.

And then if you think about it if I were to ask you to imagine a trout swimming in a river, and imagine the same trout in a jar of formaldehyde and in a laboratory. “Do they weigh the same?” You would say, “yeah.” “Do they smell the same?” “Uh, no.” “Are they the same color?” “Probably not.” “Are they the same temperature?” “Definitely not.” And even though you have no experience with any of that, you instinctively know how to apply it. These are things that people do very naturally, and we don’t know how to make machines do them.

If you were to think of a question to ask a computer like, “Dr. Smith is having lunch at his favorite restaurant when he receives a phone call.  Looking worried, he runs out the door neglecting to pay his bill. Are the owners liable to call the police?” You would say a human would say no. Clearly, he’s a doctor. It’s his favorite restaurant, he must eat there a lot, he must’ve gotten an emergency call. He ran out the door forgetting to pay.  We’ll just ask him to pay the next night he comes in. The amount of knowledge you had to have, just to answer that question is complex in the extreme.

I can’t even find a chatbot that can answer [the question:] “What’s bigger, a nickel or the sun?”  And so to try to answer a question that requires this nuance and all of this inference and understanding and all of that, I do not believe we know how to build that now. That would be, I believe, a statement within the consensus. I don’t believe we know how to build it, and even if you were to say, “Well, if you had enough data and enough computers, you could figure that out.” It may just literally be impossible, like every instantiation of every possibility. We don’t know how we do it. It’s a great mystery and it’s even hotly debated [around] even if we knew how we do it, could we build a machine to do it? I don’t even know that that’s the case.

QA:16

I think that’s part of the thing that baffles me in your book. I’m jumping a little bit around here in your book now. You do talk about consciousness and you talk about sentience and how we know what we know, who we are, what we are. You talk about the dot on pets and how they identify themselves as themselves, and with any engineering problem, sometimes you can conceive of a solution before actually the method by which to get there is accomplished.  You can conceive the idea of flying. You just don’t know what combination of anything that you are copying from birds or copying from leaves, or whatever, will function in getting to that goal: flying.

The problem with this one is that from an engineering point of view, this idea of having another human or another human-like entity that not only has consciousness but has free will and sentience as far as we can perceive it, [doesn’t recognize that] there’s a lot of things that you described in your chapter on consciousness that we don’t even know how to qualify. Which is a huge catalyst in being able to create the metadata that structures data in a way that then gives the illusion and perception of consciousness. Maybe this is where you give me your personal opinion… do you think we’ll ever be able to create an answer to that engineering question, such that technology can be built around it? Because otherwise we might just be stuck on the formulation of the problem.

The logic that says we can build it is very straightforward and seemingly ironclad. The logic goes like this. If we figure out how a neuron works, we can build one. Either physically build one or model it on a computer.  And if you can model that neuron in a computer, then you learn how it talks to other neurons and then you model 100 billion of them in the computer, and all of a sudden you have a human mind.  So that that says, we don’t have to know it, we just have to understand the physics.  So, the position just says whatever a neuron does, it behaves the laws of physics and if we can understand how those laws are interacting, then we will be able to build it. Case closed. There’s no question at all that it cannot be done.

So I would say that’s the majority viewpoint. The other viewpoint says, “Well wait a minute, we have this brain that we don’t understand how it works. And then we have this mind, and a mind is a concept everybody uses and if you want a definition, it’s kind of everything your brain can do that an organ doesn’t seem like it would be able to. You have a sense of humor; your liver may not have a sense of humor.  You have emotions, your stomach may not have emotions, and so forth.” So somehow, we have a mind that we don’t know how it comes about. And then to your point, we are conscious and what that means is we experience the world. I feel warmth, [whereas] a computer measures temperature. Those are very different things and we not only don’t know how it is that we are conscious, but we also don’t even know how to ask the question in a scientific method, nor what the answer looks like.

And so, I would say my position to be perfectly clear is, we have brains we don’t understand, minds we don’t understand and consciousness we don’t understand.  And therefore, I am unconvinced that we can ever build something like this. And so I see no evidence that we can build it because the only example that we have is something that we don’t understand. I don’t think you have to appeal to spiritualism or anything like that, to come to that conclusion, although many people would disagree with me.

QA:17

Yeah, it’s interesting. I think one thing underlying the pessimistic view is this belief that while we may not have the technology now or have an idea of how we’re going to get there, the kinetic sort of an AI explosion—that’s what I think Nick Bostrom, the philosopher has called it—may be pretty rapid in the sense that once there is material success in developing these AI models, that will encourage researchers to sort of pile on and therefore they bring in more people to produce those models and then secondly if there are advancements in self-improving AI models. So there’s a belief that it may be pretty quick that we get super intelligence that underlies this pessimism and the belief that we sort of has to act now.  What would be your thoughts on that?

Oh, well I don’t agree. I think that’s the “Is that a bear or a rock?” kind of thing. The only evidence we really have for that scenario is movies, and they’re very compelling and I’m not conspiratorial, and they’re entertaining. But what happens is you see that enough, and you do something that has a name, it’s called ‘reasoning from fictional evidence’ and that’s what we do. Where you say, “Well, that could happen, and when you see it again, and yeah, that could happen. That really could again.”  Again, and again and again.

To put it in perspective, when I say we don’t understand how the brain works, let me be really clear about that. Your brain has 100 billion neurons, roughly the same number of stars in the Milky Way. You might say, “Well, we don’t understand it because there’s so many.” This is not true. There’s a worm called the nematode worm. He’s about as long as his hair is thick, and his brain has 302 neurons. These are the most successful creatures on the planet, by the way. Seventy percent of all animals are nematode worms and have 302 neurons. That’s it. [This is about] the number of pieces of cereal in a bowl of cereal.  So, for 20 years a group of people in something called the ‘open worm project’ had been trying to model those 302 neurons in a computer to get it to display some of the complex behavior that a nematode worm does.  And not only have they not done it, but there’s also even a debate among them whether it is even possible to do that. So that’s the reality of the situation. We haven’t even gotten to the mind.

Again, how is it that we’re creative? And we haven’t even gotten to, how is it that we experience the world? We’re just talking about how does a brain work, if it only has 302 neurons, a bunch of smart people, 20 years working on it, may not even be possible. So somehow to spin a narrative that, well, yeah, that all may be true, but what if there was a breakthrough and then it sped upon itself and sped up and then it got smarter and then it got so smart it had 100 IQ, then a thousand, then a million, then 100 million. And then it doesn’t even see us anymore. That’s as speculative as any other kind of scenario you want to come up with. It’s so removed from the facts on the ground that you can’t rebut it because it is not based on any evidence that you can rebuke.

QA:18

You know, the fun thing about chatting with you, Byron, is that the temptation is to sort of jump into all these theories and which ones are your favorites. So because I have the microphone, I will.  Let me just jump into one.  Best science fiction theory that you like. I think we’ve touched on a few of these things, but what is the best-unified theory of everything, from science fiction that you feel like, ‘you know what, this might just explain it all?

Star Trek.

QA:19

Okay. Which variant of it?  Because there’s not…

Oh, I would take either….I’ll take ‘The Next Generation.’ So, what is that narrative? We use technology to overcome scarcity. We have bumps all along the way. We are insatiably curious, and we go out to explore the stars as Captain Picard told the guy he thought out from the 20th Century. He said the challenge in our time is to better yourself, is to discover who you are. And what we found interestingly with the Internet, and sure, you can list all the nefarious uses you want. What we found is the minute you make blogs, 100 million people want to tell you what they think. The minute you make YouTube, millions of people want to upload video; the minute you make iTunes, music flourishes.

I think in my father’s generation, they didn’t write anything after they left college. We wake up in the morning, and we write all day long. You send emails constantly and so what we have found is that it isn’t that there were just a few people, and as the Italian Renaissance, there were only a few people who wanted to paint or cared to paint. It was like everybody probably did. Only there wasn’t enough of the good stuff, and so only either you had extreme talent or extreme wealth and then you got to paint.

Well, in the future, in the Star Trek variant of it, we’ve eliminated scarcity through technology, and everybody is empowered, every Dante to write their Inferno, every Marie Curie to discover radium, and all of the rest. And so that vision of the future, you know, Gene Roddenberry said in the future there will be no hunger and there will be no greed and all the children would know how to read.  That variant of the future is the one that’s most consistent with the past. That’s the one you can say, “Yeah, somebody in the 1400s looking at our life today, that would look like Star Trek to them. These people like to push up a button and the temperature in the room gets cooler, and they have leisure time. They have hobbies.”  That would’ve seemed like science fiction.

QA:20

I think there’s a couple of things that I want to tackle with the Star Trek analogy to get us sort of warmed up on this and I think Kyran’s waiting here at the top to ask some of them, but I think the most obvious one to ask, if we use that as a parable of the future, is about Lieutenant Commander Data. Lieutenant Commander Data is one of the characters starring in The Next Generation and is the closest attempt to artificial general intelligence, and yet he’s crippled from fully comprehending the human condition because he’s got an emotion chip that has to be turned off because when it’s turned on, he goes nuts; and his brother is also nuts because he was overly emotional.  And then he ends up representing every negative quality of humanity. So to some extent, not only have I just show off my knowledge of the Star Trek era…

Lore wasn’t over overly emotional. He got the chip that was meant for Data and it wasn’t designed for him. That was his backstory.

QA:21

Oh, that’s right. I stand corrected, but maybe you can explore that.  In that future, walk us through why you think Gene had that level of limitation for Data, and whether or not that’s an implication of ultimately the limits of what we can expect from robots.

Well, obviously that story is about…that whole setup is just not hard science. Right? That whole setup is like you said, it’s embodying us and it’s the Pinocchio Story of Data wanting to be a boy and all of the rest. So, it’s just storytelling as far as I’m concerned. You know, it’s convenient that he has a positronic brain, and having removed part of his scalp, you just see all this light coursing through, but that’s not something that science is behind, like Warp 10 or something, the tri-quarter. You know Uhura in the original series, she had a Bluetooth device in her ear all the time, right?

QA:22

Yeah, but I guess with the Data metaphor, I guess what I’m asking is: the limitations that prevented Data from being able to do some of the things that humans do, and therefore ultimately come around the full circle into being a fully independent, conscious, free-willed, sentient being, were entire because of some human elements he was lacking. I guess the question and you brought it up in your book is, whether or not we need those human elements to really drive that final conversion of a machine to some sort of entity that we can respect as an equivalent peer to us.

Yeah. Data is a tricky one because he could not feel pain, so you would say he’s not sentient. And to be clear, sentient means, it’s often misused, to mean ‘smart.’ That’s sapient. Sentient means you can experience pain. He didn’t, but as you said, at some point in the show, he experienced emotional pain through that chip, and therefore he is sentient. They had a whole episode about, “Does Data have a soul?” And you’re right, I think there are things that humans do that it’s hard to…unless you start with the assumption everything in a human being is mechanistic, in physics and that you’re a bag of chemicals with electrical impulses going through you.

If you start with that, then everything has to be mechanical, but most people don’t see themselves that way, I have found, and so if there is something else, some emergent or something else that’s going on, then yeah, I believe that has to be wrapped up in our intelligence. That being said, everybody, I think has had this experience of when you’re driving along and you kind of space [out] and then you kind of ‘come to’ and you’re like, “Holy cow, I’m three miles along. I don’t remember driving there.” Yet you behaved very intelligently. You navigated traffic and did all of that, but you weren’t kind of conscious. You weren’t experiencing the world at least that much. That may be the limit of what we can do, that a person during that three minutes when you’re kind of spaced because that person also didn’t write a new poem or do anything creative. They just merely mechanically went through the motions of driving. That may be the limit. That may be that last little bit that makes us human.

QA:23

The Star Trek view has two pieces to it. It has a technological optimism, which I don’t contest. I think I’m aligned with you and agree with that. There’s also an economic or a social optimism there and that’s also about how that technology is owned, who owns the means of production, who owns the replicators.  When it comes to that, how precarious do you think the Star Trek Universe is in the sense that if the replicators are only in the hand of a certain group of people if they’re so expensive that only a few people learn them, or only a few people own the robots,  then it’s no longer such an optimistic scenario that we have. I’d just be interested in hearing your views there.

You’re right, that the replicator is a little bit convenient…I don’t want to say it’s a cheat, but it’s a convenient way to get around scarcity and they never really go into, well, how is it that anybody could go to the library and replicate whatever they wanted.  Like how did they get that?  I understand those arguments. We have [a world where] the ability of a person using technology to affect a lot of lives goes up and that’s why we have more billionaires. We have more self-made billionaires now; a higher percentage of billionaires are self-made now than ever before. You know, Google and Facebook together made 12 billionaires. The ability to make a billion dollars gets easier and easier, at least for some people (not me) because technology allows them to multiply and affect more lives and you’re right. So that does tend to make more super, super, super-rich people. But, I think the income inequality debate is a little…maybe needs a slight bit of focus.

To my mind, it doesn’t matter all that much how many super-rich people there are. The question is how many poor people are there? How many people have a good life? How many people can have medical care and can, you know, if I could get everybody to that state, but I had to make a bunch of super-rich people, it’s like, absolutely, we’ll take that? So I think, income inequality by itself is a distraction.

I think the question is how do you raise the lot of everybody else and what we know about technology is that it gets better over time and the prices fall over time. And that goes on ad infinitum. Who could have afforded an iPhone 20 years ago?  Nobody. Who could have afforded the cell phone 30 years ago? Rich people. Who could have afforded any of this stuff all these years ago?  Nobody but the very rich, and yet now because they get rich, all the prices of all that continue to fall and everybody else benefits from it.

I don’t deny there are all kinds of issues. You have your Hepatitis C vaccine, which costs $100,000 and there are a lot of people who need it and only a few people are going to [get it]. There are all kinds of things like that, but I would just take some degree of comfort that if history has taught us anything, is that the price of anything related to technology falls over time. You probably have 100 computers in your house.  You certainly have dozens of them, and who from 1960 would have ever thought that? Yet here they are here. Here we are in that future.

So, I think you almost have to be conspiratorial to say, yeah, we’re going to get these great new technologies, and only a few people are going to control them and they’re just going to use them to increase their wealth ad infinitum. And everybody else is just going to get the short end of the stick. Again, I think that’s playing on fear. I think that’s playing on all of that, because if you just say, “What are the facts on the ground? Are we better off than we were 50 years ago, 100 years ago, 200 years ago?” I think you can only say “yes.”

QA:24

Those are all very good points and I’m actually tempted to jump around a little bit in your book and maybe revisit a couple of ideas from the narrow AI section, but maybe what we can do is we can merge the question about robot proofing jobs with some of the stuff that you’ve talked about in the last part, which is the road from here.

One of the things that you mentioned before is a general idea that the world is getting better, no matter what. These things that we just discussed iPhones and computers being more and more accessible is an example of it.  You talked about the section of ‘murderous meerkats’ where you know, even things like crime are things that are improving over time, and therefore there is no real reason for us to fear the future. But at the same time, I’m curious as to whether or not you think that there is a decline in certain elements of society, which we aren’t factoring into the dataset of positivity.

For example, do we feel that there is a decline in the social values that have developed in the current era, in this sort of decline of social values, things like helping each other out, things like looking out for the collective versus the individual, has come and gone, and we’re now starting to see the manifestations of that through some of the social media and how it represents itself?  And I just wanted to get your ideas down the road from here and whether or not you would revisit them if somebody were to tell you and show you some sociologists’ research regarding the decline of social values, and how that might affect the kinds of jobs humans will have in the future versus robots.

So I’m an optimist about the future. I’m clear about that. Everything is hard. It’s like me talking about my companies. Everything’s a struggle to get from here to there. I’m not going to try to spin every single thing. I think these technologies have real implications on people’s privacy and they’re going to affect warfare and there are all these things that are real problems that we’re really going to have to have to think about.  The idea that somehow these technologies make us less empathetic, I don’t agree with. And you can just run through a list of examples like everybody kind of has a cause now. Everybody has some charity or thing that they support. Volunteerism, Go-Fund-Me’s are up…People can do something as simple as post a problem they have online and some stranger who will get nothing in return is going to give them a big, long answer.

People toil on a free encyclopedia and they toil in anonymity. They get no credit whatsoever. We had the ‘open-source movement. Nobody saw that. Nobody said, “Yeah, programmers are going to work really hard and write really good stuff and give it away.” Nobody said we’re going to have Creative Commons where people are going to create things that are digital and they’re going to give them away. Nobody said, “Oh yeah, people are going to upload videos on YouTube and just let other people watch them for free.” Everywhere you look, technology empowers us and our benevolence.

To take the other view is like a “Kids these days!” shaking your cane, “Get off my grass!” kind of view that things are bad now. They’re getting worse. This is what people have said for as long as people have been reflecting on age.  And so, I don’t buy any of that.  In terms of specifically about jobs, I’ve tried hard to figure out what the half-life of a job is.  And I think every 40 years, every 50 years, half of all the jobs vanish. Because what does technology do? It makes great new high-paying jobs, like a geneticist. And it destroys low-paying tedious jobs, like an order taker at a fast-food restaurant.

And what people sometimes say is, “You really think that order taker is going to become a geneticist? They’re not trained for these new jobs.” And the answer is, “Well, no.” What’ll happen is a college professor will become a geneticist and a high school biology teacher gets the college job and the substitute teacher gets hired at the high school job, all the way down. The question isn’t, “Can that person who lost their job to automation get one of these great new jobs?” The question is, “Can everybody on the planet do a job a little harder than the job they have today?” And if the answer to that is yes, then what happens is, every time technology creates great new jobs, everybody down the line gets a promotion. And that is 250 years of why have we had in the West full employment because employment other than during the depression has always been 5 to 10 percent… for 250 years.

Why have we had full employment for 250 years and rising wages? Even when something like the assembly line came out, or something like we replaced all the animal power with steam, you never had bumps in unemployment because people just used those technologies to do more. So yes, in 40 or 50 years, half the jobs are going to be gone, that’s just how the economy works. The good news is though when I think back to my K-12 education, and I think if I knew the whole future, what would I have taken then that would help me today.  And I can only think of one thing that I really just missed out on. And can you guess by the way?

QA:25

Computer education?

No, because anything they taught me would no longer be useful. Typing. I should’ve taken typing. Who would have thought that that would be like the skill I need every day the most? But I didn’t know that. So you have to say, “Wow, like everything you have, everything that I do in my job today is not stuff I learned in school.” What we all do now is you hear a new term or concept and you google it and you click on that and you go to Wikipedia and you follow the link, and then it’s 3:00 AM in the morning and you wake up the next morning, and you know something about it.  And that’s what every single one of us does, what every single one of us has always done, what every single one of us will continue to do. And that’s how the workforce morphs. It isn’t that we’re facing this kind of cataclysmic disconnect between our education system and our job market. It’s that people are going to learn to do the new things, as they learned to be web designers, and they learned every other thing that they didn’t learn in school.

QA:26

Yeah, we’d love to dive into the economic arguments in a second, but just to bring it back to your point that technology is always empowering. I’m going to play devil’s advocate here and mention someone we had on the podcast about a year ago. Tristan Harris, who’s the leader of an initiative called ‘Time Well Spent’, and his arguments were that the effects of technology can be nefarious. Two days ago, there was a New York Times article, referring to a research paper on statistical analysis and anti-refugee violence in Germany, and one of the biggest correlating factors was time spent on social media, suggesting that it isn’t always like beneficial or benign for humans. Just to play devil’s advocate here, what is your take on that?

So, is your point that social media causes people to be violent, or is the interpretation people prone to violence also are prone to using social media?

QA:27

Maybe one variant of that, and Kyran can provide his own, is that the good is getting better with technology and the bad is getting worst with technology. You just hope that one doesn’t detonate something that is irreversible.

Well, I will not uniformly defend every application of technology to every single situation. I could rattle off all the nefarious uses of the Internet, right? I mean bilking people, you know them all, you don’t need me to list it. The question isn’t, “Do any of those things happen?” The question is, “On balance, are more people using the Internet for good, than evil?” And we know the answer is ‘good.’

It has to be because if we were more evil than good as a species, we never would have survived this way. We’re highly communal. We’ve only survived because we like to support each other, forget about all the wars, granted, all of the problems, all the social strife, all of that. But in the end, you’re left with the question, “How did we make progress to begin with?” And we made progress because there are more people who are working for progress than there are…who are carrying torches and doing all the rest. It just is simple.

QA:28

I guess I’m not qualified to make this statement, but I’m going to go ahead and do it anyway. Humans have those attributes because we’re inherently social animals, and as a consequence, we’re driven to survive and forego being right at times, because we value the social structure more than we do our own selves; and we value the success of the social structure more than ourselves, and there’s always going to be deviations from that, but on average it then answers and shows and represents itself in the way that you have articulated it.

And that’s a theory that I have, but one of the things that if you accept that theory, well you can let me know or not, but let’s, for the sake of the question, let’s just assume that it’s correct, then how do you impart that onto a collection of artificial intelligence such that they mirror that? And as we start delegating more and more to that collective artificial intelligence, can we rely on them to have that same drive when they’re no longer as socially dependent on each other, the way that humans are for reproduction and defense and emotional validation?

That could well be the case, yes. I mean, we have to make sure that we program them to reflect an ethical code, and that’s an inherently very hard thing to do because people aren’t great at articulating them and even when they articulate them, they’re full of all these provisos and exceptions and everybody’s is different. But luckily, there are certain broad concepts that almost everybody agrees with. That life is better than death, and that building is better than destroying, and there are these very high-level concepts that we will need to take great pains in how we build our AIs, and this is an old debate, even in AI.

There was a man named Weizenbaum, who made a chatbot in the sixties. It was simple. You would say, “I’m having a bad day today,” and it would say, “Why are you having a bad day?” “I’m having a bad day because of my mother.” “Why are you having a bad day because of your mother?” Back and forth. Super simple. Everybody knew it was a chatbot, and yet he saw people getting emotionally attached to it, and he kind of turned on it and he said, “In the end, we never want computers.”

When the computer says ‘I understand,’ it’s just a lie, that there is no ‘I,’ and there is no understanding. And he came to believe we should never let computers do those kinds of things. They should never be…recipients of our emotions. We should never make them caregivers and all of these other things because, in the end, they don’t have any moral capacity at all. They have no empathy. They have faked empathy, they have simulated empathy, and so I think there is something to that, that there will just simply be jobs we’re not going to want them to do because, in the end, they’re going to require a person I think.

You see, any job a computer could do; a robot could do. If you make a person do that job, there’s a word for that. That’s dehumanizing. If a machine can, in theory, do a job, if you make a person do it, that’s dehumanizing.  You’re not using anything about them that makes them a human beings, you’re using them as a stand-in for a machine, and those are the jobs machines should do.

But then there are all the other jobs that only people can do, and that’s what I think people should do. I think they’re going to be a lot of things like that, that we are going to be uncomfortable with and we still don’t have any idea. Like, when you’re on a chatbot, you need to be told it’s a chatbot. Should robotic voices on the phone actually sound somewhat robotic, so you know that’s not a person? You think about R2-D2 or C-3PO, just think if their names were Jack and Larry.  That’s a subtle difference in how we regard them that we don’t know how we’re going to do that, but you’re entirely right. Machines don’t have any empathy and they can only fake it, and there are real questions if that’s good or not.

QA:29

Well, that’s a great way of looking at it, and one of the things that have been really great during this chat is understanding the origin of some of these views and how you end up at this positive outcome at the end of the day on average. And the book does a really good job of leaving the reader with that thought in mind but arms them to have these kinds of engaging conversations. So thanks for sharing the book with us and thanks for providing your opinion on different elements of the book.

However, you know, it’d be great to get some thoughts about things that you feel that inspired you or that you left out of the book. For example, which movies have most affected you in the vein of this particular book. What are your thoughts on a TV show like Westworld and how that illustrates the development of the mind of artificial intelligence in the show? Maybe just share a little bit about how your thoughts have evolved.

Certainly, and I would also like to add, I do think there’s one way it can all go south. I think there is one pessimistic future and I think that will come about if people stop believing in a better tomorrow. I think pessimism is what will get us all killed. The reason we’ve had optimism, be so successful, is there’ve been a number of people who get up and say, “Somebody needs to invent the blank. Somebody needs to find a cure for this, somebody needs to do it. I will do it.” And you have enough people who believe in one form or another, in a better tomorrow.

There’s a mentality of, don’t polish brass on a sinking ship. And that’s where you just say, “Well what’s the point? Why bother? Why bother?” And if enough people said “Why to bother?” then we are going to have to build that world. We’re going to have to build that better world. And just like I said earlier with my companies, it’s going to be hard. Everybody’s got to work hard at it. And so, it’s not a gift, it’s not free.  We’ve clawed our way from savagery to civilization and we’ve got to keep clawing. But the interesting thing is, finally I think there is enough of the good stuff for everybody and you’re right, there are big distribution problems about that, and there are a lot of people who aren’t getting any of the good stuff, and those are all real things we’re going to have to deal with.

When it comes to movies and TV, I have to see them all because everybody asks me about them on shows. So I have to go see them.  And I used to loathe going to all the pessimistic movies that have far and away dominated…In fact, I even get to think of, you know, Black Mirror, it’s like I started writing out story ideas for a show in my head, I call ‘White Mirror.’  Who’s telling those stories about how everything can be good in the future? That doesn’t mean they’re bereft of drama. It just means that it’s a different setting to explore these issues.

I used to be so annoyed at having to go to all of these movies. I would go to see some movie like Elysium and then be like, yeah, they’re the 99 percent, yeah, they’re poor and beaten down. Yeah, they’re covered in dirt. And now, yeah, the 1 percent, I bet they live in someplace high up in the sky, pretty and clean. Yeah, there that is. And then, you know, you see Metropolis, the most expensive movie ever made, adjusted for inflation, from almost a century ago. And yeah, there is the 99 percent. They’re dirty, they’re covered in dirt, everybody forgets to bathe in the future. I wonder where the…oh yeah, the one percent, yeah, they live in that tower up there.  Oh, everything up there is white and clean. Wow. Isn’t that something? And I have to sit through these things.

And then I read a quote by Frank Herbert, and he said sometimes the purpose of science fiction is to keep the future from happening. And I said, okay, these are cautionary tales. These are warnings, and now I view them all like that.  And so, I think there are a lot of cautionary tales out there and very few things that we can…like Star Trek. You heard me answer that so quickly because there aren’t a lot of positive views about the future that is in science fiction. It just doesn’t seem to be as rich of ground to tells stories and even in that world, you had to have the Ferengi, and you had to have the Klingons and you had to have the Romulans and so forth.

So, I’ve watched them all and you know, I enjoy Westworld, like the next person.  But I also realized those are people playing those androids and that nobody can build a machine that does any of that. And so it’s fiction. It’s not speculative in my mind. It’s pure fiction. It’s what they are and that doesn’t mean they’re any less enjoyable… When I ask people on my AI podcast what science fiction influenced you, they all, almost all say Star Trek. That was a show that inspired people, and so I really gravitate towards things that inspire me and inspire me in a vision of a better tomorrow.

QA:30

For me, if I had to answer that question, I would say The Matrix. And I think that it brings up a lot of philosophical questions and even questions about reality. And it’s dystopian in some ways I guess, but in some ways, it illustrates how we got there and how we can get out of it. And it has a utopian conclusion I guess because it’s ultimately in the form of liberation. But it is an interesting point you make.

And it actually makes me reflect back on all the movies that I’ve seen, and it actually also brings up another question which is whether or not it’s just representative of the times. Because if you look at art and if you look at literature over the years, in many ways they are inspired by what’s going on during that era. And you can see bouts of optimism, post-the resolution of some conflict. And then you can see the brewing of social upheaval, which then ends up with some sort of a conflict, and you see that all across the decades and it is interesting.  And I guess that brings up moral responsibility for us not to generate the most intense set of innovations around artificial intelligence, in a point where maybe society is quite split at the moment.  We might inject unfortunate conclusions into AI systems just because of the state of where we are in our geopolitical evolution.

Yeah. I call my airline of choice once a week to do something, and it asked me to state my member number, which unfortunately has an A, an H, and an 8 in it.  And it never gets it right. So that’s what people are trying to do with AI today, is it’s just like make a lot of really tedious stuff less tedious and use caller ID by the way. I always call from the same number, but that’s a different subject.

And so most of the problems that we try to solve with it are relatively mundane, and most of them are about how do we stop disease, and how do we… all of these very worthwhile things. It’s not a scary technology. It studies the past, looks for patterns in data, projects into the future. That’s it. And anything around that that tries to make it terrifying, I think is sensationalism. I think the responsibility is to tell the story about AI like that, without the fear, and emphasizing the positivity of all the good that can come out of this technology.

QA:31

What do you think we’ll look upon 50 years from now and think, “Wow, why were we doing that?” How do you get away with that, the way that we look back today on slavery and think, “Why the hell did that happen?”

Well, I will give an answer to that. And it’s not my own personal ax to grind. To be clear, I live in Austin, Texas. We have barbecue joints here in abundance, but I believe that we will learn to grow meat in a laboratory and it will be not only environmentally, massively better, but it will taste better, and be cheaper and healthier and everything.  And so I think we’re going to grow all of our meat and maybe even all of our vegetables, by the way. Why do you need sunlight and rain and all of that?  But put that aside for a minute, I think we’re going to grow all of our meat in the future and I don’t know if you grow it from a cell, if it’s still veganism to eat it. Maybe it is, I don’t know, like strictly speaking, but I think once the best steak you’ve ever had in your life is 99 cents, everybody’s just going to have that.

And then we’ll look back at how we treat animals with a sense of collective shame of that, because the question is, “Can they feel?” In the United States, up until the mid-90s, veterinarians were taught that animals couldn’t feel pain and so they didn’t anesthetize them. They also operated on babies at the same time because they couldn’t feel pain. Now I think people care whether the chicken that they’re eating was raised humanely. And so, I think that expansion of empathy to animals, who now I think most people believe they do feel pain, they do experience sadness or something that must feel like that, and the fact that we essentially keep them in abhorrent conditions and all of that.

And again, I’m not grinding my own axe here. This isn’t something that…I don’t think it’s going to come up with people, like overnight changing. I think what’s gonna happen is there’ll be an alternative. The alternative will be so much better, but then everybody would use it and look back and think, how in the world did we do that?

QA:32

No, I agree with that.  As a matter of fact, we’ve invested in a company that’s trying to solve that problem, and I’m going to post in the show notes just because they’re in stealth right now, but by the time this interview goes to print, hopefully, we’ll be able to talk about them. But yes, I agree with you entirely, and we put our money behind it. So, looking forward to that being one of the issues to be solved. Now another question is, what’s something that you used to strongly believe in, that now you think you were fundamentally misguided about?

Oh, that happens all the time. I didn’t write this book to start off by saying, “I will write a book that doesn’t really say what I think, it’ll just be this framework.” I wrote a book to try to figure out what I think because I would hear all of these proclamations about these technologies and what they could do. And so, I think I used to be way more in the AGI camp, that this is something we’re going to build and we’re going to have those things, like on Westworld. This was before Westworld though. And I used to be much more in that until I wrote the book, which changed me and I can’t say I disbelieve it, that would be the wrong way to say it, but I see no evidence for it. I think I used to buy that narrative a lot more and I didn’t realize it was less a technological opinion and more a metaphysical opinion. And so, like working through all of that and just understanding all of the biases and all of the debate. It’s very humbling because these are big issues and what I wanted to do as I said, is make a book that helps other people work through them.

QA:33

Well, it is a great book. I’ve really enjoyed reading it. Thank you very much for writing it. Congratulations! You’re also the longest podcast we’ve ever recorded, but it’s a subject that is very dear to me, and one that is endlessly fascinating, and we could continue on, but we’re going to be respectful of your time, so thank you for joining us and for your thoughts.

Well, thank you. Anytime you want me back, I would love to continue the conversation.

Byron Reese: This is Voices in AI brought to you by GigaOm, I’m Byron Reese. Today our guest is Tim O’Reilly.

He is, of course, the founder and CEO of O’Reilly Media, Inc. In addition to his role at O’Reilly, he is a partner at an early-stage venture firm, O’Reilly AlphaTech Ventures, and he is on the board of Maker Media, which was spun out from O’Reilly back in 2012.

He’s on the board of Code for America, PeerJ, Civis Analytics, and POPVOX. He is the person who popularized the terms “open source” and “web 2.0.”

He holds an undergraduate degree from Harvard in the classics. Welcome to the show, Tim.

Tim O’Reilly: Hi, thanks very much, I’m glad to be on it. I should add one other thing to my bio, which is that I’m also the author of a forthcoming book about technology and the economy, called WTF: What’s The Future, and Why It’s Up to Us, which in a lot of ways, it’s a memoir of what I’ve learned from studying computer platforms over the last 30 years, and reflections on the lessons of technology platforms for the broader economy, and the choices that we have to make as a society.

Well, I’ll start there. What is the future then? If you know, I want to know that right away.

Well, the point is not that there is one future. There are many possible futures, and we actually have a great role.

There’s a very scary narrative in which technology is seen as an inevitability. For example, “technology wants to eliminate jobs, that’s what it’s for.”

And I go through, for example, looking at algorithms, at Google, at Facebook, and the like and say, “Okay, what you really learn when you study it is, all of these algorithms have a fitness function that they’re being managed towards,” and this doesn’t actually change in the world of AI.

AI is simply new techniques that are still trying to go towards human goals. The thing we have to be afraid of is not AI becoming independent and going after its own goals.

It’s what I refer to as “the Mickey and the broomsticks problem,” which is, we’re creating these machines, we’re turning them loose, and we’re telling them to do the wrong things.

They do exactly what we tell them to do, but we haven’t thought through the consequences and a lot of what’s happening in the world today is the result of bad instructions to the machines that we have built.

In a lot of ways, our financial markets are a lot like Google and Facebook, they are increasingly automated, but they also have a fitness function.

If you look at Google; their fitness function on both the search and the advertising side is relevant. You look at Facebook; loosely it could be described as engagement.

We have increasingly, for the last 40 years, been managing our economy around, “make money for the stock market,” and we’ve seen, as a result, the hollowing out of the economy.

And to apply this very concretely to AI, I’ll bring up a conversation I had with an AI pioneer recently, where he told me he was investing in a company that would get rid of 30% of call center jobs, was his estimate.

And I said, “Have you used a call center?

Were you happy with the service?

Why are you talking about using AI to get rid of these jobs, rather than to make the service better?”

You know I wrote a piece—actually I wrote it after the book, so it’s not in the book—[that’s] analysis of Amazon.

In the same 3 years which they added 45,000 robots to their factories, they’ve added hundreds of thousands of human workers.

The reason is that they’re saying “Oh, our master design pattern isn’t ‘cut costs and reap greater profits,’ it’s ‘keep upping the ante, keep doing more.’”

I actually started off the article by talking about my broken tea kettle and how I got a new one the same day, so I could have my tea the next morning, with no interruption.

And it used to be that Amazon would give you free 2-day shipping, and then it was free 1-day shipping, and then in many cases, it’s free same-day shipping, and, this is why they have this incredible fanatical customer focus, and they’re using the technology to actually do more.

My case has been, that if we actually shift from the fitness function being efficiency and shareholder value through driving increases profits to instead actually creating value in society—which is something that we can quite easily do—we’re going to have a very different economy and a very, very different political conversation than we’re having right now.

Q: 1

Q: 4

Q: 6

Q: 23

Q: 1

So I’d love to start with the simple question, what is artificial intelligence?

Well, there is the definition of artificial intelligence of machines being able to perform cognitive tasks, that we as humans can do very easily.

What I like to think about in artificial intelligence, is machines taking on tasks for us that do require intelligence, but leave us time to do more thinking and more imagination, in the real world.

So autonomous cars, I would love to have one, that requires artificial intelligence, and I hate driving, I hate the fact that I have to drive for 30 minutes to an hour every day, and waste a lot of time, my cognitive time, thinking about the road.

So when I think about AI, I think about how it improves my life to give me more time to think about even higher-level things.

Q: 2

Well, let me ask the question a different way, what is intelligence?

That’s a very philosophical question, yes, so it has a lot of layers in it.

So, when I think about intelligence for humans, it’s the ability to imagine something new, so imagine, have a problem and imagine a solution and think about how it will look like without actually having to build it yet, and then going in and implementing it. That’s what I think about [as] intelligence…

Q: 3

But a computer can’t do that, right?

That’s right, so when I think about artificial intelligence, personally at least, I don’t think that, at least in our lifetime, computers will be able to solve those kinds of problems, but, there is a lower level of intelligence of understanding the context of where you are and being able to take actions on it, and that’s where I think that machines can do a good task.

So understanding a context of the environment and taking immediate actions based on that, that are not new, but are already… people know how to do them, and therefore we can code them into machines to do them.

Q: 4

I’m only going to ask you one more question along these lines and then we’ll move on, but you keep using the word “understand.” Can a computer understand anything?

So, yeah, the word understanding is another hard word to say. I think it can understand, well, at least it can recognize concepts.

Understanding maybe requires a higher level of thinking, but understanding context and being able to take an action on it, is what I think understanding is.

So if I see a kid going into the road while I’m driving, I understand that this is a kid, I understand that I need to hit the brake, and I think machines can do these types of understanding tasks.

Q: 5

Fair enough, so, if someone said what is the state of the art like, they said, where are we at with this, because it’s in the news all the time and people read about it all the time, so where are we at?

So, I think we’re at the point where machines can now recognize a lot of images and audio or various types of data, recognize with sensors, recognize that there are objects, recognize that there are words being spoken, and identify them.

That’s really where we’re at today, we’re not… we’re getting to the point where they’re starting to also act on these recognition tasks, but most of the research, most of what AI is today, is the recognition tasks. That’s the first step.

Q: 6

And so let’s just talk about one of those. Give me something, some kind of recognition that you’ve worked on and have deep knowledge of, teaching a computer how to do…

All right, so, when I did my PhD, I worked on affective computing, so, part of the PhD was to have machines recognize emotions from facial expressions. So, it’s not really recognizing emotion, it’s recognizing a facial expression and what it may express.

So there are 6 universal facial expressions that we as humans exhibit, so, smiling is associated with happiness, there is surprise, anger, disgust, and those are actually universal.

So, the task that I worked on was to build classifiers, that given an image or a sequence of a video of a person, a person’s face, would recognize whether they’re happy or sad or disgusted or surprised or afraid…

Q: 7

So how do you do that? Like do you start with biology and you say “well how do people do it?”

Or do you start by saying “it doesn’t really matter how people are doing it, I’m just going to brute force, show enough labeled data, that it can figure it out, that it just learns without ever having a deep understanding of it?”

All right so this was in the early 2000s, and we didn’t have deep learning yet, so we had neural networks, but we weren’t able to train them with huge amounts of data. There wasn’t a huge amount of data, so the brute force approach was not the way to go.

What I actually worked on is based on research by a psychologist, that actually mapped facial movements to known expressions, and therefore to known emotions.

So it started out in the 70s, by people in the psychology field, [such as] Charles Akemann, in San Francisco, who mapped out actual… he created a map of facial movements into facial expressions, and so that was the basis of what are the type of features I need to extract from video and then feed that to a classifier, and then you go through the regular process of machine learning of collecting a lot of data, but the data is transformed, so these videos were transformed into known features of facial movements, and then, you can feed that into a classifier that learns in a supervised way.

So I think a lot of the tasks around intelligence are that way. It’s being changed a little bit by deep learning, which supposedly takes away the need to know the features are a priori and do the feature engineering for the machinery task…

Q: 8

Why do you say “supposedly”?

Because it’s not completely true. You still have to do, even in speech, even in images, you still have to do some transformations of the raw data, it’s not just taking it as is, and it will work magically and do everything for you.

There is some… you do have to, for example in speech, you do have to do various transformations of the speech into all sorts of short term Fourier transform or other types of transformations, without which, the methods afterward will not produce results.

Q: 9

So, if I look at a photo of a cat, that somebody’s posted online or a dog, that’s in surprise, you know, it’s kind of comical, the look of surprise, say, but a human can recognize that in something as simple as a stick figure…

What are we doing there do you think? Is that a kind of transferred learning, or how is it that you can show me an alien and I would say, “Ah, he’s happy…”What do you think we’re doing there…?

Yeah, we’re doing transferred learning.

Those are real examples of us taking one concept that we were trained on from the day we were born, with our visual cortex and also then in the brain, because our brain is designed to identify emotions, just out of the need to survive, and then when we see something else, we try to map it onto a concept that we already know, and then if something happens that is different from what we expected, then we start training to that new concept.

So if we see an alien smiling, and all of a sudden when he smiles, he shoots at you, you would quickly understand that smiling for an alien, is not associated with happiness, but you will start off by thinking, “this could be happy”.

Q: 10

Yeah, I think that I remember reading that, hours after birth, children who haven’t even been trained on it, can recognize the difference between a happy and sad face.

I think they got sticks and put drawings on them and try to see the baby’s reactions. It may even be even something deeper than something we learn, something that’s encoded in our DNA.

Yeah, and that may be true because we need to survive.

Q: 11

So why do you think we’re so good at it and machines aren’t, right, like, machines are terrible right now at transfer learning.

We don’t really know how it works do we, because we can’t really code that abstraction that a human gets, so…

I think that from what I see first, it’s being changed. I see work coming out of Google AI labs that is starting to show how they are able to train single models, very large models, that are able to do some transfer learning on some tasks, and, so it is starting to change.

So machines have a very different… they don’t have to survive –  they don’t have this notion of danger and surviving, and I think until we are able to somehow encode that in them, we would always have to, ourselves, code the new concepts or understand how to code for them, how to learn new concepts using transfer learning…

Q: 12

You know the roboticist Rodney Brooks, talks about “the juice”, he talks about how, if you put an animal in a box, it feels trapped, it just tries and tries to get out and it clearly has a deep desire to get out, but you but in a robot to do it, the robot doesn’t have what he calls “the juice,” and he, of course, doesn’t think it’s anything spiritual or metaphysical or anything like that.

But what do you think that is? What do you think is the juice? Because that’s what you just alluded to, machines don’t have to survive, so what do you think that is?

So I think he’s right, they don’t have the juice. Actually in my lab, during my PhD, we had some students working on teaching robots to move around, and actually, the way they did it was rewards and punishments.

So they would get… they actually coded—just like you have in reinforcement learning—if you hit a wall, you get a negative reward. If the robot moved and did something he wasn’t supposed to, the PhD student would yell at them, and that would be encoded into a negative reward, and if he did something right, they had actions that gave them positive rewards.

Now it was all kind of fun and games, but potentially if you do this for long enough, with enough feedback, the robot would learn what to do and what not to do, the main thing that’s different is that it still lives in the small world of where they were, in the lab or in the hallways of our labs. It didn’t have the intelligence to then take it and transfer it to somewhere else…

Q: 13

But the computer can never… I mean the inherent limitations are there and that the computer can never be afraid, be ashamed, be motivated, be happy…

Yes. It doesn’t have the long-term reward or the urge to survive, I guess.

Q: 14

You may be familiar with this, but I’d like to set it up anyway. There was a robot in Japan, it was released in a mall, and it was basically being taught how to get around and if it ran into a person, if it came up to a person, it would politely ask the person to move, and if the person didn’t, it would just zoom around them.

And what happened was children would just kind of mess with it, maybe jump in front of it when it tried to go around them again and again and again, but the more kids there were, the more likely they were to get brutal.

They would hit it with things, they would yell at it and all of that, and the programmers ended up having to program it, that if it had a bunch of short people around it, like children, it needed to find a tall person, an adult, and zip towards it, but the distressing thing about it is when they later asked those children who had done that, they said, “Did you cause the robot distress?” 75% of them said yes, and then they asked if it behaved human-like or machine-like, and only 15% said machine-like, and so they thought that they were actually causing distress and it was behaving like a humanoid.

What do you think that says? Does that concern you in any way?

Personally, it doesn’t, because I know that, as long as machines don’t have a real effect on them, then, we might be transferring what we think stress is onto a machine that doesn’t really feel that stress… it’s really about codes…

Q: 15

I guess the concern is that if you get in the habit of treating something that you regard as being in distress if you get into the habit of treating it callously, this is what Weizenbaum said, he thought that it would have a dampening effect on human empathy, which would not be good…

Let me ask you this, what do you think about embodying artificial intelligence? Because you think about the different devices: Amazon has theirs, it’s right next to me, so I can’t say its name, but it’s a person’s name…

Apple has Siri, Microsoft has Cortana… But Google just has the google system, it doesn’t have a name.

Do you think there’s anything about that… why do you think it is? Why would we want to name it or not name it, why would we decide not to name it?

Do you think we’re going to want to interact with these devices as if they’re other people? Or are we always going to want them to be obviously mechanistic?

My personal feeling is that we want them to be mechanistic, they’re there not to exist on their own accord, and reproduce and create a new world.

They’re there to help us, that’s the way I think AI should be, to help us in our tasks.

Therefore when you start humanizing it, then you’re going to either have the danger of mistreating it, treating it like basically slaves, or you’re going to give it other attributes that are not what they are, thinking that they are human, and then going the other route, and they’re there to help us, just like robots, or just like the industrial revolution brought machines that help humans manufacture things better…

So they’re there to help us, I mean we’re creating them, not as beings, but rather as machines that help us improve humanity, and if we start humanizing them and then, either mistreating them as you mentioned with the Japanese example, then it’s going to get muddled and strange things can happen…

Q: 16

But isn’t that really what is going to happen? Your PhD alone, which is how do you spot emotions?

Presumably would be used in a robot, so it could spot your emotions, and then presumably it would be programmed to empathize with you, like “don’t be worried, it’s okay, don’t be worried,” and then to the degree, it has empathy with you, you have an emotional attachment to it, don’t you go down that path?

It might, but I think we can stop it. So the reason to identify the emotion is that it’s going to help me do something, so, for example, our research project was around creating assistance for kids to learn, so in order to help the kid learn better, we need to empathize with the state of mind of the child, so it can help them learn better.

So that was the goal of the task, and I think as long as we encapsulate it in well-defined goals that help humans, then, we won’t have the danger of creating… the other way around.  Now, of course maybe in 20 years, what I’m saying now will be completely wrong and we will have a new world where we do have a world of robots that we have to think about how do we protect them from us. But I think we’re not there yet, I think it’s a bit science fiction, this one.

Q: 17

So I’m still referring back to your earlier “supposedly” comment about neural nets, what do you think are other misconceptions that you run across about artificial intelligence?

What do you think are, like your own pet peeves, like “that’s not true, or that’s not how it works?” Does anything come to mind?

People think, because of the hype, that it does a lot more than it really does.

We know that it’s really good at classification tasks, it’s not yet very good at anything that’s not classification, unsupervised tasks, it’s not being able to learn new concepts all by itself, you really have to code it, and it’s really hard.

You need a lot of good people that know the art of applying neural nets to different problems. It doesn’t happen just magically, the way people think.

Q: 18

I mean you’re of course aware of high profile people: Elon Musk, Stephen Hawking, Bill Gates, and so forth who [have been] worried about what a general intelligence would do, they use terms like “existential threat” and all that, and they also, not to put words in their mouth, believe that it will happen sooner rather than later…

Because you get Andrew Ng, who says, “worry about the overpopulation of Mars,” maybe in a couple of hundred years you have to give it some thought, but you don’t really right now…So where do you think their concern comes from?

So, I’m not really sure and I don’t want to put any words in their mouth either, but, I mean the way I see it, we’re still far off from it being an existential threat.

The main concern is you might have people who will try to abuse AI, to actually fool other people, that I think is the biggest danger, I mean, I don’t know if you saw the South Park episode last week, they had their first episode where Cartman actually bought an Alexa and started talking to his Alexa, and I hope your Alexa doesn’t start working now….

So it basically activated a lot of Alexas around the country, so he was adding stuff to the shopping cart, really disgusting stuff, he was setting alarm clocks, he was doing all sorts of things, and I think the danger of the AI today is really getting abused by other people, for bad purposes, in this case, it was just funny…

But you can have cases where people will control autonomous cars, other people’s autonomous cars by putting pictures by the side of the road and causing them to swerve or stop or do things they’re not supposed to, or building AI that will attack other types of AI machines.

So I think the danger comes from the misuse of the technology, just like any other technology that came out into the world… And we have to… I think that’s where the worry comes from and making sure that we put some sort of ethical code of how to do that…

Q: 19

What would that look like? I mean that’s a vexing problem…

Yes, I don’t know, I don’t have the answer to that…

So there are a number of countries, maybe as many as twenty, that are working on weaponizing, building AI-based weapons systems, that can make autonomous kill decisions. Does that worry you?

Because that sounds like where you’re going with this… if they put a plastic deer on the side of the road and make the car swerve, that’s one thing, but if you literally make a killer robot that goes around killing people, that’s a whole different thing.

Does that concern you, or would you call that legitimate use of the technology…?

I mean this kind of use will happen, I think it will happen no matter what, it’s already happening with drones that are not completely autonomous, but they will be autonomous probably in the future.

I think that I don’t know how it can be… this kind of progress can be stopped, the question is, I mean, the danger I think is, do these robots start having their own decision-making and intelligence that decides, just like in the movies, to attack all humankind and not just the side they’re fighting on…

Because technology in [the] military is something that… I don’t know how it can be stopped, because it’s driven by humans…

Our need is to wage war against each other… The real danger is, do they turn on us?

And if there is real intelligence in artificial intelligence, and real understanding and need to survive as a being, that’s where it becomes really scary…

Q: 20

So it sounds like you don’t necessarily think we’re anywhere near close to an AGI, and I’m going to ask you how far away you think we are… I want to set the question up as saying that, there are people who think we’re 5-10 years away from general intelligence and then there are people who think we’re 500 years [away].

Oren Etzioni was on the show, and he said he would give anyone 1000:1 odds that we wouldn’t have it in 5 years, so if you want to send him $10 he’ll put $10,000 against that. So why do you think there’s such a gap, and where are you in that continuum?

Well, because the methods we’re using are still so… as smart as they got, they’re still doing rudimentary tasks. They’re still recognizing images—the agents that are doing automated things for us, they’re still doing very rudimentary tasks. General intelligence requires a lot more than that, that requires a lot more understanding of context.

I mean the example of Alexa last week, that’s a perfect example of not understanding context, for us as humans, we would never react to something on TV like that and add something to our shopping cart, just because Cartman said it, where even the very, very smart Alexa with amazing speech understanding, and taking actions based on that, it still doesn’t understand the context of the world, so I think prophecy is for fools, but I think it’s at least 20 years out…

Q: 21

You know, we often look at artificial intelligence and its progress based on games where it beats the best player, that goes back to [Garry] Kasparov in 97, you have of course Jeopardy, you have Alpha Go, you had… an AI beat some world rated poker players, what do you think…And those are all kind of… they create a stir, you want to reflect on it, what do you think is the next thing like that, that one day, snap your fingers and all of a sudden an AI just did… what?

Okay, I haven’t thought about that… All these games, what makes them unique is that they are a very closed world; the world of the game, is finite and the rules are very clear, even if there’s a lot of probability going on, the rules are very clear, and if you think in the real world—and this may be going back to the questions why it will take time—for artificial intelligence to really be general intelligence, the real world is almost infinite in possibilities and the way things can go, and even for us, it’s really hard.

Now trying to think of a game that machines would beat us next in. I wonder if we were able to build robots that can do lots of sports, I think they could beat us easily in a lot of games because if you take any sports game like football or basketball, they require intelligence, they require a lot of thinking, very fast thinking and pathfinding by the players, and if we were able to build the body of the robot that can do the motions just like humans, I think they can easily beat us at all these games.

Q: 22

Do you, as a practitioner… I’m intrigued by it, on the topic of general intelligence, intrigued by the idea that human DNA isn’t really that much code, and if you look at how much code that we are different than say a chimp, it’s very small, I mean it’s a few megabytes.

That would be, how we are programmatically different, and yet, that little bit of code, makes us have general intelligence and a chimp not.

Does that persuade you or suggest to you that general intelligence is a simple thing, that we just haven’t discovered, or do you think that general intelligence is a hack of a hundred thousand different… like it’s going to be a long slog and then we finally get it together…?

So, I think [it’s] the latter, just because of the way you see human progress, and it’s not just about one person’s intelligence. I think what makes us unique is the ability to combine the intelligence of a lot of different people to solve tasks, and that’s another thing that makes us very different.

So you do have some people that are geniuses that can solve really really hard tasks by themselves, but if you look at human progress, it’s always been around combined intelligence of getting one person’s contribution, then another person’s contribution, and thinking about how it comes together to solve that, and sometimes you have breakthroughs that come from an individual, but more often than not, it’s the combined intelligence that creates the drive forward, and that’s the part that I think is hard to put into a computer…

Q: 23

You know there are people that have, amazing savant-like abilities. I remember reading about a man named [George] Dantzig, and he was a graduate student in statistics, and his professor put two famous unsolvable/unsolved problems on the blackboard, and Dantzig arrived late that day.

He saw them and just assumed that they were the homework, so he copied them down and went home, and later he said he thought they were a little harder than normal, but he solved them both and turned them in… and that like really happened.

It’s not one of that urban legend kind of things, you have people who can read the left and right page of a book at the same exact time, you have… you just have people that are these extraordinary edge cases of human ability, does that suggest that our intellects are actually far more robust than they are? Does that suggest anything to you as an artificial intelligence guy?

Right, so coming from the probability space, it just means that our intelligence has a wide distribution, and there are always exceptions in the tails, right?

And these kinds of people are in the tails, and often when they are discovered, they can create monumental breakthroughs in our understanding of the world, and that’s what makes us so unique.

You have a lot of people in the center of the distribution, that is still contributing a lot, and making advances to the world and to our understanding of it, and not just understanding, but actually creating new things.

So I’m not a genius, most people are not geniuses, but we still create new things and are able to advance things, and then, every once in a while you get these tails of a distribution intelligence, that could solve the really hard problems that nobody else can solve, and that’s a… so the combination of all that actually makes us push things forward in the world, and I think that kind of combined intelligence, I think that artificial intelligence is way, way off.

It’s not anywhere near, because we don’t understand how it works, I think it would be hard for us to even code that into machines. That’s one of the reasons I think AI, the way people are afraid of it, it’s still way off…

Q: 24

But by that analysis, that sounds like, to circle that back, there will be somebody that comes along that has some big breakthrough in general intelligence, and ta-da, it turns out all along it was, you know, bubble sort or….

I don’t think it’s that simple, that’s the thing, and solving a statistical problem that’s really, really tough, it’s not like… I don’t think it’s a well-defined enough problem, that some will take a genius just to understand…

“Oh, it’s that neuron going right to left,” and that’s it… so I don’t think it’s that simple… there might be breakthroughs in mathematics, that help you understand the computation better, maybe quantum computers that will help you do the faster computation, so you can train much, much faster than machines so they can do the task much better, but, it’s not about understanding the concept of what makes a genius. I think that’s more complicated, but maybe it’s my limited way of thinking, maybe I’m not intelligent enough with it…

Q: 25

So to stay on that point for a minute… it’s interesting and I think perhaps, telling, that we don’t really understand how human intelligence works, like if you knew that.. like we don’t know how a thought is encoded in the brain… like if I said…Ira, what color was your first bicycle, can you answer that question?

I don’t remember… probably blue…

Q: 26

Let’s assume for a minute that you did remember. It makes my example bad, but there’s no bicycle location in your brain that stored the first “bicycle”… like an icon, or database lookup…like nobody knows how that happens… not only how it’s encoded, but how it’s retrieved…

And then, you were talking earlier about synthesis and how we use it all together, we don’t know any of that… Does that suggest to you that, on the other end, maybe we can’t make a general intelligence… or at the very least, we cannot make a general intelligence until we understand how it is that people are intelligent…?

That may be, but yeah. First of all, even if we made it, if we don’t understand it, then how would we know that we made it?

Circling back to that… I think the way we… it’s just like the kids, they were thinking that they were causing stress to the robot because they were giving it… they thought they understood the stress and the effect of it, and they were transferring it onto the robot.

So maybe when we create something very intelligent that looks to be like us, we would think we created intelligence, but we wouldn’t know that for sure until we know what is… general intelligence really is…

Q: 27

So do you believe that general intelligence is an evolutionary invention that will come along if, in 20 years, 50 years, 1,000 years… whatever it is, that it is something that will come along out of the techniques we use today from the early AI, like, are we building really, really, really primitive general intelligence, or do you have a feeling that a real AGI is going to be a whole different kind of approach in technology?

I think it’s going to be a whole different approach. I think what we’re building today are just machines that do tasks that we humans do, in a much, much better way, and just like we built machines in the industrial revolution that did what people did with their hands, but did it in a much faster way, and better way… that’s the way I see what we’re doing today…

And maybe I’m wrong, maybe I’m totally wrong, and we’re giving them a lot more general intelligence than we’re thinking, but the way I see it, it’s driven by economic powers, it’s driven by the need of companies to advance, and take away tasks that cost too much money to do by humans, or are too slow to do by humans…

And, revolutionizing that way, and I’m not sure that we’re really giving them general intelligence yet, still, we’re giving them ways to solve specific tasks that we want them to solve, and not something very very general that can just live by itself, and create new things by itself.

Q: 28

Let’s take up this thread, that you just touched on, about, we build them to do jobs we don’t want to do, and you analogize it to the Industrial Revolution… so as you know, just to set the problem up, there are 3 different narratives about the effect this technology, combined with robotics, or we’ll call it automation, in general, are going to have on jobs.

And the three scenarios are: one is that, it’s going to destroy an enormous number of quotes, low-skill jobs, and that, they will, by definition, be fewer low skilled jobs, and more and more people competing for them and you will have this permanent class of unemployable… it’s like the Great Depression in the US, just forever.

And then you have people who say, no, it’s different than that, what it really is, is, they’re going to be able to do everything we can do, they’re going to have escape… Once a machine can learn a new task faster than a person, they’ll take every job, even the creative ones, they’ll take everything.

And the third one says no, for 250 years we’ve had 5-10% of unemployment, its never really gotten out of that range other than the anomalous depression, and in that time we had electricity, we had mechanization, we had steam power, we had the assembly line… we had all these things come along that sure looked like job eaters, but what people did is they used the new technology to increase their own productivity and drive their own wages higher, and that’s the story of progress, that we have experienced…So which of those three theories, or maybe a fourth one, do you think is the correct narrative?

I think the third theory is probably the more correct narrative. It just gives us more time to use our imagination and be more productive at doing more things, improve things, so, all of a sudden we’ll have time to think about going and conquering the stars, and living in the stars, or improving our lives here in various ways…

The only thing that scares me is the speed of it, if it happens too quickly, too fast… So, we’re humans, it takes, as a human race, some time to adapt. If the change happens so fast and people lose their jobs too quickly, before they’re able to retrain for the new economy, the new way of [work], the fact that some positions will not be available anymore, that’s the real danger and I think if it happens too fast around the world, then, there could be a backlash.

I think what will happen is that the progress will stop because some backlash will happen in the form of wars, or all sorts of uprisings, because, in the end, people need to live, people need to eat, and if they don’t have that, they don’t have anything to live for, they’re going to rise up, they’re not just going to disappear and die by themselves.

So, that’s the real danger, if the change happens too rapidly, you can have a depression that will actually cause the progress to slow down, and I hope we don’t reach that because I would not want us, as a world, to reach that stage where we have to slow down, with all the weapons we have today, this could actually be catastrophic too…

Q: 29

What do you mean by that last sentence?

So I mean we have nuclear weapons…

Q: 30

Oh, I see, I see, I see.

We have actual weapons that can, not just… could actually annihilate us completely…

Q: 31

You know, I hear you  Like…what would “too fast” be? First of all, we had that when the Industrial Revolution came along… you had the Luddite movement, when Ludd broke two spinning wheels you had the thresher riots [or Swing riots] in England in the 1820s, when the automated threat, you had the… the first day the London Times was printed using steam power instead of people.

They were going to go find the guy who invented that and strings him up, you had a deep-rooted fear of labor-changing technology, that’s a whole current that constantly runs, but what would too fast look like?

The electrification of the industry just happened lightning-fast, we went from generating 5% of our power from steam to 85% in just22 years…Give me a “too fast” scenario. Are you thinking about the truck drivers, or… tell me how it could “be too fast,” because you seem to be very cautious, like, “man, these technologies are hard and they take a long time and there’s a lot of work and a lot of slog,” and then, so what would too fast look like to you?

If it’s less than a generation, let’s say in 5 years, really, all taxi drivers and truck drivers lose their job because everything becomes automated, that seems to be too fast. If it happens in 20 years, that’s probably enough time to adjust, and I think… the transition is starting, it will start in the next 5 years, but it will still take some time for it to really take hold, because if people lose those jobs today, and you have thousands or hundreds of thousands, or even millions of people doing that, what are they going to do?

Q: 32

Well, presumably, I mean, classic economics says that, if that happened, the cost of taking a cab goes way down, right? And if that happens, that frees up money that I no longer have to spend on an expensive cab, and therefore I spend that money elsewhere,  which generates demand for more jobs, but, is the 5-year scenario… it may be a technical possibility, like we may “technically” do it, if we don’t have a legislative hurdle.

I read this article in India, which said they’re not going to allow self-driving cars in India because that would put people out of work, then you have the retrofit problem, then every city’s going to want to regulate it and say well, you can have a self-driving car, but it needs to have a person behind the wheel just in case. I mean like you would say, look, we’ve been able to fly airplanes without a pilot for decades, yet no airline in the world would touch that, in this plane, we have no pilot… even though that’s probably a better way to do it…So, do you really think we can have all the taxi drivers gone in 5 years?

No, and exactly for that reason, even if our technology really allows it. First of all, I don’t think it will totally allow it, because for it to really take hold you have to have a majority of cars on the road to be autonomous. Just yesterday I was in San Francisco, and I heard a guy say he was driving behind one of those self-driving cars in San Francisco, and he got stuck behind it, because it wouldn’t take a left turn when it was green, and it just forever wouldn’t take a left turn that humans would… The reason why it wouldn’t take a left turn was there were other cars that are human-driven on the road, and it was coded to be very, very careful about it, and he was 15 minutes late to our meeting just because of that self-driving car…

Now, so I think there will be a long transition partly because legislation will regulate it, and slow it down a bit, which is a good thing. You don’t want to change too fast, too quickly without making sure that it really works well in the world, and as long as there is a mixture of humans driving and machines driving, the machines will be a little bit “lame,” because they will be coded to be a lot more careful than us, and we’re impatient, so, that will slow things down which is a good thing, I think making a change too fast can lead to all sorts of economic problems as well…

Q: 33

You know in Europe they had… I could be wrong on this, I think it was first passed in France, but I think it was being considered by the entire EU, and it’s the right to know why the AI decided what it did. If an AI made the decision to deny you a loan, or what have you, you have the right to know why it did that… I had a simple question which was, is that possible? Could Google ever say, I’m number four for this search and my competitor’s number three, why am I number four and they’re number three? Is Google big and complicated enough, and you don’t have to talk specifically about Google, but, are systems big and complicated enough that we don’t know… there are so many thousands of factors that go into this thing, that many people never even look at, it’s just a whole lot of training…

Right, so in principle, the methods could tell you why they made that decision. I mean, even if there are thousands of factors, you can go through all of them and have not just the output of their recognition, but also highlight what were the attributes that caused it to decide it’s one thing or another. So from the technology point of view, it’s possible, from the practical point of view, I think for a lot of problems, you don’t, you won’t really care. I mean, if it recognized that there’s a cat in the image, and you know it’s right, you won’t care why it’s recognized that cat. I guess for some problems where the system made a decision that you don’t necessarily know why it made the decision, or you have to take action based on that recognition, you would want to know. So if I predicted for you that your revenue is going to increase by 20% in the next week, you would probably want that system to tell you, why do you think that’s happened, because there isn’t a clear reason for it that you would imagine yourself, but, if the system told you there is a face in this image, and you just look at the image, and you can see that there’s a face in that image, then you won’t have a problem with it, so I think it really depends on the problem that you’re trying to solve…

Q: 34

We talked about games earlier and you pointed out that they were closed environments and that’s really a place with explicit rules, a place that an AI can excel, and I’ll add to that, there’s a clear cut idea of what winning looks like, and what a point is. I think somebody on the show said, “Who’s winning this conversation right now?”

There’s no way to do that, so my question to you is, if you walk around an enterprise and you say “where can I apply artificial intelligence to my business?” would you look for things that looked like games?

Like, okay, HR you have all these successful employees that get high performance ratings, and then you have all these people you had to fire because they didn’t, and then you get all these resumes in.

Which ones more look like the good people as opposed to the bad people? Are there lots of things like that in life that look like games… or is the whole game thing really a distraction from solving real-world problems, nothing really is a game in the real world…

Yeah, I think it’d be wrong to look at it as a game because of the rules… first, there is no real clear notion of winning.

What you want is progress, you have goals that you want to progress towards, you want, for example, in business, you want your company to grow.

That could be your goal, or you want the profits to grow, you want your revenue to grow, so you make these goals because that’s how you want things to progress and then you can look at all the factors that help it grow.

The world of how to “make it grow” is very large, there are so many factors, so if I look at my employees, there might be a low-performing employee in one aspect of my business, but maybe that employee brings to the team, you know, a lot of humor that causes them to be productive, and I can’t measure that.

Those kinds of things are really, really hard to measure and, so looking at it from a very analytic point of view of just a “game,” would probably miss a lot of important factors.

Q: 35

So tell me about the company you co-founded, Anodot, because you make an anomaly detection system using AIs.

So first of all, explain what that is and what that looks like, but how did you approach that problem?

If it’s not a game, instead of… you looked at it this way…

So, what are anomalies? Anomalies are anything that’s unexpected, so our approach was: you’re a business and you’re collecting lots and lots and lots of data related to your business.

In the end, you want to know what’s going on with the business, that’s the reason you collect a lot of data.

Now, when today, people have a lot of different tools that help them kind of slice and dice the data, ask questions about what’s happening there, so you can make informed decisions about the future or react to things that are happening right now, that could affect your business.

The problem with that, is that basically… why isn’t it AI?

It’s not AI because you’re basically asking a question and letting the computers compute something for you and giving you an answer; whereas anomalies, by nature, are things that happen that are unexpected, so you don’t necessarily know to ask the question in advance, and unexpected things could happen.

In businesses, for example, you see a certain revenue for a product you’re selling going down in a certain city, why’s that happening?

If you don’t look at it, and if you don’t ask the question in advance, you’re not even aware that that is happening… so, the great thing about AI, and machine learning algorithms, is they can process a lot of data, and if you can encode into a machine, an algorithm that identifies what are anomalies, you can find them in very, very large scale, and that helps the companies actually detect that things are going wrong, or detect the opportunities that they have, that they might miss otherwise.

Where the endgame is very simple, to help you improve your business constantly and maintain it and avoid the risks of doing business, so, it’s not a “game,” it’s actually bringing immediate value to a company, highlighting, putting light on the data that they really need to look at with respect to their business, and the great thing about machine-learning algorithms, [is] they can process all of this data much better than we could, because what do humans do?

We graph them, we visualize the data in various ways, you know, we create queries from databases about questions that we think might be relevant, but we can’t really process all the data, all the time in an economical way.

You would have to hire armies of people to do that, and machines are very good at that, so, that’s why we built Anodot…

Q: 36

Give me an example, like tell me a use case or a real-world example of something that Anodot, well that you were able to spot that a person might not have been able to…?

So, we have various customers that are in the e-commerce business, and if you’re in e-commerce and you’re selling a lot of different products, various things could go wrong or opportunities might be missed.

For example, if I’m selling coats, and I’m selling a thousand other products, I’m selling coats, and now in a certain area of the country, there is an anomalous weather condition that became cold, all of a sudden I’ll see, I won’t be able to see it because it’s hiding in my data, but people will start buying… in that state will start buying more coats.

Now it’s not like if… if somebody actually looked at it, they would probably be able to spot it, but because there is so much data, so many things, so many moving parts, nobody actually notices it.

Now our AI system finds…“Oh, there is an anomalous weather condition and there is an uptick in selling that coat, you better do something to seize that opportunity to sell more coats,” so either you have to send more inventory to that region to make sure that if somebody really wants a coat, you’re not out of stock.

If you’re out of stock, you’re losing revenue, potential revenue, or you can even offer discounts for that region because you want to bring more people to your e-commerce site, rather than the competition, so, that’s one example…

Q: 37

And I assume it’s also used in security or fraud and whatnot, or are you really focused on an e-commerce-use case?

So we built a fairly generic platform that can handle a wide variety of use cases.

We don’t focus on security as-is, but we do have customers that, in part of their data, we’re able to detect all sorts of security-related breaches, like bot activity happening on a site or fraud rings—not the individual fraud of an individual person doing a transaction—but, it’s a lot of the time, frauds are not just one credit card, but somebody actually doing it over time, and then you can create or you can identify those fraud rings.

Most of our use cases have been around more business-related data, either in ecommerce, ad tech companies, online services.

And so online services, anybody that is really data-dependent to run their business, and very data-driven in running their business, and most businesses are transforming into that, even the old-fashioned businesses are transforming into that because that data has a competitive advantage, and being able to process that data to find all the anomalies, gives you an even larger competitive advantage.

Q: 38

So, last question: You made a comment earlier about freeing up people so we can focus on living in the stars.

People who say that are generally science fiction fans I’ve noticed.

If that is true, what view of the future, as expressed in science fiction, do you think is compelling or interesting or could happen?

That’s a great question.

I think that that, what’s compelling to me about the future, really, is not whether we live in the stars or not in the stars, but really about having to free up our time to think about stars, to think about the next big things that progress humanity to the next levels, to be able to explore new dimensions and solve new problems, that…

Q: 39

Seek out new life and new civilizations…

Could be, and it could be in the stars, it could be on Earth, it could be just having more time, having more time on your hands, gives you more time to think about “What’s next?”

When you’re busy surviving, then you don’t have any time to think about art, and think about music, and advancing it, or think about the stars, or think about the oceans, so, that’s the way I see AI and technology helping us—really freeing up our time to do more, and to use our collective intelligence and individual intelligence to imagine places that we haven’t thought about before…

Or we don’t have time to think about before because we’re busy doing the mundane tasks. That’s really for me, what it’s all about…

Q: 40

Well, that is a great place to end it, Ira.

I want to thank you for taking the time and going on that journey with me of talking about all these different topics.

It’s such an exciting time we live in and your reflections on them are fascinating, so thank you again…

Thank you very much, bye-bye.

Q-A: 1