Transcripts For CSPAN2 Brian Christian The Alignment Problem 20240711

Human learning. Hes best known for contributions documented in bestselling book, most human and algorithm to live by. Christian is taking questions such as we continue to rely on systems to solve our problem, what happens when ai becomes a problem. Computer science and poetry, christian has spent career tackling the ethical and technological implication of a society thats become more reliant on technology. The work has caught the attention of Technology Leaders like elon musk who said christians book was night table reading. How can we get a handle on this technology before we lose control . He argues that although we train to make decisions for us, eventually humans will need humans. We will be discussing a lot in the next hour and i want to have your questions too. If youre watching live with us, please put the questions on in youtube so i can work them into the conversation today. Thank you, brian, welcome, thank you for joining us. Its my pleasure, thank you for having me. Great. This is not your first book. I want to ask the obvious question and it does set the conversations here, why did you decide to tackle this topic now with this book . Great question. So the initial seed for this book came after my first book had come out and as you had mentioned vanity fair reported that elon musk, it was his bedside ading and i found myself in 2014 attending a Silicon Valley book group that was a bunch of investors and entrepreneurs and they had see the thing about elon musk reading the boo and they inted him to join and to my surprise, he came and there was this really fascinating moment at the e of the dinner when the organizers tnked me and evyone was getting up to go home for the night and elon musk foed everyone to sit back down and h said, no, no, no, but seriously, what are we gng to do about a im not letting anyone leave t room until you either give me a convincing counterargument why shouldnt be worried about this or you give me andea for something that w can do about it. D it was quite a memorable vignette, and i found myself, you kn, drawing a blank. I didnt have a convincing argument why we shouldnt be worried. I s aware of the conversation arnd risks of ai, for some people its like a Human Extinction level risk and other people are focused on the presentday ethical problem. I didnt have aeason why we should b word about it but i dnt have a concrete suggestion on what to do. That was the general consensus in the room at that time. Hisuestion of so seriously whats the plan, kind of haunted me while i was finishing my previous booknd i really began to see starting and would say around 2016 a Dramatic Movement within the field t actually put some kind of plan together both on the ethical questions and the sort of further further into the future safety questions. An both of those movements have grown i would say explosively between 2016 and now and these questions of ethics and safety and whats in the book i describe as t alignment problem, how do we make sure that the objective that this system is crying out is, in fact, what what we are intending for it to do. These things have gone from kind of marginal and somewhat philosophical questions at the margins of ai to really today making central questions of the field and so i wanted to tell the story of the movement and figure out where in a way answering elons question. Whats the plan . What are we doing . One of the interesting things as i was getting into it, theres a lot of obviously its very complex technology and program that goes into this but i dont think a lot of people are aware that this has already been applied even situations in our society today. We started the program, you present a number of examples of failures of ai tools to accomplish what they hope to perform but one of your examples could not be more timely. Thats the algorithms that are being used by judges not just here in california but specifically to what im getting into in california where instead of cash bail, a jge will use algorithm to determi whether or not a suspect is released or remains in jail while waiting for trial. Proposition 25 that wou we reaffirm a law andeplace with system andts a complicated issue but what really surprise med when i was looking at it, the naacp and human rights oppose this proposition because of whats theyre saying of inequities of the algorithm. Why don we get into that example and build from there. What happens with the algorithm and how did we get there . Absolutely. Theres nearly 100year long history of statistical in 1920s called actuarial methods, an attempt to create science of probation and parole and that as i say started in the 20s and 30s but really took off with the rise of personal computers in the 80s and 90s and today its implemented in almost every jurisdiction in the u. S. , municipal count state, federal. And there has been an increasing scrutiny thats come with that and its interesting watching the Public Discourse pivot on some of these tools. So, for example, the n york city w writing the new york city times Editorial Board w writing up to 2015 the lters, open letters saying its time for new york state to join the 21st century. You know, we need something thats objective, that is evidencebased, weant just be relying on the whims of folks in roads behind the bench. We need to bring some actual sciences. Shply, that that pition changes and just by months later the end of 2015, beginning of 2016ew york city is running a series of articles saying, algorithms are putting people in jail,acial bias, we need to really throw the brakes and callinout by name this particular tool whi is called compass which is one of the most widely used tools throughout the united states, and the question has ts really iited an entire subfield condition statistics around the question of what does it mean to s that a tool is fair . Younow, this system is designed toake predictions if someone will reoffend if they are released onrobation or pendg trial, pretrial. What does it meano take these concepts that exist in the law, things like, you know, treatment, equal oppornity, et cetera, 14th amendment protections, et cetera. What ds it mean to actually turn them into the language of codend how do we look at a tool like this and way whether we feel comfortable actually deploying this . And you are giving some examples of how, you know, a black suspect and a white suspect with similar background and how much more likely the white suspect was to go free including surprising at one point one of the white suspects who was what were and so what was goes into, if you will, the baking of that cake that built these biases because the biases were not intentionally baked into it, but theyre still hard baked in some ways. Yeah. So this is a very big conversation. Your likelihood to not make your court appointment. The second iso commit a nonViolent Crime while youre pending trial, andhe third is to commit a Violent Crime pending trial. Th question is where does the da come from on which these models are trained. And if you look at Something Like failure to appear in court, well, if you fail to appear in court, the court knows about i it by defition, right . So thats going to be fairly unbiased regardles of who you are. If you dont show up, the court knows about it. If you look atomething like nonViolent Crime, its the case that, for example, if you, if youoll young white men and young black men in manttan about their rate of lfreported marijuana usage, they selfreport that they use marijuana at the same rat and if you look at the arrest ta, the bck person is a 15 times more likely to be arrest ared for using marijuana than the white person i in manhattan. In other jurisdictions it might be 8 times as likely, i think in iowa. You know, it varies from place to place. To is tts a case where its really important to remember that the model claims to be le to predict crime, but what its actually predicting i rearrest. And so rearrest is kind of this imperft and systematically so proxy for what we really care about, which is crime. Its ironic to me because as partf this project, researching these systems, i went back into the historical literature when they fir startegetting used which was in illinois in the 1930s. At the time, a lot o the objections were coming from th conservatives, from the political right. And, yeah, ironically making almost the same argument that progressives are making now, but from the other pseudo. So conservatives in the lat 30s were saying, now wait a minute, you know, if this if a bad guy is able to evade arrest expect system dsnt know that he and the system doesnt know tha he committed a crime and will recommend the release and other people like him. Now we hear it bei made from the left which is to say if someone wrongfully arrested and convicted, they go into the data as a criminal, and it will recommend the detention of other people like them. Thiss really the same argument just, you know, framed in differentays. T at thats a very real problem. And werstarting to see groups like, for example, the partnership on a. I. Whichs kind of a nonprofit Industry Coalition at facebook, google and a number of groups, inact, a hundred different stakeholders recommending tha we dont take these predictions of nonviolent rearrest as seriously as we take, for example, the prediction of [inaudible] and the second component that want to highlight here, i mean, its a very vt question, b the second thing thats worth highlighting is this question of what do you do with the prediction once you have the conviction. So theres say youre, youve got a higher than average chance that youre going to fail to make your court, scheduled court appointment. Well, thats a prediction. Theres a separate question, which is what do we do with that information. Now, one thing you could do with that information is put the person in jail while they wait for the trial. Now, thats one answer. It turns out theres an emerging body of research that shows things like you send them a text message reminder, theyre much more likely to show up for their court appointment. And there are people proposing Solutions Like providing Daycare Services for their kids or providing them with subsidized transportation for the court. So theres this whole separate question which is as much as scrutiny is rightfully being directed at the actual algorithmic prediction, theres a much more system you can question which is what do we do wi those predictns. And if youre a judge and the prediction says that this person is going to fail to reappear, well, ideally youd want to recommend some kindf text message ale for this many as opposed to jail but that may not be available to you in that jurisdiction. So, you know, you have to kind of work with what you have x thats a system you can problem. Thats not necessarily an algorithm per se but the algorithm is sort of caught in the middle, if you will. Lets take it out of the crime d punishment era into the business area. You talk later in the book about about naudible] amazon coming up with this a. I. System that would help cull job applicants, and what they were finding was that a lot of men [inaudible] were baked into t way t system were being trained and th way the systemas being used. And i think also when you get to this, you still have the question at the end, well, why are we trying to findeople just he the people we had. Tell us aut that and how dud they get in how did they g in. Yeah. So, yeah, this is a story that involves amazon around the year 2017. But by no means are they a unique example, just happens to be the example of amazon. But they, like many companies, were trying to design something to take a little bit of the workad off of human recruiters. And if you have an open position, you start getting x number of residents coming in, ideally youd like some kind of algorithmic system to do some triage and and tell you, okay, thes are the resumes that are worth forwarding on or lking at more cloly, these are ler priority. And in a somewhat huge or ironic twist, amazon decided they waed to rate applicants on a scale of one to five stars. So so rating the prospective employee the same way customers rate their products. But to do, that they were using a tube of computational model called word vectors. Without getting too technical, for people or who are familiar with kind of the rise of Neural Networks, these Neural Network modelshat were very successful at Computer Vision arod 2012 also started to move into computational linguistics around 2013. And in particular, there was this very are remarkable family of mels that were able toort of imagine words as these points in space. And so if you had a document, you could predict the missing word based on e other words that were nearby in this kind of abstract threedimensional spas, if you can imagine that. But they had a lot of cool other properties that you can actually do arithmetic with words. You could do king minus man plus woman and search for the point in space that was neare to that, and you would get queen. You could do tokyo minus japan pl england and get london. These sorts of things. If is so these numerical representations of wds that fell out of this Neural Network ended up being useful for this surprisingly vast array of tasks. And e of these was trying to figure out the, quoteunquote, relevance a given tv to a given job. One wayou could do it i say here are all the resumes of the people weve hired over the years, find allhose points of space, and then for any new resume, letsust see which of the words have the kind of positive attributes and which have the negative attributes. Okay, well, it sounds good enough, but when t team at amaz started looking at this, they found all sorts of bias. So, for example, the word women was assigned as a penalty. So if you played, you know, you went to a Womens College or you on the Womens Society of something, the word women on that was getting aing negativ deduction. So yes. Because it was even farther away from the more successful words that its been trained to watch for, is that right . Thats right. It doesnt appear on the, you know, typical resume that did get selected in the past, and its similar to other words that also appeared. Thats right. So, of course, the team, the red flag goes off, and they say, okay, we can delete this attribute from our model. They start noticing that its also applying deductions to womens sports like field hockey, for example. So they get rate of that rid of that. And then they start noticing that its picking up on all of these, like,ery subtle syntaxical choices that were more typical of male resumes than female. Words like executed and captured, like i executed a stragy to capture market value, certain phrases that were just more typical of men. And at that pnt they basically gave up, they scrped the project entirely. And in the book i compare it to something that happened witthe boston symphony orchest in the 1950s where they were tryg to me the orcstra, which had been ver maledominated, a little bit more equitable. So they decided to hold the auditions behind a wooden screen. But what someone found out only later was that as the auditior walked out onto the wooden par parquet floor, of course they could identify whether it was a flatsoled shoe or a highheeled shoe. So it was not until the 70s when they instructed people to remove their shoes bore they entered the room thatinally started to see the gender balance in the orchestra start to balance out. And the problem with with these language models is they basically always hear the shoes, right . Theyre detecting the word executed, theyre detecting the word captured, and the teams in this case just gave up and said we dont fl comfortable using this technology. Whatever its going to do, its going to identify some very subtle pattern in th engineering resumes west virgiad had before weve d before, the gender balance is off this, so its just going to sort of replicate that into the future which is not what we want. So in this particur case, they just walked away. This is a very active area of research of how do you debias a language model. Youve got all these points in space, and you try to identify, you know, subspaces within this giant threedimensional thing which represent gender stereotypes. Can you delet those dimensions while preseing the rest. This is an active area, andts kind of ongoing to this day. How much amazon spend develong that in. Its a great question. You know, theyre pretty tightlipped about it. St of it comes from aeuters article where pple werent giving tir name, so i wasnt able to do a lot of followup. But as i understand, they actual not only disbanded the product, but they disbanded the team and redistributed them to engineer other things. So really washed their handsment yeah. I just however many, i assume millions, were p into that, you knowthey could have hired an extra [inaudible] to cull their stuff. Yeah, tts right. Well, another example i wanted to get io, a different angle, the definitely the selfdriving c. You talk about inhe book the fality that happened because the way the car was recognizing a person. Thats right. Again, explain that and what happen with. Yeah. So this was the death of elaine in tempe, arizona, in the 2018, the first pedestrian killed by a selfdriving car. The sort o r d uber vehicle. And the full kind of national highway Transportation Safety board review came out at the end of last year, so fortunately i was able to get some of that into the book before it went to press. It was very illuminating to read the kind of official breakdown of everything that went wrong because it was one of these things where probabl six or seven separate things went wrong. And you think had it only been but for that entire suite of things going wrong, it might have ended differently. One of the things that was the happening was it was using a sort of deep Neural Network to do object dection, but it had never been given an example of a jaywalker. So in all of the Training Data that this model had been trained on, people walking across the street were perfectly core lated with, you know,ebra strip. They were perfectly correlated with intersectio and so forth. So the model just didnt really know what it was seeing when it saw thisoman crossing the street in the middle of the street. And most object Recognition Systems are taught to classify things int exactly one of a discreet number ofategories. Is so they dt know how to classify stuff that belongs to more tn one category, they dont know how to classify something that seems like its in no category. This is one of the things theyve made headwayn only recently. In this particular case, the woman was walking a bicycle. And so this set the object recognition system kind of into this fluttering state where it first thought she was a cyclist, but she wasnt moving like a cyclist. Then it thought she was a pestrian, but it was recognizing the shape of the bicycle. Thent thought maybe its just some object thats been sort of blowing or rolling into t road. No, i think it is a person, no, i think it is a biker. And due to a quirk in the way th the system was built every time it cnged its mind about what type of entity it was seeing, it would reset the motion prediion. So it constantly predicting, you know, this is how a typical pedestrian would move, etc. , and extrapolating, o. K. , this is, as a result, where i think theyre going to be in a couple of seconds from the now, and if that intersects the car, its going to do something. But every time it changed its mind, it started recomputing that prediction, so it never stabilized on a prediction. So there were additional things here with overrides that the uber team had made because in 2018 most cars already have this very rudimentary form of selfdriving, automatically brake or in some cases swerve. They had to override e that to add their own system in, and those two systems interacted in weird ways. But i think to object recognition thing itself is, for me, very pneumatic, and theres this question of certainty and confidence that when a Neural Network says im 99 sure this is a person or whatever it might be, how do we know if those probabilities are well cl wrap ited, how does the system know what to do with them. I think many people kind of within the uncertainty community would argue that the mere fact that you are changing your mind should be a huge red flag, and you need to slow the car down. That alone. And that wasnt done. So, yeah, i mean, its very heartbreaking to think about how all of these engineering decisions add up to this event that, you know, would have been so much better to have avoided. I guess the Silver Lining is that there are lessons in there that are really taken the heart not just in industry, but also academia saying, okay, we really need to get to the bottom of this question of certainty uncertainty. Because i think thats a very human thing. You dont want i mean, this exists in the medical literature. You dont want to take an irreversible action in the face of uncertainty, you dont want to take a high impact action. You see also in the law in things kind of like, whats the term, like a preemptive judgment. Im forgetting the term, but a judge might issue an order in advance of deciding what the real thing would be because theyre trying to prevent some irreparable harm. And so theres the question for the Machine Learning community which is how do we take some of these same ideas, not wanting to make an irreversible choice in the face of uncertain few or in the case of a high impact situation, that requires us to quantify impact and quantify uncertainty and have a plan for what to do when we find ourselves there. So all those pieces need to come together, but were seeing progress being made on all of those fronts. And, yeah, it cant happen soon enough. In these examples that w talked about and in others that are inhe book and others that you didnt includ in the book but that you see, are the culprits the same . Is it the sam general problem that you think needs to be addressed . And then im going to ask do you think it is being addressed. I think theres one broad problem in the field known as the alignment problem, and thats where the book gets its title which is how do we make sure the objective in the system is exactly that which we want the system to do. And i think all of the examples that weve highlighted so far have shown us cases where one must be very careful to think about how to translate the human intention into an actual machine objective, right . We think we want to we think we can measure [inaudible] but we can only measure rearrest. You know, we think we can classify objects into different categories, but many objects belong in more than one category or we dont always know what category to put them in, and we have to act knowing that we dont know. So all of these things, and there are many other manifestations as well, i think speak to this fundamental issue of alignment. But the actual mechanics are different. Sometimes theres a problem with the Training Data, sometimes theres a problem with the model architecture so one problem we havent touched on yet this kind of black box issue of interpret about, explainability, how do we know whats going on inside a model, how can we trust the output and reverse engineer what about the input generate ared that output. There are questions of what is the socalled objective function of the system, what is the quantity were trying to minimize or maximize and how do we define that. So each component of the system has its own, you know, manifestation of the alignment problemment and each of them, to your second question, it is being addressed, yes. And that, for me, is really the striking thing that makes where we are now so different from, you know, where we were when we were elon musk cornered me and a bunch of executis in a room, and no one had any good ideas. I think were seeing a absolutely remarkable shift in the field that even just from i talk to one researcher, a ph. D. Student, who said wn he went to the biggest industry conference in 2016 and said he worked on a. I. Safety, people ki of raised their eyebrows at himike he was a little bit kooky or paranoid or something of. He came back a year later in 2017 and theres like an entire daylong workshop or a. I. Safety, a b 2018, its a significant fraction of what was presented at the conference. So in absolute numbers, the amount of peoe working on this is still quite with small, b the growth even over that short time has been, to my mind, astonishing. And, you know, i think we cant, it cant come soon enough. So i encourage all motivated, you know, undergrads and High School Students to get excited about it. Theres a lot of work to be done. In talking about learning and development thats going on on the a. I. , in the a. I. Research and development field, is the actual commercialization of the technology then ahead of where it should be . I mean, and, you know, should this also be, all still be modeled and not put on our roads, put in our courtrooms . Thats a great question. Yeah, in some ways, the criminal justice stuff has an 85year history at this point, but its as if were still playing catchup. So you can think of it as a kind of race. Can the understanding catch up to the actual impmentation. And i think weve seen that with social media, you know . There were some decisions that facebook made abo how to run their news feed rankingal grut. , and the details are somewhat tenical. They went from supervised learning to reinfcement learning. We dont have to unpack that exactly, but basically the narrowmindefocus on always priotize the content that will get the most clicks on that thincreated a situation where extreme contentas being promoted, people were burning out, people were leaving the platform in addition to the many other kind of societal externalities that that was creating. And they were automobileo replace it with a able to replacit with a more advance model that you could burn someone out or they would start to distrust the platform,tc. And fittingly, you could note that the point of that model was to maintai user retention and, you know, these things that were considered [inaudible] i think there really is a question of when we think about the alignment problem, i the system doing what we want. I think whe you looked at actual industry, there the meta question which is whats it that we want the system to be doing, and whens we in that sense as and whos we in that sentence as well. And i think those questions are going to loom ever larger. We have seen in general media kind of more urgency put, you know, the past few years when this topic does come up in one way or another, theres more urgency behind the people who are saying we need to thinking about this. And because, as you said, its being rolled out, one of our audience member asks about widespread facial recognition and possible orwellian, mentioned 1984 implications. In the book you talk about facial rognition technology and the inappropriately funny result of it which was just absurd but also insulting and, obviously, today has worse implications. But could you talk a bit about facial recognition, another thing that, in fact, here in lifornia became, i think, a proposition about whether or not to use this technology. Ing so tell us about that and how it fits into what youre talking about. Yeah. I mean, so this is coming through the legal system now if im remembering correctly. I think the first case in, i want to say minnesota or wisconsin, of someone being arrested but being incorrectly identified in a facial recognition database. So a lot of this stuff is going through the court system and probably heading for the supreme court. On the technical side, yeah, theres this, there is this really unfortunate and kind of hard to ignore pattern in particular of ethnic minorities being incorrectly recognized or categorize, etc. , by facial Recognition Systems. And, you know, one of the famous examples was with the Software Developer in 2015, a group of photographs that he had taken of friends captioned by Google Photos as guerrillas. Another example an mu e t researcher mit researcher when she was an undergraduate Computer Scientist doing these facial recognition homework assignments, she had to borrow her roommate in order to check and make sure that the face system worked because it didnt work on her. Or it worked on her only when she wore a white mask. And this really set off an investigation of why, why does this keep happening, whats the underlying thing. And there are ooh there are a couple different components to it e, but i think one of the main ones is there have been, i think, a preexisting lack dazal attitude to how these [inaudible] were put together in the first place. So, you know, part of what led to the rise of computer recognition was the internet, and suddenly if you needed half a million examples of a human face in order to train your systemming, well, in the 80s you were totally out of luck. But now that we have the internet and Google Images and all these things, you can just download a million faces and put it into your system. The question is which faces are you downloading . So the most Popular Research database for many years was one developed in the late 00s called faces in the wild. And they thought, okay, what we want to do is understand if, in fact, two faces are the same person. So they had this clever ideas, well scrape newspaper headlines or newspaper images because theyre all labeled with this person a this person and this person, and that withdraw well have ts giant data set. We can decide are the two people the same image or whatever. Theroblem is youre at the mercy of who was in the photographs in the late 2000s, and the answer is president george w. Bush. And, in fact, an analys of the labeled faces in the wild data set that was done just a few years ago show there are twice as many pictures of george w. Bush in the database as all black women combined. Which is just insane. I mean, i youre trying to build something out t be fair to the people who, you know, collected that data, they werent this is kind of an acadic research project, this was not intended to be used in any actual system. But these data sets have a way of kind of sticking around. Someone just downloads it off the internet, or its the most cited thing, so if you want your paper to be cited. And its very striking if you look at the original papers, and i dont want to single them out because this is widespread, the word diversity is getting used in the start of late 00s, early 2010s to mean lighting and pose. So theyll say this is the most diverse data set assembled to daut, but what they mean is we have people from the time, people in the dark or people that are lit weirdly. Now at the beginning of the 2020s, its very striking some of these come databases now appear with a warning label on it that says when we said diversity, we meant a very specific thing, and we want to flag it as very much not diverse in the sort of demographic sense. So theres a lot of work thats being done there, spearheaded by people like joy at mit and [inaudible] at google brain to bring a lot more focus on equalizing the error rates across differentth nick groups, making sure that the ethnic groups, making sure9 that the Training Data actually respects the population that the molds going to be used on and thinking about the representation of tech itself. And so i think in 2019 only less than 1 of Computer Science ph. D. S were africanamerican. And so theres a lot of work to be done in the field itself to address that question of representation. And so were seeing groups like black a. I. Which had a number of initiatives including scholarships and grants trying to equalize that not, as i say, not just in the Training Data, but, indeed, in that field, so so the question from the audience about, you know, kind of maledominated answers being baked into the design. In the book while youre discussing debiasing word and setting, moving gender [inaudible] you mentioned a work that a group of researchers did im going to mispronounce his name yeah. And adam [inaudible] yes. Yeah. And you wrote about [inaudible] social bias, unquote. A requirement for any of his work because of all the different interpretations one has to take in and understanding the of the social science that has to somehow be mixed in with the of the Computer Science, right . I thinks absolutely right. And that, to me, is one of the really strike thing things about where the field finds itself at this moment. No longer can Data Scientists and Software Engineers think of emselves as purely doing engineeringr purely doing mathatics. That we have just gotten to a point where these systems are absolutely enmeshed in kind of human practices of how is the data collected, how is the data generated. The question thathe human respondents wereeing ask, how is it worded because youre going to get different answers based on how it was worded, and what population of people were you sampling it from. So, you know, are people on amazon mechac cl tur presentative or not representative of other groups that, you know, might respond to the same thing. So were very mucin this ment of, to my mind, kind of exhilaratingly interdisciplinary work tt needs to happen and is happening. Between the Computer Science machinelearning community and social scientists, philosophers,th cysts ethicists, lawyers, cognive scientists, theres a l of really interesting work bng done at the intersectn of a. I. And infant cognition. Were now sort of at the point where a. I. Essentially resembles an infant,nd so the machinelearning is going to developmental psychologist and saying whats your best theory for, y know, the curiosity of an infant or the novelty bias that small kids have or the exploratory play that kids will have with a toy t figure out how it works. Because we needo um port that into our a. I. System to solve this problem. And in turn, the a. I. System might be unlocking some of these questions a to why thenfants is these sorts of drives. So there are many, manyronts on which the social sciences, i think, are kind of uniquely positioned at that interface with Computer Science. Were seeing manyore papers with a really diverseet of skills among the authors, and i think thats the kind of thing, to me, thats very encouraging. Well, as we mentioned at the gunning, you have an interest beginning, you have an interesting skill set yourself, author, poet, programmer. I expect. The venn diagram is barely touching circles. But did that the set of skills help you maybe see out of some blinders that might otherwise have affected what you think . Yeah. I mean, you could joke that poetry e and programming have in common, you know, a sense of scrutiny over semicolons. But beyond that, yeah. I mean, my background also is philosophy. When ias a student, i was really intereste in this question of what i thinking, what does it mean to have a mind, be conscious, etc. And philosophy of min sort of takes one angle on that question. And a. I. Answers it in a different way by sort of trying to actually make it. And so i think, yeah, broadly speaking, you kw, its been a question for 2500 years of western philosophy what does it mean to human, what makes us unique and specialnd distinct, and, y know, aristotle t day cart or and so forth have considered ourselves animals, and i think theres nev been a more interesting time to be thinking abouthis area because now is hav a completely new standard of comparison, the computer, and we get a totally different set of answers, you know . So aristotle and descartes end up decing analytical reasoning is the cor of what it means to be human because that what dogs and monkeys cant do. I dont think who tnks seriously about a. I. Thinks deductive anatical reason is the [inaudible] of human eerience. You get a different kind of answer morebout empathy, imagination, social ties, teamwork, collaboration, etc. So, yeah, for me, i mean, think in some ways i feel very lucky that i have this very eclectic set of interests and happened to be alive at the time in Human History where these two disciplines are on an absolute collision course. Someone in our audience asks do you believe that humanlike machines will ever be achieved . In other words, single entity. There are a number of things that people mean by the word singularity. So theres some people for whom it means something that is also called the hard or takeoff where theres kind of an abrupt moment in time where a. I. Starts selfimproving and just blindsides us. I dont necessary i dont really see that. Im in the camp of what is sometimes called the slow takeoff where i think a. I. s just going to be getting weirder and spookier and more uncanny until we just accept that it does everything we would, you know, require a thinking or intelligent thing to do, but that there wont be a sort of sharp elbow turn where this, like, suddenly happens overnight. Its, from my perspective though, its totally inevitable. I mean, theres a long history within Computer Science going babb to, you know, the 1940s Claude Shannon was asked do you ever think a machine can think . He said, of course, im a machine, and i can think. And if you have that kind of secular world view, you know, a nondualist, and you think, well, the brains made of atoms, computers are made of atoms, its a surgeon level of complexity certain level of complexity, and i think thrillingly and sport of spookily, were on that road map. Open a. I. Just introduced a language system a few months ago that has 175 billion parameters. Well, if you compare that to the number of synapses in the human brain, its about oneone thousandth of the complexity of the human brain. Doesnt sound very impressive, u know . 0. 1 . But the average model size within that field of a. I. Is doubling every three months. Sof you do the math, that means that we should expect models to exi that he the synaptic complexity of the human brg sometime in the spring of human brain sometimen the spring of 2023. Thats not very far away. Somehow i thi its going to really srt to come to the surface. I dont know what the answe will be, but this is, i think, one of the really riveting things about this moment. Yeah. There was a story recently about the two computers that i forget the details, they spontaneously developed their own lguage for communicating with each other that was faster than [inaudible] and in talking about just the need that we have for a. I. To develop in certain ways, you know, to align with what we want it to do and what our needs are, thinking about [inaudible] solving a proem in a completely different w. Is it possible that we will get Artificial Intelligence that is much more advanced and that is able to deliver usesults tha we then are more understandable, re in alignment with what we want to see, but that their way of reaching it will be totally alien . Thats a great question. I think, i think both of those possibilities are live possibilities. Finish i think one of the things that sometimes forgotten when we talk about a. I. , theres a certain lens of probability that we can cast onto this and so forth. But there are real choices to be made about the architecture of these systems. For example, i mean, its already the case with selfdriving cars, for example, that the you can do whats called training the systemmened to end. You can just have a giant blob of Neural Networks, and you put the cam a feed in the bottom camera feed in the bottom, and the Steering Wheel commands come out the top, and you have no idea whats going on in the middle. Theres an increasing science to the question of how do you pop the hood and figure out what is going on in the network, but also how do do you constrain the network in certain ways . For example, can you constrain it to be modular so that the system is naturally divided into these subcomponents that you can analyze individually . Say, okay, i know what this thing is doing, so let me now worry about whats going on over here. Theres a lot of really encouraging results in that space. And so i think to your question will a. I. Be able to do what we want but in a way thats totally inscrutable, yes. But thats not necessarily the only way that can happen. And so i think we will have some more agency there than i think is sometimes appreciated to build the kinds of systems that we feel we can trust. Wn you talk about artificial intligence, how many peopl ask you about terminator . You know, its funny. Its been verynteresting to watch the questions i get you know, evolve. And when my first book came out in 2011, 2012, a lot of pple were asking me, you know, is a. I. Coming for my job. By 2014, 2015 people were asking me is a. I. Going to destroy all of humanity as we know it. So the stakes had really gone up there. And within the research community, the cautionary tale has kind of shifted from one that feels more like, yeah, sort of a disobedient system to one that feels more like a system i use the analysis in the book of like the sorcerers awe prentice awe prentice. Theres a famous thought experiment that comes from a Research Institute of whats called the paper clip maximizer. You know, its fictional [inaudible] imagine a paper clip factory, you buy this a. I. And you say we really want to increase the output of our paper clips. Unfortunately, this thing is so good that it basically turns the entire universe into paper clips including yourself and your loved ones. So, you know, thats a little bit caricatured, obviously, but that is, i think, the kind of thing that people that work on alignment are worried about. Its not a system going rogue, its not a system deciding that humans need to be exterminated, its a system in a sort of poignant way trying to do what it thinks we want it to do but realizing, you know, we realize perhaps all too late that we werent quite specific enough, you know . And, again, the sorcerers ace prentice, ng missouri das, one of missouri das, one of these things. Part of what solving that alignmen problem would mean is feeling comfortable commune if candidating an intention to a system like that communicating an intention hike that without needing to get every specific detail right before pressing the butn. We have a system tt has a flexible the scientific term is corrigible, but it can adapt on the fly it takesback. We can say takes feedback. We can say, hold on, thats not what i meant. And that is the, i think, the kind o thing that makes mysel and a lot of people who work on this area a little bit more relaxed than we were three, four years ago. Ne of our audienc members notes that s a former computer engineer 3060 years ago. Systems analysts, not ogrammers. He asks what do ty want to be called today. I want to turn that though and ask since you were talking about how the field now is becoming much more interdisciplinary looking to learnings and input from other fields, are the people who are in the a. I. Field changing . Are they, are there people from other fields who are like, ah, i want to go into that field or, you know, are the people from other majors switching over into this different than they were ten years ago . Thats an interesting question. Like, the initial point about being a systems analyst, i was thinking about the head of a. I. At tesla, andre, has in this notion of what he calls software 2. 0. Software 1. 0 was programming, you know, line one. Is x then line two do y. Software 2. 0 is the world were in now with Machine Learning where you dont write code. You provide a set of training examples and say do Something Like that. Theres sort of a debate over whether theres going to be a software 3. 0 which is thi totally general a. I system that you just have to make the question. You dont give explicit Training Data, but you work with it after the fact,nd gbt3 is sort of in that categy. And working withpt3 feels a lot more like [inaible] where youre writing aost like an essayrompt. So its a language model that degned to, essentially, just fill in the blanks in the text. You can use it to do all sorts of things. You can say theollowing is a Python Program [inaudible] list of integers, bla, and it wi break the code for y. You can say the following is an argument that the most common objections against x. And then it will give you some, you know, little essay. And what it mea to sort of use a modelike thattarts to feel a lot more like how t work with another person. How do you word something so that, you know, the meaning comes through or you get the tone that you want or the style that you want, etc. , etc. So there is, i think, going to be this new category of people who are not exactly programmers, not exactly Machine Learning people or statistics people, but people who are sort of wrangling the giant models through natural language. And thats kind of a new job that doet really exist yet. And thats going to require, i think, a very interesting set of skills. Its going to require, you know, certailinguistty capability because the wordshat you use are ve important. Its also going to require intuition out how these large models are train thatll help you figure out why it might not be doing something you want. More broadly, i think as these questions as the questions that Machine Learning i dealing with bece more and more human, it does invite a ctain type of person who might not have felt like they belonged to now feel like they do belon and fool like their skull set can feel like their skill setan plug into that and their interests can plug into that. I think that is a shift that is really just beginning. Who are you trying to reach with this book . Who are you hopping will read it hopi will read it . Is i the general aience . Is ithe a. I. Commuty . Is it the comnies and entities that are trying to use this technology . Thats a great question. I think there there are a couple audiences. One is just the general public. I think relative to other fields in science, a. I. , machine lening and even these questionof bias, ethics, safety have been sufficiently in the press tha a lot of people are aware of them whether or no theyve really taken the time to kind of go deep and understand the underlying issue. And so part of what i want to do is say this debate is already happening, i wanto try to raise the level of the debat give people some of the con accept p chul insights conceptual insights,ome of the basic vocabulary so they can feel comfortable talking about it. I i think it also, you know, theres a huge class of people who went through their career with a particular kind of training tt never seemed like it required them to know about Machine Learning. Maybe youre a judge. Now suddenly in the year 2020 youre being hande these gorithmic [inaudible] you a diagnose diagnostician aw youre being theres a lot of people out there that suddenly need some level of familiari in this area, so i think the book can fill a need there as well. And then lastl i hope it can just grow the field. I genuinely think its one of the most exciting and most important things thats happing not just in Computer Science, but in science period. And so if i can reach, you know, undergrads and get them excited about this area, get them bugging their advisor, give me a cool project i cano on safety, you know, what can i do, lets get started, i think that, for me, wl feel really good, if i can grow that fie and bring ev more books into the moment. I think thatll be a very good thing. Weve got time for just one more question, so this is the time travel question. In the book you, of course, talk about some of the earliest steps in developing a. I. But now 20 years in time, what would you expect to see the state of a. I. Both in terms of research and direction as well as in actual deployment [inaudible] twenty years is interesting. I mean, theres a joke in a. I. That, you know, starting in 1955 a. I. Was always 20 years away and still is. But i think, i mean, realistically it will, there will be sort of, of course, a generational replacement that happens over that time. And i think the way that we now have people who we wehink of as kind of digital nives, social media natives, theres going to be a generation of, like, a. I. Nates that, y know, thatration of people that are just being that generation of people just being born now, they will grow up in a world where they may not get drivers licses. 9 it may become illegal, you know, morally outrageous, how can you justify a human driving a car, you know, its so dangerous. And they will, i think, come to understand themselves as inhabiting a world in which there are allhese different systems of which have different degrees of what you might call intelligence, differen degrees of what you might call agency, differt incentives that ahine with our o align with our own to one degree or not and interfaces that increasingly look like the way people talk to each other, you know . Kids have no problem talking to alexa, for example. They find it fairly normal that theres some kind of system that you can just chat with, which is sort of remarkable even if you think back ten years. At was totally not something anyone was familiar with. So, yeah, i think in se ways the bounry will start to get blurry between that technical skill set and just the skill set of navigating the world of other humans because these systems will sort of increasingly start to feel like they kind of speak that same lguage, you know . You can communicate by gesture, oh, younow, you sort of reach for something and youre like, no, no, no, the other one. Or you communicate i words, i think were going to have this new generation for whom that just is the way the world works and hopefully, we will set them up to have a reasobly good world, indeed,t that pnt. Very good. Well, i thank brian christian, author of the alignment problem, i thank you f joining us today. Wed also like to thank our audience f watching and participating online. If youd like to support the commonwealth clubs effort, please visit commonwealth club. Org online. Im john zipperrer, thank you, iwishing you a good day and hoping you stay safe and healthy. Thanks a lot, everybody. Heres whats coming up this evening on booktv in prime time. Youll see the zillionth annual seventh annual kirkus prize. Adam huggen bottom talks about higgingbotham talks about the disaster in ukraine. John lynn say on whether technological advances in military technology are helping or hindering our soldiers in combat. Deborah stone argues that numbers arent objective and explains numerous ways that figures impact our lives daily. And a Data Scientist looks at the history of census taking around the world. That all starts this evening at 6 45 eastern. For more schedule information, visit booktv. Org. Actress and activist jane fonda reflects on her efforts to speak out on environmental issues. In this portion of the program, she offers her thoughts on the astronaut of the federal government. We also have to really dig deep into ourselves and figure out who we are, who do we want to be. Yeah. And we have to fundamentally and i hope we will, and i think we will we have to change the way we think and feel and function and and learn to care for each and not let these dog whistling politicians who really dont care about us at all lead us down a dead end road, which is whats happening now. But, you know, i always tend to look at the bright side. Covid has, covid didnt break us. Covid exposed where we were already broken. And so people saw things that i dont think they were aware of. I think they didnt realize how our federal government is, has been so weakened and crippled by the guy thats in the white house right now. And when youre facing a pandemic and a climate crisis, you need a strong federal government thats coordinated and strategic and prepared. And people are now faced with what happens when you dont have that. Thats one thing. Another lesson from covid is Pay Attention to the experts. The medical experts and the scientist which has not been happening, and i think people see what happens when you dont do that. And then i think that people are seeing the essential workers, you know, the farm workers, the nurses, the domestic workers, the delivery people, all the people that make our lives function that are missing so much and getting so little in returnment clapping for them in the evening isnt enough. We have to really fight for them. Not just for now, but in the future that theyre able to earn a decent living so they can support themselves. All these things that reflect on who we are as a people are happening at once. And i think that were being shaken awake, so i feel very hopeful. To watch the rest of this program, visit our web site, booktv. Org, and search for jane fonda or the title of her book, what can i do, using the search box at the top of the page. Heres a look at some Publishing Industry news. Former president barack obama released the first of his twopart memoir this week. The book, titled a promised land, covers president obamas early political career, his road to the white house and part of his first term. The books publisher, crown a division of pension one random house has printed 3. 4 million copies for north america and an additional 2. 5 million for the international market. Publishers weekly spoke to editors and other publishing executives about a potential president ial memoir by president trump. Many suggest that he would seek an advance that rivals the 65 million that former president and first lady obama received for their respective message hours. Memoirs. In orr news, New York Times White House Correspondent Maggie Haberman writing a book on her 20 years of covering president trump. The book will be published by Penguin Press and will be available in early 2022. Also in the news, several of americas larger publishers reported sales gains for the Third Quarter of the year that ended september 30th. Simonsimon Schuster Saul a 28 saw a 28 increase, Harper Collins saw a 13 jump in sales. And James Patterson was the best selling author of the last decade. His total sales were higher than stephen king, David Baldacci and john grisham combined and made up for one in seven books sold in the thriller genre. Booktv will continue to bring you new programs and publishing news. You can also watch all of our past programs anytime at booktv. Org. Welcome to the National World war ii museums evening presentation webinar. My name jeremy collins, and for those of you watching on zoom, some brief housekeeping remarks. You are an attendee of our zoom vent tonight, that means you do not have video or audio privileges, but you can interact with our moderator and guest by writing your question in the q and a box. The moderator will be reviewing those during the question and answer session which will conclude tonights program. And now to introduce the moderator, its my pleasure to pass this program over to dr. Rob citino. Rob . Thanks,