Transcripts For CSPAN2 Brian Christian The Alignment Problem 20240711

The Commonwealth Club of california. I am john zipperer, Vice President of media and editorial and a moderator for this program. We would like to thank our members, donors and supporters for making this and all of our programs possible. We are grateful for their support and we welcome others joined them and supporting the club drink these uncertain times. I am please be joined by brian christian, government author of the new book the alignment problem Machine Learning and human values. His best offer to first get a contribution to the Tech Industry and document in his best bestselling book the most human human and algorithms to live by. In his new book he poses questions such as every content neutral on Artificial Intelligence to solve our problems, what happens when ai itself becomes a problem . With degrees in philosophy, Computer Science and poetry, christian has spent his career tackling the ethical advancing society and become more reliant on technology. His work is got the attention of Technology Leaders such as elon musk once said of his 2011 book, the human human was his night table reading for its new book the alignment problem Christian Lazar depends on ai systems and questioned whether not it can help us. How can we get a handle on technology before we lose control . He argues although we train the system to make decisions for us eventual human when the humans we will be discussing about in the next hour i want to your questions. If youre watching live please put your questions in the text chat a few tips i can work them into our conversation today. Thank you brian. Welcome. Thank you for joining us. Its my pleasure. Thank you for having me. This is not your first book of course. I want to ask in the opening question the obvious question but i think is good job of setting the place for conversation that is why did you decide to tackle this topic now with this book . Great question. The initial feed for this book came after my first book and come out and as you would mention vanity fair reported that elon musk was his bedside reading and i found myself in 2014 attending a Silicon Valley book group that was a bunch of investors and entrepreneurs, and they had seen this thing about elon musk reading the book and invited him to join. To my surprise he came. There was this really fascinating moment at the end of the dinner when the organizers thanked me and everyone is getting up to go home for the night, and elon musk force everyone to sit back down and he said, no, no, no. Seriously, what ever going to do about ai . Im not let anyone leave this rheumatoid you either give me a convincing counter argument but we shouldnt we developed this or give me an idea for something we can do about it. It was quite a memorable vignette. I found myself drawing a blank. I didnt have a convincing argument why we shouldnt be worried. I was aware of the conversation around risk of ai. For some people its like a Human Extinction level risk. Of the people are focused more on the present day ethical problem. I didnt have a reason why we shouldnt be worried but it didnt have concrete suggestion for what to do. That was the general consensus at that time. His question of so seriously, whats the plan . Kind of haunted me while i was finishing my previous book. I begin to see starting i would say around 2016 a Dramatic Movement within the field to actually put some kind of plan together, both on the ethical questions and the sort of further into the future safety questions. Both of those movements have grown explosively between 2015 and now. These questions of ethics and safety and what in the book i describe as the outline a problem, how did make sure the objective the system isarried out is, in fact, what we are intending for it to do. These things have gone from marginal and somewhat philosophical questions at t margins of ai to really today making up the central question in the field. I wanted to tell the story of that movement and figureut in a way answering his question, whats the plan, what are we doing. One interesting things as i was getting io this was theres lot o very compl technology and program the goes to this b i dont think a lot of people are aware of how this is being applied in some even lifeanddeath situation in our society today. We were talking bore the start of the program that you psent a number of examples of ai tools to a conference with their hoping to perform one of her examples connecting more time and thats the algorithms being used by judges not just in california but, instead of cash bail a judge will use this algorithm to determine whether not a suspect is released or remains in jail while awaiting trial. This fall california has ballot proposition 25 that would reaffirm a law that would do away with cash bail and replace with this algorithmbased system. Its a very, get issue but what surprised when its looking at it was the naacp and Human Rights Watch proposed this proposition because of what theyre saying building inequities of the algorithm. Why dont we get into that example and build from there. Whats the problem with the algorithm and how did we get there . There is a nearly 100 year long history of statistical what were in 1920s called actuarial methods in her role. This attempt to create a science of probation and parole. That as i say started in the 20s and 30s but really took off with the rise of personal computers in the 80s and 90s. Today its implemented in almost every jurisdiction in the u. S. Municipal, county, state, federal. There has been an increasing scrutiny that is come along with that and its been interesting watching the Public Discourse have it on some of these tools. For example, the New York Times was writing the New York Times Editorial Board was writing up through about 2015 these letters saying its time for new york state to join the 21st century. We need something that is subjective, that is evidencebased. We cant just be relying on the whims of folks in robes behind the bench. We need to bring some kind of actual science to this. Sharply that position changes and by just months later the end of 2016, the New York Times was running a series of articles saying algorithms are putting people in jail, algorithms have seeming racial bias. We need to throw on the brakes. Calling out by name this particular tool which is called compass which is one of the most widely used tools throughout the united states. The question, this really has ignited an entire subfield within statistics around this question of what does it mean to say that a tool is fair . This system is designed to make predictions about whether someone will reoffend if they are released, on probation or pending trial, pretrial. What does it mean to take these concepts that exist in the law, things like disparate treatment or equal opportunity, et cetera, 14th amendment protections, et cetera. What does it mean to turn them into the language of code and how to look at a tool like this and say whether we feel comfortable deploying this . You were giving examps of how a black suspect and a white suspect with similar crimes, similar backgrounds andow much more likel the white suspect was to go free, including surprising one of the white suspects it was [inaudible] what goes intohe baking of that cake that build devices in your question advices were intentionally baked into it but they are still hard baked in. This is a very good conversation. One placeo start is to look at e data that goes into the systs. One of the things that the syems are trying to do is predict one of three things. Lets think about a pretrial case for now. Typically a tool like compass is predicting three different things. One is your likelihood to not make your court appointments. The second is to commit a nonViolent Crime while your pending trial here if there is to commit a Violent Crime awaiting trial. The question is where does the data come from which these models are trained . If you look at Something Like failure to appear in court, if you fail to appear in court the court knows about it by definition, right . Thats going to be fairly unbiased innocent regardless of who you are. If you look at Something Like nonViolent Crime, its the case that, for an example if you hold young white men and young black men in manhattan about their rate of selfreported marijuana usage, they selfreport the use marijuana at the same rate and yet if you look at the arrest data, the black person is 15 times more likely to be arrested for using marijuana in the white person is in manhattan. In other jurisdictions it might be eight times as high in iowa. It varies from place to place. Thats a case where its really important to remember that the model claims people to predict crime but what is predicting is we arrest your so we arrest is this imperfect and systematically so proxy for what we really care about which is crime. Its ironic to me because as part of this project researching the systems i went back into the historical literature when they first started getting used, which was in illinois in the 1930s. At the time a lot of the objections were coming from the conservatives come from the political right. Ironically making almost the same argument progresses are making abut from the other side. Conservatives in the late 30s were saying wait a minute, if aa bad guy is able to evade arrest and the system doesnt know he committed a crime and the system treats him like he is innocent and will recommend his release and recommend the release of other people like him. Now we hear it being made from the left which is to say, someone is wrongfully arrested and wrongfully convicted they go into the Training Data as a bad person, as a criminal and it will recommend the detention of the people like them. This is the same argument just frank in different ways. Thats a very real problem and we are starting to see groups like, for example, the partnership on ai which is a nonprofit Industry Coalition of facebook, google and a number of groups, almost 100 different stakeholders, recommending that we dont take these predictions of nonviolent rearrest as seriously as we take for example, addiction of failure to repent. The second come over i want to highlight, its a very fast question but the second think thats with high sliding is this question of what you do with the prediction once you have a prediction. Lets say youve got a higher than average chance that youre ing to feel to make your court, scheded court appointments. Thats a prediction. Theres a separate question which is what do we do with that infoation. One thing you could do is put the person in jail while they wait for the trial. Ats one answer. It turns o theresn emerging body of research that sws things like if you send him a text message reminder, they are much more likely to show up for their court appoiment. There are people proposing Solutions Like providing day care servis for the kids or providing them with subsidized transportation to the court if thats an issue. Theres this who separate question which i as much is going on, as muc scrutiny is rightly directed at the actual algorithmic prediction, theres a much more systemic question which is what do we do with those predictions . If you are a judge and the predictions that this pern is going to fail to reappear, ideally you would want to recommend some kind of text message alert for thes opposed to jail. That may or may not be available to you in that jurisdiction. You have to that systemic problem. Thats not an algorithm per se and algorithm is sort of caught in the middle if you will. Lets take it out of the crime and punishment area into the business area. You talk later in the book about hiring. Amazon has come up with this ai system that would help it cull job applicants in with your fine is they were disproportionate a lot of men. The reason for this also worth baked into way the system was being trained in the way the system was being used and also when you get to the questions at the end of like why were you trying to find people who are just like the people you had . Tell us about that and how did they get so yeah, this is a story that evolved amazon around the year 2017. But by no means either unique example. Just happens to be the example of amazon. They like many coming for trying to design a system that could take a little work load off of human recruiters. If you have an open position you start getting x number of resumes coming in. Ideally you would like some kind of algorithmic system to triage and tell you these are the resumes that are worth forwarding on looking at more closely, and these are lower priority. In a somewhat cute or ironic twist, amazon decided they wanted to write applicants on a scale of one to five stars. Rating perspective points the same way amazon customers rate amazon products. But to do that they were using a type of computational language model called word and beddings and beddings. Without getting too technical for people are familiar with the rights of Neural Networks, these Neural Network models were very successful at Computer Vision around 2012 also start to move into computational linguistics around 2013. In particular there was this very remarkable family of models that were able to imagine words as these points in space. If you had a document you predict a missing word basin of the words nearby and abstract 300 dimensional space if you can imagine that. These models had a bunch of cool other properties. You could you arithmetic with words. You could do king minus man plus women and search for the point in space nearest to that and you would get queen. You could do tokyo minus japan plus england and get london. These sorts of things. These numerical representations of words that fell out of this Neural Network into that and useful for this surprisingly vast array of tasks. One of these was trying to figure out the quoteunquote relevance of a given cd to a different job. When we could do is just say there are all the resumes of people we heard over the years, throw those into this work model and find all those points in space and for any new resume lets just see which of the words in that cv have the kind of positive attributes in which have negative attributes. Sounds good enough. When the team at amazon started looking at this they found all sorts of bias. So, for example, the word women was assigned a penalty. If you went to a Womens College or on the Women Society of something, the word women on that cv was getting a negative deduction. Getting a negative rating or whatever becausets located farther away from the more successful word that it has been trained to watch for, right . Thats right. It doesnt appear on the typical sume that did get selected in the past and it i similar to other words that often appear. Of crse 15, the red flag goes off and they say we can delete this attribute from our model. They start noticing thatts also applying deductiontill like womens sport like field hockey. They get rid of that. Or Womens Colleges, smith college, so they get rid of that. And then they start noticing its thinking about all of these very subtle syntactical choices that were more typical of male engineering resumes than female. For example, the use of the words like executed and captured, like i executed a strategy to capture market value or whatever. Certain phrases that were more typical of men. And at that point they basically gave up and scrapped the project entirely. In the book i compare it to something that happened with the boston Symphony Orchestra in the 1950s where they were trying to make the orchestra of which a been very male dominated a little more equitable so they decided to hold the auditions behind a wooden screen. But when someone found out later was that as the addition or walked out onto the wooden parquet floor, of course they could identify whether it was a flat you or a high future. It was not until i think the 70s when he instructed the people to remove their shoes before entering the room. That finally start to see the gender balance in the orchestra started balance out. The problem with these language models is a basically always hear their shoes. Theyre detecting the were executed or captured. The team in this case just gave up and said we dont feel comfortable using this technology. Whatever its going to do its going to identify some very subtle pattern in the engineering resumes weve had before, the gender balance was off, so its just going to sort of replicate that in to the future which is not what we want. In this particular case they just walked away. Its a very active area of research. How do you d bias a language model . You have all these points in space and you try to identify subspaces within this giant threedimensional thing that represents gender stereotypes. Can you delete those dimensions while preserving the rest . This is an active area and it is kind ongoing. How much did amazon spend developing that . Its a great question. They are pretty tightlipped about it. Most of what we know comes from a reuters article where people were giving their names i wasnt able to do a lot of followup but as i i understand that actually not only disbanded the products but the disbanded the team that made an redistributed the engineers. So they really washed their hands. Im asking because however many i assume millions were put into that, couldve hired an extra h. R. Person or two. Thats right. Another example i want to get into from a different angle is a selfdriving car. You talk in the book about the photography that happened because of the way the car was recognizing this person. Again, explained that. This was the death of Elaine Hertzberg in tempe, arizona, in 2018, the first pedestrian killed by a selfdriving car with uber vehicle. The full kind of national highway Transportation Safety board review came out at the end of last year, so fortunately i was able to get some of into the book before it went to press. It was very illuminating to read the kind of official breakdown of everything that went wrong because it is one of these things where probably six or seven separate things went wrong, if it only been but for that entire suite of things going wrong i think it mightve ended differently. One of the things that was happening was it was using a sort of network to do object detection but it can never been given an example of a jaywalker. So and all of the Training Data this model had been trained on, people walking across the street were perfectly correlated with steeper stripes. They were perfectly correlated with intersections and so forth. A model just didnt really know what it was seen when it saw this woman crossing the street in the middle of the street. Most abject Recognition Systems are taught to classify things into exactly one of the discrete number of categories. They dont know how to classify stuff that seems to belong to more than one category. They dont know what a class for stuff that seems like its not in any category so this is again one of these active Research Problems the field is making headway on only recently. This particular case the woman was walking a bicycle and so this set the object recognition system kind of into this fledgling state where it first thought she was a cyclist but she wasnt moving like a cyclist, and thought she was a pedestrian but it was recognizing the shape of a bicycle. Then the thought maybe its just some object that is blowing or rolling into the road. No, i think it is a person to know, i think it is a biker. And due to a court in the way the system was built every time they changed its mind about what type of entity it was seen, it would reset the motion prediction. Its costly predicting this is how typical addition would move, as a cyclist would move, et cetera, and extrapolating as resu why think theyre going to be in a couple of seconds from now. That intersects the gardens going to do something. Every tim a change its mind, started recomputing that prediction. It never stabilized on a prediction. There were additional things here with overrides that the uber tm had made, normal and 22 mt cars have this self driving come they will automatically break for in some cases worth. And to override that,ort of at their own system in and those systems interacted in weird ways. The object rognition thing itself is for me very pragmatic. Theres quite set certain in confidence tt when a a neural netwk says im 90 sure thi is a person or whatever, how d we know if the probabilities are well calibrated . How does the system know what to do with them . I think many people kind of within the deep learning on certainty Certain Community would argue the mere fact that youre changing your mind shoulde a huge red flag and you need to slow the car down. That alone, and that wasn done. It very heartbreaking to think about how all of these engineering decisions add up to this event that would have been so much better toave avoed. The silver linin is that there are lessons in there that we take to heart, not just an industry but also in academia saying we need to get to the bottom of this question of certainty uncertainty. Thats a very human thing. You dont want thi exists in the medical literature. You dont want to take an irreversible action in the face of uncertainty. You dont want to take a highimpact action. Youee also in the law with things like kind of which the term aye preveed judgment. Im forgetting the ter buttigieg might issue an order in advance of the deciding what the real think wld be because theyre trying to prevent some irreparabl harm. Theres a question for the Machine Learning community which is hardly how do we take some t same ideas not one to me an irreversible choice in the face of uncertainty or a very high i fact, situation, that requires us to quantify impact and quantify uncertainty a have a plan for wha to do when we find ourselves there. All those pieces need to come together bute are seeing progress being made on all those fronts. It cant happen soon enough. In these examples that we talked about and others in the book and other you didnt include in the book, are the culprits the same, is it the same general problem that you think needs to be addressed . Den going to a do think it is being addressed . Theres one broad problem which in t field is not as the aligent problem and thats with the book gets its title which is just had weake sure that the objective in the system is exaly that which we want the system to do. All of the examples we highlighted so far has shown us cases where one must be very careful to think about how to translate the humanontingent into an actual machine objective. We think we want, we think we can measure reoffend but we cant. We can only measure rearrest. We think we can hire promising candidates. We can only he candidates who superficially resemble previous ndid spirit we think we can classi objects into different cateries that many objects belong in me than one category we dont always know what category to put them in. Knowing that we dont know. Allfhese things and there are many other manifestations as well speak to this fundamental issue of alignment but the actual mechanics are dferent sometimes theres a problem with the Training Data. Sometimes theres a problem with a model archicture. So one problem in touch son yet this is black fox issue of interpretability a explainability, how to know whats going on inside the model, how can we trust the output and reverseengineer, what about the input generated that output. There are questions of what is the socalled objective function of the system. What is the quantity were trng to minimize maximize and w do we define that . Each component of the system has its own manifestation of the alignment problem and each of them, to yourecond question, is it being addressed . Yes, and that for me is the striki thing that makes whe we are in now so different from where we were when elonusk cornered me in a bunch of executives in the room and no one had any a sickly good idea. We seeing an absolutely remarkable shift in the field. Even just from, i talked to oe researcher phd student, he said he worked on a ice age. People raise their eyebrows at him. He was a little bit kooky a little paranoia or something. He came your later in 2017 and is like an entire daylong workshop on ai safety. By 2018 its its a significant fraction presented at the conference. In absolute numbers the in othr people work of on this is still quite small but the growth even over that short time to my mind is astonishing. I think we cant, it cant come soon enough so i encourage all motivated undergrads and High School Students to get excited about this area. Theres a lot of work to be ne. In talking about learning and development thats going on on the ai, t ai research and develoent field is the actua commercialization of this technology ahead of where it shld be . Should this also be modeled and not put on our roads, put in our courtrooms . Thats a great question. In some ways, the criminal juice stuff as they say its got 85 your history at at this point but its as if were still playing catchup in terms of the analysis relative to the deployment. You can think of it sort of as a kind of race. Cant the understanding catch up to the actual implementation . Can. I think we seen that with social media. There were some decisions that facebook made about how to run their news feed ranking algorithm in the details were somewhat technical. They went from supervisory to reinforcement learning. We dont have to unpack that they basically the narrowminded focus on always prioritize the content that will get the most clicks on that thing, created a situation where extreme content is being promoted. People were burning out, leaving the platform in addition to many other societal externalities that was creating. They were able to replace it with a more advanced model that factored in you can burn someone out or they would start to distrust the platform, et cetera. Cynically, you could note that part of the point of that model was to maintain user retention and casings that were good for the bottom line. I think there really is a question of when you think about the alignment problem, is a system doing what we want, when you look at actual industry theres the metaquestion which is what is a we want the system to be doing . And who is really in that sentence as well . Those questions are going to loom ever larger. We have seen in general media more urgency as he sres, when this topic has come up theres more people say we need to be thinking about this. When you delete the think that all the implications. Cause as you said, it is always being rolled up whatever audience members asked about chinas widespread use of facial recognition and possible orwellian 1984rediction. In the book what you do talk out facial Recognition Technology and the inappropriately funny results of it which were just absurd but also insulting and worse implications but he could talk about facial recognition . Another thing in california, he became a proposition about whether or not to use these technologies. Tell us a bit about that and how it fit into what you are talking about. This is, t the legal system now, if im rememring correctly i think the first se, i want to say minnesota or wisconsin o someone being arrested by being incorrely identified in aacial recognition case. A lot of this stuff is going through the court system probably hea to the supreme court. On the technical side, theres this reay unfortunate thing and that of hard to ignore pattern in particular ethnic minorities being incorrectly regnized or categorize, et cetera my face Recognition Systems. One of the famous examples was the software developer, jackie, in 2015. Group of photoaphs he had taken of himself and friends caption by Google Photos as guerrillas. And another example is an mit researcher, which was an undergradue computer scientists doing these facial recognition homework assignments, she had to borrow a roommate in order to check and make sure that the face system worked because it didnt work on her. It worked on her onlyhen she wore ahite mask. This really set off an instigationf why does this ep happening . With the underlying thing . There are a couple different components to it but i think one of the main ones i is that evei thin a preexisting lackadaisical attitude about how these databases of faces were pu together in the first place. Part of what led to the rise of comput recnition was the internet. Suddenly if you needed happen merely examples about human face or to train your system, well, in t 80s we were totally t of luck but nowe have internet and Google Images you can just download a million facesnd put into your system. The question is which faces are you downloing . The most Popular Research database for many years was one develop in the late labeled faces in the wild. They thought what we want to do is understand why to faces are, in fact, the same person. We had thi clever idea, newspaper headlines or newspaper images because they all are labeled with this person and this person and this person. That way we will have this giant data set and we can decide are these two images as a person. The problem is you were at the mercy of who was on front page articles in the late 2000 and this was george w. Bush. And, in fact, an analysis of the labeled faces in the wild date is set down just a few to go showed there were twice as many pictures of george w. Bush in the database as all black women combined, which is just insane. If youre trying to build something, to be fair to the people who collected that data, this was a kind of acamic research project. This is not intded be used in any actual system but these data set that way for sticking around if someone just downloads it off the internet for the most exciting sighted in. You want your paper to be cited. Striking if youook at the regional papers, and dont want to single them out because it was widespread. The word diversity is getting used in the sort of early 20 steaming lighting and pose. Poster feelay this is the most verse data set a civil today. What mean is with people from the side, people in the door. W its verytriking. Some of these old databes now appear with a warning lab on it that says when was the diversity within a a specific thing and we want to flag it at ry much not the first in these demographics. Thers a lot of work that is beingone, spearheaded by peopleike joy at mit and goog brain, to bring more focus on equalizing the error rates across different ethnic groups, making sure the database, the Training Data reesents the populationhat the model will be used on. Thinking also abo the representation of techtself. I think in 2019 only less than 1 of Computer Science phd for africanamericans. Theres a lot of work to be done in t field itself toddress that question of representation. We are seeing groups like blackening ai which has numerous initiatives including scholarships and grants and things like that, trying to eqlize that not as they say no just in the Training Data bu in the field itself. Theres a question from the audience about kind of maledominated answer by being late into the design. In the book while you are discussing the biased and embedding, removing gentrification or genderneutral, as mentioned rlier, you mention of work that a gup of interested i will mispronounced the name. You noted 15 the five computer scientists found themselves doing social science. That would seem to be a requirement for any of the work beuse all the different interpretations one has to take in and understanding the fuzziness of the social science that has to some of the mixed in with the hardness of the Computer Science, right . I think thats absolutely right and that to me as one o the really striking things about where the field finds itself at this moment, which is no lger can dated scientists and Software Engineers think of themselves as purely doing engineering or purely doing mathematics, that we have just gotten to a point where these systems are absolutely enmeshed in kind of human practices, how was the Data Collected regenerated, the question that the human respondent were being asked, how is it worded what you would get different answers based on how it was worded. And what population of people were you sampling from . By people on amazon mechanical representative or not representative of other groups that might respond to the same thing. We are very much in this moment of to my mind kind of exhilarating only interdisciplinary work that needs to happen and is happening. Between the Computer Science Machine Learning community and social scientists, philosophers, ephesus, lawyers, cognitive scientists, theres a lot of really interesting work being done at the intersection of ai and infant cognition. Were at the point where they are essentially resembles an infant, and so the Machine Learning community is going to development of psychologists and saying whats your best theory for the curiosity of an infant, or the novelty buys that small kids have . Or the exploratory kids will have with the particular toy to see out works. We need to import that into our ai system to solve this problem. In turn the ss and might be unlocking some of these questions of why you kids have the sort of drives . There are many fronts on which this is happening. Social sciences i think our kind is uniquely positioned at that interface Computer Science. We are seeing many more papers with a really the first step of skills among the authors and thats the kind of thi to me its very encouraging. Ase mentioned at the beginning you have an interesting author, poe and programmer. I suspect the venn diagram, you barely touched circles, but does that set of skill help you maybe see out at some binder set otherwise might help,hat do you think . Philosophy of mind takes one angle on the question and ai answers it in a different way by trying to actually make it. Broadly speaking, its been a question for 2500 years of western philosophy what it means to be human, what makes us unique and special in the space and aristotle, descartes and so forth have answered that question by comparing ourselves to animals and i think theres never been a more interesting concept to be thinking aboutthis area because now we have a completely new standard of comparison. A totally different set of answers. So aristotle and descartes and a society that analytical delivery reasoning is at its core of what it means to be human because thats what monkeys cant do. I dont think anyone thinks that analytical reasoning is the seat of the humanexperience. Its more about imagination, social ties, teamwork, collaboration, etc. So for me, i think in some ways i feel very lucky that i have this very eclectic set of interests that happened to be aligned that the two disciplines areon an absolute collision. Seone in our audience asks celebrity achieved, in other words larry. Therere a number of things that people meany the word singularity so theres some people for whom it means havi something also called the hard chaos which means theres an abrupt moment of time and th coincides us. I donnecessarily, i dont really see that. Im in the camp of where to take software i think ai is going to be getting weirder and more uncny and we just accept it as everything we would require itto do. But that there wont a sort of sharp bow turn where thissuddenly happens. From my perspective of its totally inevitabl. And its a long tradition in Computer Science going back to the 1940s did you ever think of teens and he said of course, and its to havehat kind of secularity, worldview. Youre not a dueli and you think the brain is madof atoms, its all about this so of emerging behavior of complexity. And think thrillingly and sort of sneakily were on the road now. So i just relead a system call tbc three months ago and hundd 75 billion parameters. , if you compare the extra numbers to the human brain its abo one 1000 of the human brain. So its not very impressive, there are. 1 percent five but the average model size in that field of ai is still only three months so we do the math, that means we should expect models that have a synaptic influence on the human brain sometime in the spring of 2023. So sooner or later i think it still feels a little bit scifi are going to really start to come to the surface. I dontknow what the answers will be. I think thats one of the riveting things about it. There was a story recently about these two computers that had to communicate with each other and they spontaneously develop their own language for communicating with each other using whatever else they had available. And in talking about anticipating the need that we have for ai to develop in certain ways, to align with what we wanted to do and what our needs are, thinkable so robots solving a problem that in a completely different way. Is it possible that we will get Artificial Intelligence that is much more advanced and that is able to deliver us results are more understandable, more in line with what we want to see or their way of reaching it will be sort of alien. Thats a great estion. I think both of those possibilities are widespread. One of the things that sometimes is forgotten when we talk about ai, theres a certain blend of inevitability to thi question of progress. But there are real choic to be made out the architecture of the systems for example, its already self driving carfor example. Trading the system and to and you can have a giant blob of Neural Networks and put the camera feed in the bottom and these commands come out the top and the technology is whats going on. There an increasing science to the queion of figuring out what is going on. But also how doou constrain the network in certain ways, can it be modular. So that e system is naturally divided into parts you can and analyze individually. Let me now worry about whats going on here. Theres a lot of really encouraging results in that space so i think your question will ai be able to do what we want but in a way thats totally inscrutable, yes but not necessarily th only way it can happen. I think we will have more agency there that i think is appreciated to build the kinds of systems that we know we can trust. When you talk about Artificial Intelligence often people ask you about terminator and skynet. Its been interesting to watch the canonical questions that i get evolve and when my first book came out in 2011, 2012 a lot of people were asking me his ai for myjob. By 2014, 2015 people were asking me if ai going to destroy all of humany as we know it. So theres a case that really has gone up there. And within the research counity, the Cautionary Tales has shifted fr one that feels more like yes, sort of a disobedient syem to one that feels more like a system, i use the analogy in the book of a sorcerers apprentice. Its trying to be helpful bu it doesnt quite knowwhat you wa it to do. Its a thoug experiment that comes from chine intelligence whats called the paperclip model so its an experiment of an paperclip factorrun by ai and you say we want to increase the output of ourpaperclips. But unfortunately its so good that turns the entire universe and paperclips and creates yourself and your loved ones. So thats a lile bit caricatured obviously but at is the thing that people work on alignment or worried about, noa system owing road, is not a system deciding that humans are to be exterminated, its a system that is in a sort of one way trying to do wh it thinks we want it to do. But realizing that we reize that we werent quitespecific enough. I think a sorcerers apprentice or one of these things so part of what the alignment problem is, part of whats solving that problem would mean is feeling comfortable communicating attention to a system like that without necessarily needing every specific detail before we press the button. And the scientific term incorrigible it can take feedback, we can say hold on, thatsnot what i. And that is the kind of thing that i think makes myself and a lot of people who are working in this area a little bit more relaxed and we were four years ago. Whatever audience members notes that a former computer engineer, he says people all wanted to besystems analysts. And he asked what do they want to be called today. I want to ask your talking about how the field now is becoming much more interdisciplinary, going into inuxes of other skills. Are the pele who are in ai field cage cnging, are the people who are like, i want to go intohat field or some of the people th mars switchg over into this, then may they were 10 years ago. Thats an interesting question. The question, the initial point about being a systems analyst i was thinking about the head of ai tesla. Has this notion of what he calls software 2. 0, that software 1. 0 was programming one. If asked, do. Software 2. 0 is the role that were in now with machery where you dont write co, u provide a set of training and say heres Something Like that. Theres sort of a debate over whether its going to be a network 3. 0 which is fully general ai system that can make the questions. You get the Training Data but you work with it after the fact and tec three is in that category area and working with pvc free feels a lot more like science where your writing almost like an essay prompt. So if tv 33 is a language model designed to fill in the blanks. You can use it to do also to things. You can say the following is a program and list of integers like it will write python code foryou. You can say the following event is an apartment that has the most permanent objections and it will give you some little essay. And what it means to sort of use a model like that, starts to feel a lot more like how to work with another person, how do you work something so that the meeting comes through or you get what you want for the style that you want. So it is i think going to be this new category of people are not expecting, not exactly programmers, not exactly analytics people but people who are sort of wrangling these giant models through natural language. Thats kind of a new job that doesnt really exist yet. Thats going to require, its an interesting set ofskills. Its going to require the worst things are very important and it also requires how these models are trained to help you figure out how not to make it do something. More broadly i think as these questions that Machine Learning is dealing with become more and more human, it does invite a certain kind of person who might not have felt like they belonged to now feel like they belong. To feel like their skill set can plug into that. Thats what were starting to see and i think that is a shift that is really just beginning. Who are you trying to reach with this book . Is it general audience, is it the ai community or is it entities that are using disabilities, doctors . There are a couple audiences. One is the general public. I think relative to other fields in science, ai, Machine Learning and these questions of bias, a lot of people are aware of them. Whether or not theyve taken the time to go deep and understand the underlying issues so one issue is to say this debate is already happening. I want to try to raise the level of the debate. If people give the conceptual insight and some of the basic vocabulary you can feel comfortable talking about these things and tackling them all. Theres a huge class of people who went through their career with a particular type of training that didnt ever seem like it required them to know about Machine Learning. Suddenly its 2020 and youre being handed these algorithmic representations if youre a lawyer or a medical diagnostician and your getting machine predictions about cancer and so forth. Theres a lot of people out there that need some level of familiarity in this area and i hope the book can fulfill a need there as well. Lastly, i hope we can expand the field. I think this is one of the most exciting and important things happening toscience , not just to this science but science period. If i can reach undergrads and get them excited about this area, give me a cool project i can do, what can i do . I think that for me will feel really good. It will bring even more folks into this and that would be a good thing. We got time for one more question so this is thetime travel question. In your book you ta about one of the earliest attets at developing ai but what realistically would you have said to see the state of ai both in research and direction . 20 years is interesting. Therea joke in ai that ai was always 25 years awayand it still is. But i think realistically, there will be of course a generational replacement that happens over that time and i think the way that we now have people who think of that kind of digital native, social media native, theres going to be generation of ai natives. That generation of people that are just being born now, they will grow up in a world where they may not get drivers licenses. How can you justify a human driving a car . And they will i think come to understand themselves as living in a world in which there are all these different systems with different degrees of what you mightcall intelligence, different degrees of what you might call agency , different incentives that align with our own to one degree or not. And interfaces that increasingly look like the way people talkto each other. It has no problem talking to a lightbulb forexample , they might make it fairly normal that theres some kind of system that you can just chat that is remarkable even if you think back 10 years. That was not something anyone was familiar with. So yes, i think in some ways the balancing will start to get blurry in the technical skill setand adjust the skill set of navigating the world of the human. It will sort of increasingly start to feel like they speak that same language. You can communicate and you sort of reach your point with something and you grasp of the for the itemit thinks youre reaching for but then you go no no its the other one where you communicate in words with another person. I think were going to have a newgeneration that sees that thats the way the world works. And hopefully we will set them up to have a reasonably good world indeed at that point. Very good. Thanks to brian christian, the alignment problem, through learning and we also like to thank our audience for participating online. If youd like to watch more programs or support commonwealth efforts, please visit Commonwealth Club. Org online. Thank you, i wish you a good day and stay healthy. Thanks a lot. Weeknights this month we are featuring book tv programs as a preview of whats available everyweekend. Tonight its a look at business and economics. Starting at eight Eastern University of virginia business professor and freeman discusses responsibility and affects and he says unite into essential businesses and history professor jeanette garrison exploresthe period of financial innovation between 1888 and 1930. And its effect on us capitalism through the story of the st. Lukes campaign in virginia, the first and only back. Run by blacks and mit professor Thomas Levinson looks at how the leaders of the 15th century scientific revolution applied their new ideas to people, money and markets and as a result embedded modern finance. That begins at 8 pm eastern and enjoyable tv this week and every weekend on cspan2 area. Ok tv has taug nonfiction books and authors every weekend coming up this weekend, saturday at 9 pm eastern, former president barack obama reflects on his life and political career in his newly released memoir the promised land. Sunday at 9 pm eastern on afterwards, director Sally Hubbard and her book seven ways the corporations rule in your life. But take back control. Es interviewed by blooerg News Reporter david is cough and former appellate judge and the George Mason University law professor Douglas Ginsburg in his book voices of our republic examines the constitution through the eyes of judges, legal scholars and historians. What book tv on cspan2 this weekend and be sure to watch indepth live, sunday, december 5 at noon eastern with our guest author and chair of africanamerican studies at princeton university. So nice to meet you