Alignment problem. It was released just last week. He offers theoretical contributions to the industry documenting in his bestselling book the most human human outer rhythms to live by. In his new book, question such as if we continue to rely on Artificial Intelligence to solve our problems, whathappens when ai itself is the problem. Greenbergs last seen Computer Scientists tackling the obstacles of technological implication of an advanced new society that has become more like on technology. His work on Technology Leaders such as eli must once put the question in his 2000 book the most human human for his night table reading. In his new book the alignment problem is dependence on ai questions whether or not the system can help us or the systems simply are a replication of an inevitable human desire, how can we handle ontechnology. Here he argues that all these things make decisions for us, essentially we will be discussing a lot of the next hour and i want to ask your questions to watching live , read our questions in the chat on youtube. So i can work them into our conversation. Thank you brian and welcome. Thank you forjoining us. You for having me. This is not your first book. I want to ask kind of the ultimate question that is the obvious question we ask an author but were just setting the plate for our station here and that is why did you try to tackle this kind of thing now in this book. Its a great question. The initial piece for this book came after my first book had come out as you had mentioned, vanity fair reported that on must, it was his best bedsidereading and i found myself in 2014 attending a Silicon Valley book group. That was investors and entrepreneurs and they had seen this thing about elon must reading a book and decided to joiand to my surprise he did and there was this really fascinating moment at the end of the dier when the organizers me and everyone stood up to go home for t night. And elon must urged everyone to sit back down. He said no, but seriously, what are we goingto do about ai . Im not letting anyone leave this ro until either you give me a convincing counterargumentwhthere should be this or you ca give me an idea for something else. And it was quite a memorable. And i found myself drawing a blank. I didnthave a convincg argument. And i was aware of the conversation around ai, some people its like a human expansion level risk. Other people are focused more on presentday ethical problems. I didnt have a reason why we shouldnbe worried about it but i didnt have a concrete suggestion and that was the general consensus in the room at that time. So this queson of so seriously, whats the plan . I was finishing my previous book and i really began to see around 2016 a Dramatic Movement wiin the field to actually put some kind of plan together. Both on the ethical questions and the sort of further into thefuture safety questions. And both of those movements have grown i woulday explosively to 2016 and now. And these questions of ethics ansafety and whats in the book i described as the alignment proble how do we make sure the objective that the system is carrying out is in fact what were intending for it to be. These things have gone from card kind of marginal and somewhat philosophical questions to really today making up i would say the central question. And so i wanted to tell the story of that movement to figure out in a way the question, whats the plan. What are we doing . One of the interesting thoughts about getting into this was theres a lot of opportunity, complex Technology Going into this but i dont think a lot of people are aware of that the systems already being applied into lifeanddeath situations. We when we were talking to the program, you picked out a number of examples in the helplessness they hope to perform one of those examples could not be more true and that his algorithms are being invited judges notin california but what im getting into in california , where instead of ch bail, a judge will use an album rhythm to determine wheer or not a suspect is released or is allowedo only try and be small californians are boarding that proposition. That would reaffm along that would replace it with this algorithm system and its a very complicated issue but what reay surprised me is the naacp and Human Rights Law Firm this proposition because of built in and equities of the alrithms. So whether we get into that kind of build from there. Wh happens with those problems that algorithms encounter d how do we get there . Is a nearly 100 yearlong series of statistical orhat in the 1920s were called actuarial methods and parole. This attempt to create a kind of science of probation and parole. And that as i say started in the 20s and 30s but really tookff with the rise of persal computers may in the 90s. And today its implemented almost every jusdiction in the us, and there has been an increasing scrutiny thcome along with that. And itseen interesting watching the public discour on some of these tools. So f example, the New York Times was writing, the New York Times Editorial Board was writing through out 2015 the letters saying its te for new york state to join the 21 century. We need objectives that are evidencebased, anwe cant just be relying on the whims of folks in robes behd the bench. We nd to bring science. Shortly, that position changes. And i just months later the end of 2015, the new yo times writes articles saying algorithms are putting people in jail. Algorithms have succded racial bias, we need to throw the brakes and calling out my name this particular tool which is called compass which is one of the most widely used tools. Throughout the united states. And the question has this really ignit an entire subfie within statistics around the qution of what does it mean to say that the tool is fair. This system is designed to make predictions about whethersoone will reoffend. Its if theyre released on probation or if their pending trial in a pretrial. What does it mean to get ese concepts into law. Things like disparate treatment for equal opportunity etc. Working keep a minimum protections etc. What does it mean to turn ose. And how do we look at tool like ts and say whether we feel comfortable actually to the point. It was interesting when you get a handle on how a black suspect in the white suspect for similar crimes, similar backgrounds and much more likely. [inaudible] including one what suspect. [inaudible] what were, so what goes into the baking of that cake that built these biases because biases were notintentionally into it. But there still hard baked in. Yes. This is a very big conversation. I think one place to startis to look at the data that goes into the systems. So one of ththings that these systems are trying to do is predict one of three things. Lets just think about the pretrial case for now. A tool like compass is producing three different things. One is your likelihood to not make your court appointment. The second is to commit a nonViolent Crime while your pending trial athe third is is there a Violent Crime pending trial . The question is where does the data me from . And if you look at Something Like care in court, the court knows about it by defition. So thats ing to be fairly unbiased, regardless of who you are. If you look at somhing like nonViolent Crime , it the case that foexample if you pull yog white men and young black men about their purported marijuanusage they self report that they use marijuana at the same rate andet if you look at the arrest data a black person is 15 times or kely to be arrested for using marijuana that a white person is. In manhattan, and in other jurisdictis it might be eight times aslikely in iowa. Varies from place to place. Thats the case where its really important to remember that the model to be able to predict crime but whats actually predicting is rearrests so rearrests this nd of in person and systematically so proxy for what we really care about witches crime. Its ironic to me because part of the project of reseching the systems i went back into the historical literature, when i first startegetting views which is inillinois in 1930 at the time, a l of the objections were coming from the conservatives, from the political right. And ironically mang the same argument being made now from the other side so conservatives in the late 30s were saying it a minute, if a bad guy is able to evade arrest and the system doesnt knowthat he committed a crime , hes innocent and though recommend his release and recommend other people like m. Now we the argument is ing made from the left whh is to say someone is wrongfuy arrested and convicted , they dont have any concted indexing data and will recommend that attention of otherpeop like them. Its really the same argument , just amed in different ways but thats a vy real problem and we are starting to see groups like for example the partnership for ai which is a Industry Coalition with aumber of groups100 different stakeholders recommending that we dontake these protecons of nonviolent rearrests as seriously as we take predictions of nature. The second component i want to highlight here, its the second thing is worth covering is thisquestion o what do you do with the prediction once you have . So lets say youve got a higher than average that youre going to fail to make your court appointment. Thats a prediction. Thesecond question is what we do with that information. One thing we do that information is put a person in jail. While they wait for their trial. Now thats one answer and it turns out theres an emerging File Research that shows you send them a text message reminder, they are much more likelyto show up for their court appointment. And there are people proposing Solutions Like providing Daycare Services for their kids for providing this subsidizedtransportation to the court. So theres a whole separate question which is as much as is going on, as ch scrutiny is being directed at the actual algorithmic prediction, thers a much more systemic question is which iswhat do we dwith those predictions . And if youre a judge and the person is gog to failto reappear , ideally youd want to recommend some kind of text message for them as opposed to jail but that may not be available to you in that jurisdiction so you have to kind of go with what you have and thats a systemic problem, not necessarily algorithms per se and the algorithm sort of caught in the middle. Lets take it out on the crime and punishment area into the business area and you talk later in the book about hiring and amazon coming uwith this ai system to help it cold job applicants and what they were finding was that a lot them, the reason for this also were make into the way thsystem was being trained in the way the system was being used d also when you get to this, it also has the question, why were you trying to find people but tell us about that and how did they get into it . This is the story that involves amazon at the yea 2017 but by no means are they unique examples here. The example of amazon bu like Many Companies they were trying to design a little bit of the work load off of t human employers and if you have an open position and you start paying x number of residents coming in, ideally youd like some kind of algorithmic system to do some triage and say ese are the resumes that have rked or theyre lookinat more closely these were lower priorities. And somewhat skewed or ironic twists, amazon deced they wanted to weigh applicants on a scale of 1o 5 stars so they rate prospective employees the same way. But to do that, they were using a type of computational language model called word vectors and without getting too technical, for people who arfamiliar with the rise of Neural Networks, these Neural Network dels were very successful. Around 2012 all of them started to move to computational linguistics. And in particular there was this very remarkable faly of models that were able to sort of imagine words as these points in space. So you have documentwords based on the other words that were unified and 300 dimensional space. But ese models had a bunch of really cool other operties. You could actually do nd of arithmetic with words. King midas man plus one. D search for the point in spacewas arer to that. And get clean. You could do tokyo minus japan plus england and get london. And these were the same so these numecal representations of words that fe out of this Neural Network and it being useful for this drivingly vast array of tasks. And one of these was tell us try to figure out the quote unquote relevance of a given job. And so one way you can do it is just say here are all the resumes of people weve hired over theyears. Throw those into thiwork model and find all this point in space and thenor any new resume, see which of the words have the kind of positive attribus in which have negative attributes. Sods good enough but this team started looking at this and they found all sorts of bias. For example the word women was assigned a penalty so if you are going to do a Womens College or Womens Society of something, then the word women on a tv would get a negative deduction. So yes, getting a negative rating or whatever because its further away from the more susceptible words that its been trained to watch for. Doesnt appear on the typical resumes collected in the past. And its similar to other words also here. Of course the team, the red flag goes off and they say releases attribute from our model. They start noticing its also applying deductions like womens field hockey for example. Or Womens College. So they get rid of that. And then they start noticing that its picking up on all these very subtle choices that were more typical of resumes and females so for example the use of words executed and capture. I executed a strategy to capture market value, phrases that were more difficult. And at that point they basically gave up, they scrap the project entirely. And in the book i compare it to something that happened with the boston Symphony Orchestra in the 1950s where they were trying to make the orchestra which had been a little bit more equitable so this they decided to hold auditions behind a wooden screen. But whats someone found out only later was that as the audition or walked out onto the wooden floor, the horse they could identify whether it was a flat shoe or a highheeled shoe so it wasnt until the 70s when they officially requested people remove their shoes for entering the room at finally started to seegender balance. So the problem with these language models is that they basically always adhere to shoes, their detecting the word executed, captured and the team in this case gave up and said we dont feel comfortable using this technology. Whatever its going to do, its going to have some very subtle patterns and the engineering detects before, the gender balance is off there so its just going to sort of replicate that into the future which is not what we want soin this case , it just walked away but a very active area of research, how do you the bias language model. Youve got a point in space and you try to identify spaces within this giant thing which represents gender disparity and you leave those dimensions while serving the rest. It is area and its kind of ongoing tothis day. How much did azon spend developing that its a great question. Thre pretty tightlipped about it most of what we know comes from an article where, i wasnt able to da lot of followup but as i understand they actually not only gave the product but they dianded the team that made and distributed their engineers to other things. Im asking because i assume millions were put into that that could have hired an extra hr person. Another example i wanted to get to and might be coming from a different angle is you talk in the book about the fatalities that ppened because of the wathe car was recognizing this person. Again, explain that. This was the death of Elaine Herzberg in tempe arona in 2015, the first pedestrian killed by a self driving car. It was an r and d uber vehicle and the transptation safety review or came out and i go into some of th in the book and it was very illuminating. To read the kind official breakdown of everythinthat went wrong cause it was one ofhese things where probably six or seven separate things wentrong. Had only been but for that entire slate of things going wrong it might have been prevented. One of the things ppening was it was using sort of the neur network to apply object detection but it had never been given as an exame of a jaywalker. So in all of the Training Data that it had been trained on, peoplealking across the street were perfectly correlated with viewers and they were perfectly correlated with intersections. So, they dont how to classify stuff thatoes into more tha one category or classify how things are not in any category so this is again one of these active Research Problems that the field and making headway on only recently but i this particular case the woman was lking a bicycle and so this sets the object rognition system kind of into this uttering state where at first they thought she was a cyclist but she wasnt moving like a cyclist and then and thought it was too pedestrn but recognizes she had the bicycle and thought just its something object thats been blowing or rolling into the rd but no, i thinkts a person, no its a biker but due to a quirk in the way the system was built every time it changed its mind about what type of entity it was seen it would reset the motion prediction so its constantly protecting this is how a typical pedestrian would move or a cyclist would move et cetera and extrapolating as a result where i think they will be in a couple seconds from now. If that intersects a car that will do something but every time it changed its mind started re computing that prediction and so its never stabilized on a prediction. So, there were additional things here with overrides that the hoover team had made because in 2018 most cars already had this very rudimentary form of self driving like automatically break or swerve and to override that and at their own system in and those in weird ways but i think the object recognition thing itself is, for me, very schematic and there is this question of certainty and confidence that when Neural Network says and 99 sure this is a person or whatever it may be how do we know those probablys are well calibrated and how does the system know to do with them and i think many people within the deep learning uncertainty community now would argue that the mere fact that you are changing your mind should be a huge red flag and slow the car down but that alone in that wasnt done. So, yeah, its very heartbreaking to think about how all of these engineering decisions at up to this event that, you know, would have been so much better to have avoided. I guess the Silver Lining is that there are lessons in their that have been taken to heart, not just in industry but also in academia saying we really need to get to the bottom of this question of certainty, uncertainty because thats very human thing and you dont want this to exist in the medical literature and you dont want to take an irreversible action in the face of uncertainty or take a highimpact action and they say in the law and in things like kind of what is the term . Preemptive judgment and im forgetting the term but a judge might issue an order in advance of deciding what the real thing would be because they are trying to prevent some irreparable harm and so the question for the Machine Learning community which is how do we take the same ideas not wanting to make an irreversible choice based with high impact or uncertainty that requires us to quantify impact and quantify uncertainty and have a plan for what to do when you find yourselves there. So, all those pieces need to come together but we are seeing progress being made on all of those fronts and yeah, it cant haen soon enough. In these examples we are talking about and in others in the book and that you did not include, how does the culprit, is that the fame same general problem that you think needs to be addressed and then i will ask do you think its [inaudible] judge to your satisfaction . There is one broad problem in the field known as the alignment problem and that is where the book is titled which is just how do we make sure that the objective in the system is exactly that which we want to do and i think all the examples weve highlighted so far have shown us cases where one must be careful to think about how to translate the human intention into an actual machine objective. We think we can measure re offense but we cant spread we can only measure rearrest. We think we can hire promising candidates but we can only hire candidates that superficially resemble previous candidates. We think we can classify objects into different categories but the objects belong in more than one category or we dont always know what category to put them in and we have to act knowing that we dont know. So, all of these things and there are many other manifestations, as well, speak to this issue of alignment but the actual mechanics are different. Sometimes theres a problem with the Training Data and sometimes there is a problem with the model architecture so one problem we havent touched on yet is this black box issue of interpretability, explain ability and how do we know whats going on inside the model and how we can trust the output and reverse engineer what about the input generated the output and there are questions of what is the socalled objective function of the system and what is the quantity or trying to minimize or maximize having to define that so each component of the system has its own manifestation of the alignment problem and age, to your second question, is not been addressed in that for me is the striking thing that makes where we are now so different from where we were when we were elon musk cornered me and executives in room and none had any particularly good ideas. I think we are seenn absolutely remarkable shift in the fieldhat even just from, i talked to one researcher phd udent who said when he went to the biggest industry conference in 2016 and worked on ai safety people raiseheir eyebrows at him like it was cookie or a little bit paranoid and he came back a year later in 2017 and there was an entire day long workshop on aiafety. By 2018 its a significant fraction of t papers he presented at the conference. In absolute numbers the amount ofeople working on this is still quite small but the growth even over that short timeo my mind astonishing and i cant come soon enough and i encourage all motivated, undergrad and High School Students to get excited about things because theres a lot of work to be done. There is obviously a lot of learning and Development Going on in the ai Research Field and is the actual commercialization of this technology ahead of where it should be . You know, should this also be where its being modeled and not put on a road or put in our courtrooms . Great christian. Yet, in some ways with the criminal justice stuff, as i say, 85 year history at this point but its as if we are still playing catch up in terms of the analysis regular to the deployment and so you can think of it as aace. Can thenderstanding catch up to the actual implementatio i think we see that with social mea. You know, there were some cisions that facebook made about how to run their new spe ranking algorithm in the details arsomewhat technical and one from supervised learning to reinforcemt learning we dont have to unpack that exactlyut basically the narrowminded focus on always prioritizing the content we will get the most clicks on that thing and created a situation where extreme was being promoted and people were burng out and leaving the platform in addition to the many other societal externalities that was creating and ty were able to replace it with the morse advanced model that ctored in you could burn someone out or start to distrust the platform et cetera. Cynically u can note that part the point of that model was to maintain user retention in thes things that are good f their bottom line and i think there reall is a question of when you think about the alignment problem, is the system doing what we wt . When you look at actual industries there is the meta question which is what is it that we want theystem to be doing . D who is we in that sentence as well . I thi those questions will loom ever larger. We have seen in general media more urgency put in the past few years that when the topic does come up [inaudible] we need to be thinking about this community to think about education and because, as you said, this is being rolled out one of our audience members asked about china, widespread use of facial recognition and possible well into 1934 indication so in the book you talk about facial Recognition Technology and t inapprriately funny result of it which was absurd but also insulting but could you tk a bit about facial recognition and thats another thing that fects here in california and became a proposition in about whether or not to use these technologies so tell us a bit about that and how it fits into what youre talking about here. Yeah, this is coming through the legal system now and if i remember and currently the first case and i want to say minnesota or wisconsin of someone being arrested by being incorrectly edified in a facial recognition database. A lot of the stuff is going through the court system and probably headed to the supreme court. On the technical side yeah, there is this really unfortunate and hard to ignore pattern in particular of ethnic minorities being incorrectly recognized or categorized et cetera by facial recognition. One of the famous examples was the Software Developer jackie in 2015 a group of photographs he had taken of himself and a friend caption by Google Photos as guerrillas and another example is the mit research joy [inaudible] was an undergraduate Computer Scientist doing these facial recognition homework assignments and she had to borrow her roommate in order to check and make sure the face system worked because it did not work on her. It worked on her only when she whirled white mask. This set off an investigation of why does this keep happening and what is the underlying thing. There are a couple different components to it but i think one of the main ones is there have been, i think, a preexisting lackadaisical attitude to have these databases of faces were put together in the first place. So, part of what led to the rise of computer recognition was the internet and suddenly if you needed half a million examples about human faces to train your system while, in the 80s you are totally out of luck but now that we have the internet and Google Images you can just download a million faces and put it into your system. The question is which faces are you downloading . The most Popular Research database for many years was one developed in the late called labeled faces in the wild and they thought okay, what we want to do is understand other two faces are the same person so we have this clever idea that we will do newspaper headlines or images because they are all labeled. With this person and this person and that weight we will have the giant database we can decide are the images the same person but the problem is youre at the mercy of w was in front page news photograph in the late 2000s and the answer is georg w. Bush. Then present georgia be bush and in fact, an alysis of the labeled faces was done a few years ag showed that there were twice as many picturesf george w. Bush in the database as all black women combined. Which is just insan if youre trying to build something and to be fair, to the people who colcted the data werent, this is an Academic Research project and it was not intended to be used in any actual system but these data tests have a way of sticking around if someone downloads it off the internet or if you want your [inaudible] is very striking. If you look at the original papers and i dont want to single them out because it was widespread but the word diversity is getting used in this early 2010 to mean lighting imposed so they will say this is the most diverse data set assembled to date but what they mean is we have people from the dark and the side and people [inaudible] now at the end of 22, beginning 2020 very striking some of these old databases now appear with a warning label on it that says when we said diversity we meant a very specific thing we want to flag it is very much not diverse in these demographics. So, there is work being done there too spearheaded by people like joy at mit and tony at google brain who to bring more focus on equalizing the error rates across different ethnic groups and making sure the database in the Training Data represents the population that models will be used on and thinking about the representation of tech itself so i think in 2019 only less than 1 of Computer Science phds were africanamerican. And so, there is a lot of work to be done in the field itself to address that question representation and so we are seeing groups like lac and ai which is a number of initiatives including scholarships and grants and things like that. Trying to equalize that not, as i said in the Training Data but in the field. The question from the audience about you know, talking about maledominated answers being baked into the design. In your book lay discuss word embedding, removing gender interpretations and genderneutral iterations and you mentioned a group of researchers did not mispronounce the name and adam collide. The team of five Computer Scientists [inaudible] that would seem to be a requirement for any of the work because of all the different interpretations one has to take in and understanding the fuzziness of the social science that has to somehow be mixed in with the harness of the Computer Science. I think thats absolutely right and that to me is one of the really striking things about where the field is itself in this moment which is that i think no longer can you know data scientists and Software Engineers think of themselves as purely doing engineering or purely doingathematics. Weve just gotten to a point ere the systems are absolutely enmeshed in human practices how is the Data Collected are generated and how i the question that the humans fondants were being asked and w is it worded because it will get different answers based on how it w worded and what population of people where you sampling it from so our people on amazon are they represented or not of other groups that might respond to the same tng. So, we are very much in this moment of,o my mind, exhilarating the interdisciplinary work that needs to happens happening between the Computer Science Machine Learning community and social scientists, philosophers, lawyers, cognitive scientists and there is a lot of interesting work being done at the intersection of ai in infant cognition and we are the pnt where ai resembles an infant and so the Machine Learning counity is going to develop mental psychologt saying what is your best theory for the curiosityf an important or the velty that small kids have. Or the exploratory place with kids who hav a particular interest in a toy to figure out how it works and we need to import that into our ai systems to solve this problem. In turn, ai system might be unlocking some of these questions ando there are many, many fronts on which thiss happening, social sciensts i think are kind of uniquely positioned at that interface with comter science and we are seeing many more papersith a really diverse set of skills among the authors and i think thats the kind of thing that is very encouraging. As we mentioned at the beginning you have an interesting yourself. Youre an author and poet and ive studied the then diagram about poets and programmers and does that set of skills help you may be see out of some blinders. Yeah, i could joke that poetry and programming assaults of scrutiny over a; yeah, my backgrnd is that when i was a student i was interested in this question of what is thinking and to be conscious et cetera and philosophy of mind sort of tak one angle on that question and ai answers itn a different way by trying to aually make it and so i think broadly speaking its been a question for 2500 years of western philosophy what does mean to be human and wha makes it unique and aristotle have answered the question about animals and i think there is never been more interesting time to be thinkingbout this area because we now have a completely new standard and you get aifferent set of answers and so banded up deciding analytic delivery reasoning is at the core of what it means a dogs and monkeys. Deductive analytic reasoning is t seat of the Human Experience and we get a different kind of answer and emthy and imagination and social ties,eamwork, collaboration et cetera and i feel lucky i have this very eclectic set of interests that happen to be alive at the time in Human History where these two disciplines are on an absolute collision course. Someone in our audience asks do believe that human machines will be achieved [inaudible] there are a number of things that mean by the word singularity so there are some people for whom it means its also called the hard take off where theres an abrupt moment in time for ai to recursively improve and i dont necessarily or i dont really see that. Im in the camp of what is sometimes called the slow take off where i think ai will be getting weirder and spookier and more uncanny until we just accept that it affects everything that requires thinking or intelligent things to do but there wont be a sharp elbow turn where this suddenly happens overnight and from my perspective its totally inevitable and there is a long history of in Computer Science going back to the 1940s do you ever think machine and im a machine and i can think. If you have that kind of secular worldview and you think the brain is made of atoms the computers are made of atoms and its all about the emergent behavior of a certain level of complexity and i think thrillingly in sort of spookily we are on that roadmap so open ai just released a system, languished system a few months ago that has 100 of the 5 billion parameters and if you compare that to the number of synapses in the human brain is about one, 1000 of the human brain thatoesnt sound impressive, 0. 1 bu the average model size within that field of ai is doubling every three months so if you have to do the math that means tha we should expect mods to exist that have a complexity of the human bra sometime in spring of 2023 and thats not very farway. Sooner or later i think the quesons that still feel scifi are going to really starto start the circus. I dont know what thenswers will be but i think this is the riveting thing. There was a story recently about these two compute that i forget the details but they had to commit a gate with eh other and apparently spontaneoly developed their own language for community with each other [inaudible] and in talking about just the need that we have for ai to develop in certain ways into aligned wh what we wanted to do and what our needs are think about the robots solving a problem in a completely different way, is it possible we will get Artificial Intelligence that is mh more advanced and th is able to deliver us results that are more understandable and more in line with what we want to see but that their way of reaching it would be totally alien. Thats a great question. I think both of those possibilities are live possibilities. I think one of the things that is sometimes forgotten when we talk about ai theres a certain lens of inevitability that we can cast on to this question of progress and so forth but there are real choices to be made about the architecture of these systems. For example, its already the case with self driving cars for example that he can do what is called training the system and to end and have a giant blob of Neural Networks and put the camera feed in the bottom and the Steering Wheel commands, the top and you no idea whats going on and there is an increasing science to the question of how do you pop the hood and figure out what is going on and how do you constrain the networks in certain ways but for example, could you be modular and so that the system is naturally divides into the subcomponents that you can then analyze and say okay, i know this thing is doing so let me not worry about this but there is a lot of really encouraging results in that space. I think your question will ai be able to do what we want but in a way that is totally inscrutable and yes, thats not necessarily the only way that can happen. So i think we will have some more agency there that i think sometime appreciated to build the kinds of systems that we feel we can trust. When you talk about Artificial Intelligence people have asked you about [inaudible] in china. Its interesting to watch the questions that i get evolved and when i first book came out in 2011, 2012 a lot of people asked his ai coming for my job and by 2014, 2015 people rescue me is ai going to destroy all of humanity as we know it and the stakes have gone up there. Within the Research Committee the cautionary tale has shifted from one that feels more like yeah, this disobedient system to one that feels more like a system and i use the analogy in the book like the sorcerers apprentice. Its trying to be helpful but it doesnt quite know what you wanted to do and theres the famous experiment that comes from knowing what and Machine Intelligence the paperclip maximizers so you know, its a fictional thought experiment of imagine your paperclip factory and you buy this ai and say we really want to increase the output of our paperclips but unfortunately this thing is so good that it basically turns the entire universe into paperclips, including yourself and your loved ones and so thats a little bit caricatured obviously but that is, i think the kind of thing people work on alignment are worried about. Its not a system going rogue and not a system deciding that humans are amino, need to be exterminated but a system in a poignant way trying to do what it thanks we want it to do but realizing that we realize perhaps all too late that we werent quite specific enough and again, its the sorcerers apprentice or king midas orne of these things. You know, pt of what the alignment oblem or part of what is solving tt alignment problem would mean is feeling comfortable comnicating and intention to a stem like that without necessarily needing to get every specific detail right before we press the button and having a system that i flexible and the scientific term is courage of all but can adapt onthefly and c take feedback and we can say thats not what i meant, hold on and that is the kind of thing i think makes myself and people work o this area a little more relaxed then we were three,our years ago. One of our audience members notes thathe former computer engineer 30, 60 years ago and says Software People all wanted to be system analysts not programmers and i want to actually turn that a bit and ask if youre talking about how we feel now is becoming much more interdisciplinary what can you do with input from other fields and are the people who are in the ai field changing . Other people from other fields were there like i want to go into that field or, you know, are the people earlier major switching over into this different from they were ten years ago . Interesting question. The initial point about being a system analysts i was thinking about the head of tesla has this notion of what he called software 2. 0. Software 1. 0 was programming line one. If asked then line two, why and then software 2. 0 is the world were in out with Machine Learning. You dont write code provide a set of training examples and say to Something Like that. There is a debate over whether they will be a software 3. 0 which is fully general ai system they just have to dont give it explicit Training Data but work with it after the fact and dp three is in that category and working with [inaudible] feels like this soft science thing where your writing is like an essay prompt so its a language model that is designed to essentially fill in the blank in a piece of tech and you couldnt use it to do all sorts of things and say the following is a Python Program with a list of integers blank novel right python code for you. You can say the following is an argument that reads the most common objectives and then it will give you some little essays. What it means to use a model like that starts to feel more like how to work with another person and how do you work something so that you know, the meaning comes through regret the tone you want or the style you want et cetera. There is, i think, going to be this new category of people who are not exactly programmers and not exactly Machine Learning people or statistics people but people who are sort of wrangling these giant models through natural language and that is a new job that doesnt really exist yet. That will require i think a very interesting set of skills and will require certain linguistics so the work you use are very important and use an about how these more models are trained that helps you figure out why they might not be doing something. More broadly, i think, as these questions, as questions that Machine Learning is dealing with d you become more and more human it does seem to invite a certain kindf person who might not hav felt like they belonged to not feel like ty do belong and feel like their skills can plug into that and tre is interests can and thats what we are starting to see and i think that is ahift that isust beginning. Hat were you trying to reach with this book . Who are you hoping will read it and take away general audience, ai community, companies and entities that are trying to apt this . Great question. A couple audiences but o is the general public and relative to other fields in science ai, Machine Learning and even questions of bias and safety have been sufficiently incorrect that a lot of people are aware of them whether or not theyve really taken the time to go deep and understand the underlying sue. Part of what i want to do is say this debate is already happening and i want to try to raise the debate and give pple some of the conceptualnsights and the basivocabularies so that we can feel comfoable talking about the sink that is now [inaudible] and it also there is a huge question of people who went thrgh this career with rticular training that never seememed like it required them o know about Machine Learning and its year 2020 in your hand at these algorithmic recreation so your lawye or youre a medical diagnostician and now youre being givenhese machine versions about answers and so forth and there is a lot of people out there that suddenly need some level of familiarity and i hope the book can fill any there as well and then lastly i hope itan just grow the field. I generally think this is one of the most exciting andost important things happening not just in Computer Science but in the science period. If i can reach [indible] and if i can get undergrads excited about this area and im buggi their advisor, give me a cool project i can don safety and les get started. I think that will feel really good if i can grow that field and that will be a good thing. We got time for just one more question so this is the time travel question. In her book talk about some of the earliest steps in developing ai but what realistically would you expect to see the state of ai both in terms of research and direction . Twenty years was interesting. There is a joke and ai that starting in 1955 ai was always 20 years away and still is but i think, i mean, realistically it will, there will be of course a generational replacement that happens over that time. I think the weight we now have people who we think of as digital natives, social media native and there will be a generation of ai natives that generation of people are being worn now and they will grow up in a world that they might not get drivers licenses and may become a legal, morally outrageous so how can you justify human driving a car, its so dangerous. They will, i think, to understand themselves as inhabiting a world in which there are always different systems with different degrees of what they might call intelligence, different degrees of agency and different incentives that align with our own to 1 degree or not and interfaces that increasingly look like the way people talk to each other. Kids have no problem talking to alexa they find it fairly normal that there is some kind of system that you can just chat with witches remarkable even if you think back ten years and that was totally not someone anyone planned on. Yet, in some ways the boundary will start to get blurry between that technical skill set and just the skill set of navigating the world of other humans in the will increasingly start to feel like they speak that same language and you can communicate by gesture and point to something and have a provide what it thanks describing the item but you say the other one in the way you can indicate with the personal system or something but we will have this new generation for whom that just is weight the world works. And hopefully we will set them up to have a reasonably good world indeed at that point. Ver good. A lot of things to the author of the book the alignment problem for joining us today bute like to think audienceor watching and if yd like to watch more programs with local efforts please visit commonwealth club. Org online. Im john and im wishing you a good day and stay safe and healthy. Thanks a lot everybody. Youre watching tv on cspan2 on this holiday weekend. Television for serious readers. Here is a program to watch out for this thanksgiving day and tonight at 6 00 p. M. Eastern former president barack obama reflects on his life and political career and later historian David Reynolds discovers his new biography of Abraham Lincoln and Space Shuttle endeavor pilots provides an inside look at space travel and exploration. Find a complete schedule booktv. Org or on your program guide. During a Virtual Event hosted by Harvard Bookstore former format michael singer discussed the challenges his face during the counter protests in charlottesville in 2017. In this portion of the program he offers his thoughts on the role of government could play in addressing the current protest against Police Brutality. There are nativity books out there that are fantasized about politics and about government that are constructed around the hollywood eyes narrative where there is a hero and a villain and theres a clear structure and there is a clear take away and so the fact of it is when you are in a crisis especially now were social media and where the extremes on both sides created such intensity and such conflict in the cadence of it is so rapid and its only time that its unlikely the hollywood or the sanitized version will merit anything about the actual leaders went through and the reason its so important is if we are going to handle right now today everybodys minds around the crisis the last two weeks which is horrific racist Police Brutality sitting on top of a 400 years of organized oppression and brutality towards black and brown people in this country and where there has been some progress but clearly not nearly enough and in a lot of ways weve backpedaled and so this is the reality of the experience that people have an people witnessed Police Brutality and everett witnessed 60 average black family has 16 times less wealth than an average white family and the average blacks are four times as likely to be sentenced to be charged with marijuana crimes as whites so the disparities are current and present and real and they rest began in history and those are all true facts but the aggression is if we will deal with them it will require government and not require or government is a means to the end of solving problems in the government we have is not the government of dictatorships and not giving an order but it is humans, a lot of it is on strong different people positions and passions getting together in a deliberative process and a lot of Times City Council chambers and trying to come up with answers using government to get it done. If we will deal in minneapolis there was an article in the New York Times which said that for them to really for them to dismantle their police force and deal with this longstanding problem theyve had that will take at least a year for them to deal with the budget and the police union and the contracts and the way the funds are obligated and the Police Department and that will require government and that will require what this book is about witches people willing with thick enough skin will big enough hearts and who care about the ins and outs of what quote unquote leadership is which is just ordinary people taking on a position of trying to get something done in the book, its hard. Local government is hard. Its most proximate in the most intense kind of government i would say because when you are feeling somebody that is right in your face at the Grocery Store or in the parking lot or the bagel shop as i say in the book or when people are frustrated it will be right in the City Council Chambers and a lot of this will be incredibly intimate and in the book, you know, the book is a pretty intense to write pretty intense to read because you had a lot of open conflict and a lot of demands and anger but to be the devil is in the details. If we will deal with injustice in the country we need to understand it ourselves to watch the rest of this program is at our website book to be. Org type michael singer or the title of his book, cry havoc, in the search box at the top of the page. Good morning. Welcome to the focus on teaching and technology conference. Organized regionally and hosted by the university of missouri st. Louis. My name is tia and i direct the teaching and learning and serve as the assistant vice provost for academic innovation. Much goes into the planning of a conference like this and its satisfying to see a number of records of colleagues across region taken advantage of artwork. For the First Time Ever we had over 1200 registrations. Welcome to our attorney attendees, new attendees and remote who m
comparemela.com © 2020. All Rights Reserved.