Transcripts For CSPAN2 Everybody Lies 20170617

Of Columbia University and harry and will viciously attack seth and the book he has produced yourself will get the opportunity to defend himself. There we will take some questions from the audience and follow that same procedure. All right. Seth . Thank you dan for the introduction. And for inviting me. And this is a book everybody lies. It is about five years of research i have been doing. So i will describe what it is. And for the last 80 years if you want to know, what people want, why people did the things they do, what people are going to do, you basically one main approach. You asked them. You conduct a randomized survey. So you go out and ask people questions. And there is a main problem with this approach which is that people tend to lie to surveys to make themselves look good. So if you ask people immediately before an election, are you planning to vote . The overwhelming majority of americans in the survey will say sure, i am going to vote in the election. They do not want to admit that there were not voting in the election. It is kind of considered socially unacceptable to not vote in the election. My favorite example is the General Social survey asked two men and women in the United States how frequently they have sex. Whether it is heterosexual, homosexual and whether they use a condom. So you can do the math on this. And american women said that they have sex about once a week and use condoms 20 percent of the time. That means these 1. 1 billion condoms every year. American women with heterosexual encounters. American men you ask them the same question and bay city is 1. 6 billion condoms in heterosexual encounters. And if you think about this place technically is by definition to the same. So we are know someone is not on the truth. Somebody is lying. So who is telling the truth . Men or women . Neither according to data from nielsen which attracts every condom sold in the United States only 600 million condoms are sold in the United States every year. So apparently i want to find about the sex that they have. They are all lying. So we have only had surveys but i think we now have a new tool to understand the human psyche which is the searches people make on google. That is the stuff i have been researching for the last five years. People care obama was black when deciding to vote for him. If you ask americans the overwhelming majority of americans, 98 , 99 dont care that obama was black in the election. That was why a lot of people assumed, concluded we lived in a postracial society back in the day. There was this idea voters voted for obama and didnt care that obama was black. Could you use Google Searches because people are so honest, tell google things they might not tell anybody else about socially acceptable attitudes, could you use Google Searches to get the real answer of the effect race played in peoples voting decision. I made a map of racist search volume and this is the percentage of Google Searches, charged racist word, you can guess what it is, the first thing that struck me about this data, in the time period i was using people make these searches in the same frequency they are searching migraine and daily show and economists. It wasnt by any stretch a fringe search. These are mostly for jokes marking africanamericans. That is a big team. The other thing that struck me is it looked very different from the map i wouldve expected of racism. If you asked me where racism was highest against africanamericans in the United States, i would have guessed racism is concentrated in the south, had a strong north and south divide. And in the deep south, southern mississippi and louisiana, you can see the map of darker red meaning higher frequency of searches that it is higher in western pennsylvania, industrial michigan or illinois. The real divide the search data reveals is not north versus south but east versus west. It is much higher, east of the Mississippi River drops substantially west of the Mississippi River. Because people are so honest could you use this data to measure how much obama really lost in the 2008 election . You cant just compare racist searches to vote for obama because it might be places that have high racist searches would have opposed any democratic candidate in 2008 so that wouldnt be a fair comparison. I compared obamas vote total to previous, was ranked similarly liberal. You can you read it in the book, a very strong, significant relationship, places that had highest racist search volume this in michigan. Support obama less than previous candidates and you can control education or demographics or political views or cultural views and nothing changes the relationship, that was a big factor, i conclude obama lost for Percentage Points from racism which was higher than you get from another measure and he got one to two Percentage Points. This paper languished in the academic world for a little bit and very recently the trump phenomenon was started, trump was saying racially charged, making a lot of racially charged comments and people were questioning how he is doing so well even when saying things they are not supposed to say and was racism driving this support . Nate cohen of the New York Times asked for date on the racist search volume, he had data on support for trump in the republican primary and all the variables he could test whether it was age, education, economics, the single highest correlation you can find was the racist search volume for trump. This doesnt mean everybody who supports trump is racist but it does mean some of his supporters were and it did drive his progress in the primary. There are all kinds of things you can do with this data. Measuring child abuse, abortion and pretty much all very depressing topics. It is depressing and horrifying but i put a lot of jokes, there is a lot of value to knowing these parts of the human psyche that we dont arent usually talk about so i will give one more example of the research i have done. If you remember the San Bernardino terrorist attack in 2015, when two americans, Muslim American name, shot up their coworkers at a party and as soon as the attack happened almost within minutes, you saw a huge spike in nasty searches on muslims, the number one search for muslims after the San Bernardino attack was kill muslims which was another one where it is not like what people are looking for google but people do express thoughts on google, they search things like kill muslims or muslims are evil or i hate muslims and they were getting out of control immediately after this attack. After the San Bernardino attack, trying to calm down this islamohphobia, he kind of, addressed these attitudes. With the big news outlets, unlike a lot of people in this room i am an obama supporter but i found the speech beautiful, spectacular, it is best and he talks about moving a sermon, the responsibility of all americans to not give in to fear, appeal more to freedom, it is our responsibility to treat everybody the same no matter their religion. It was a moving speech in all the traditional sources, really love this speech and gave it great reviews in the New York Times or the la times or other news organizations, great job, he hit this one out of the park as far as explaining to people why they should not give into islamohphobia. You can breakdown google data minute by minute. I decide to see what happened in these searches for kill muslims and i hate muslims and these angry searches as obama was speaking, how do they compare before the speech, during the speech, after the speech. I did the comparison and found not only did these searches not drop as obama hopes, they didnt even stay the same. Everything obama was saying totally backfired as far as calling this angry mob. This is surprising, there was one line obama did give it had a different response which is obama says we have to remember that muslims Muslim Americans are our friends and neighbors, sports heroes at the men and women who will die for our country. A huge spike in interest, the top descriptor of muslims on google was not muslim terrorists or muslim refugees but muslim athletes followed by muslim soldiers, the searches date up for a week afterwards. You can compare most of the lines about responsibility, they were a lecture, sermon, didnt tell anybody anything they didnt already know. You compare that to the line about athletes and military heroes, that was provoking curiosity, new information. I wrote this up in the New York Times, analysis, when you write an article in the New York Times, two week later obama gave a speech about islamohphobia in a baltimore mosque and stops with all the lecturing, didnt talk about, to do anything, on the curiosity strategy. Muslim americans, how farmers and merchants, a copy of the koran and Muslim Americans feel skyscrapers of chicago and this speech got a lot of attention. And searches after this speech, a drop in searches for kill muslims, i hate muslims, following his speech, those are two speeches, islamohphobia. The data that you can turn as seemingly unpredictable, the common angry mob is Something Like a science but research has to be done. These people are not necessarily a small number, and come into inner harbor to participate in laboratory experiments. And can use to potentially how they calm down an angry mob. And these great speeches. And the internet, i take the attacks from the other panel. Have a conversation about the book, not the entire book, and internet porn. And in the opening presentation. I enjoyed this book. Some of the jokes i wanted to give the opportunity to share his initial thoughts. Appreciate and deeply misguided some people will take lessons from seths presentation and learn the science of stirring up an angry mob so maybe it is too bad to have done that. Too late to test that any further. I came here a little early and went to the museum of American History where they had an old directorate from the 1800s, an address directory from philadelphia and for the men in the directory and add their profession, a captain, a shop, cooper, agent, a gentleman, gardener, attorney and alterman, gunsmith, whose occupation was businesses listed as shoemakers tool, baker, turpentine distiller and a few others. It made me realize there used to be a lot of available data and everybody used to know everybody. In some sense what seth is saying, the source of data and how much we can learn, part of that is the feeling that we need to learn about such data because people are harder to know about, 100 years ago or 200 years ago, didnt need a lot of polling, how many people they could get out to the polls next month. It is good to have a Historical Perspective about this. Regarding everybody lies i had a couple thoughts, as a person who does use a lot of opinion polls i am impressed at how honest people are, what people are doing, the best ideas you wont get it completely right. 60 of americans vote, 70 , after the election, more people plan to vote, if you plan to vote after the election, and a few more say they did and actually do. Not that far off. When you ask people who they plan to vote for, they are very accurate Hillary Clinton had 52 of the two party vote, 51 , the polls were off in some states i do not think the evidence is not point to people lying to pollsters, differential nonrespondents, the clue to why people are honest in pauls, with motivations. Why you should respond, it is ridiculous people are trying to make money off of me and i spent 20 minutes answering somebody else question, if you answer a poll you might as well be honest. The whole point of answering the political poll is to say i support her. It was a little different, in the air up of gallup polls, not that many people were surveyed. If you were surveyed by gallup, you would be one of 1500 americans, the newspapers the next day, 50 of americans, you would have a big impact and it was rational to respond not so much, you might as well tell the truth, figuring out how many balloons you use when having sex or whatever, people might be misremembering. In the past year, and people arent sincere, what is the true self . Is the true self the person who googles is that who you really are . I dont know about that, when googling rasul racial jokes is not the truth either, just another aspect of who you are. The really interesting question is what is the incentive process to tell the truth . When someone gives a talk that says everybody lies it raises a certain paradoxical m and seth does have incentive to get things right. If he gets things wrong, people like me, set the goal of discovery and would like to learn things. Im interested in the Data Journalism. It is important lets say we have three data journalist and the panel right here. Data journalism is playing a large role in the life of our society. We had a lot of discussion can we trust science just because something is published in the top journal should you believe it . A journal like science or nature is like a brand name. We see this in Data Journalism, the New York Times has to be careful. When they make mistakes they tend to correct themselves. Other sites trying to get hit, Malcolm Gladwell makes a lot of mistakes. I dont know that he has incentive to get things right. Apparently he doesnt have such incentive to get things right. I think he may have an incentive not to admit he got anything wrong. I made enough mistakes in my career like i am already wet so i dont mind being dumped in the tank one more time. Tough to make mistakes first. We had a thing about politicians and the clinton era, it would be great if they could first and a politician to prison and let them out to run the country because in first we wouldnt have the suspense of when they are going to get caught and they would have more sympathy for the ordinary person having been in prison for a while. It should be a requirement for every data journalist to make some big mistakes to get it out of the system so maybe seth will say those things in his second book. What do you think . The value of organically generated data, and how much did you lie when you were standing there . I am a compulsively honest person and the only reason i call it everybody lies big data, new data, and what the internet reveals about who we really are except for me. With the voting, they compare peoples individuals who said they voted individuals, their actual voting behavior, people exaggerate their voting a huge problem with surveys, people give weird answers, they answer randomly. Say they didnt vote which is really bizarre. That played a role, we talked about this before. A not trivial role, random answering, and we have a definitional difference on what constitutes lying a. Andrew think you have to be consciously aware you are deliberately misleading a survey and if you ask people why they did things, frequently not consciously aware of things. They want me to that is one of the areas they are not very good at when you ask other things people are not good at predicting in the future. They are overoptimistic, explaining reasons they did things and people are not so honest when admitting their desires so this one area of who you will vote for in an election is a study over and over again by polls, they are okay or having new problems but definitely a lot of areas where surveys will become a smaller part of understanding of the human psyche relative to the sources we talk about in the book. What do you . Tell us what you see. The book was very well written, material that can be difficult to grasp or not familiar with statistics and this was very easy to understand. You open with a story about your grandmother if i am not mistaken, who you should meet, who would be a good person to meet. That was a very good opening. I dont think we are at the point yet, at least on the Political Center anything that has to do necessarily with understanding the percentage of americans who believe x, y and z. And based off of Google Searches. So i do not want anyone to get the four think that it is a popular case. I dont say we are making a case but is certainly something going on. One of the things i want to ask you know i love that 2008 map, the change from kerry to obama. Then you have donald trump. It also seemed to me that the map was somewhat, im not familiar if youve done this yet, correlated with how the vote change from obama to cleansing. It seems to me there was a correlation where clinton basically did even worse a number of areas that obama was already trending bed. It doesnt mean that it is not because people were racist. They could be racist and that is why they were changing the mind but it could simply be whether they were racist against obama could be the fact that now racial views are increasingly correlated with how people vote whether it is for democratic or republican party. I think that is another thing we have to keep in mind. Something we always talk about correlation. I think that continues to be the case. In terms of using google and the truth serum, i google a lot of strange things to be perfectly honest with you i mean who was on the cast of whos the boss . I mean you know, these are the strange things. Does that mean all is and im the biggest fan of whos the boss . No maybe i was interested produced a particular reason. So again, i think google on its own can give us a keen understanding what is interesting for germanamericans but not necessarily a given thought of okay, why are they necessarily searching that . And then the final thing i will say which i think was, i think i emailed you about this was the book is not about politics. There is a specific thing about sports and being a mets fan versus a yankees fan. With facebook data . Yes, it was used for that. I think this is a type of the thing where in fact you can use surveys that actually replicate the findings that you see on facebook so that we know for instance i had done my own research which showed that in new york there obviously to Baseball Teams. The new york mets in the new york yankees. Four surveys have indicated was that the mets, there will be more fans when they do better and when they do worse there are fewer mets fans. We are finding what was put in the book along the same lines that people, what is it eight years old . Eight years old when a particular Baseball Team is doing well all of a sudden there a lot more fans of that particular franchise. I think that is a type of thing that can also be very interesting and cool to use because conducting polls although are getting cheaper with online surveys people do not necessarily know to answer the questions or do not necessarily have the ability to go out and conduct a survey. Using this type of data can help confirm pull data or perhaps make any finding on perhaps something more trivial but may in fact be interesting to people. Do you agree with that . Not totally. That example is you can measure how much basically, how much a team is winning a championship when every year of your childhood effects when youre an adult. So you will see people born in 1978 and 1961, all of these men were boys in the mets were eightyearold bents and they wanted championship. And you can see this across a whole bunch of teams that you kind of see that obviously it is not perfect but when a kid is eight years old, they have a lot more fans. But the point of the study is the value of the data. In a survey i dont think you can replicate this in a survey. You can see this as a small change over time. You can compare but you can see the changes over time. To really see that type of subtle pattern, you need data on how many mets fans are there among men born in 1978 and how many mets fans are there among men born in 1979 and 1980. You need this for every single team. And you can never see that in a survey with 1000 or 2000 people. So these big data sources you can really zoom in to tiny populations and you still have a big sample. So they have samples for any year in any team combination. Theyre going to have example for that were surveys will not. Basically facebook covers everybody as you were saying. In the book, you cleverly work in some papers based on the that is more like the census of 1810 in a way than the internet, sort of organically generated data. I can see how he of the full population of people youre interested in. You can do these things. It is very different from the google data. We have no way really to look at subtext except for some geographical level and we have no real baselines. But we really need, so we can compare things all the time but as we were staying to find what percentage of people are actually going to vote for donald trump based on searches is because we dont have a way to know what absolute levels mean as opposed to relative levels which are more meaningful. Those are two very separate types of data. I just wanted to insert that there are questions that we demand less or more precision from. It is actually not hard. You can actually use just like crude search data to find out like to predict elections pretty closely. You can get it within five or 10 percent just by looking at who people are searching for. On the other hand predict an election within five or 10 percent is not going to be that helpful. If you want to know what the portion of americans are you know, using balloons when theyre having sex or whatever then be off by 10 percent is not such a big deal. So a lot of it has to do i think with the success of seths work is hes getting away from im going to answer the question everyone else is asking. So the search is not representative sample. But what is this off by week 10 or 20 percent. A lot of questions are like that. So you have to do it where you ask questions where you are not demanding that level of precision sometimes. I totally agree with that. You might know one of the most famous examples of you think Google Searches scientists can predict, they said they tried to protect the rate of fluid in the United States in a given week based on people making searches for coffee or flu in a week. It kind of blew up a little bit. One of the problems with google flu was that our flu models are really really good. You can get really really close by just assuming that the flu is going to be the same as it was previously. So you have. 95 or google data is always going to be as the answer says i totally agree, somewhat noisy. It will never be perfect. It will probably get better as we learn to weight it. It is really a question of if it can be a model that has been welldeveloped for many years. But i think theres only areas that help. I was in it google contemplation makes more sense than google flu. Because he would have no information about whats going on in the United States. Then the noise of the Google Search data will still be kind of nill or close to nill. Are the cdc people doing that . Dino . And ill people that have time on their hands they may be doing google constipation. Im wondering if you might know . I think after the google flu thing came out there was a lot of excitement. They started doing things but the google flu everyone kind of lost a little bit. I think you lost a lot of momentum. I think one of the things that will be interesting as we go forward as i hinted in my last answer, the survey is becoming less and less expensive. At least on an entry ground level. Everyone can conduct a survey. Im going to be interested to see how Google Searches along with survey monkey searches, how all of this quote big data or small data that is cheap can be used together to figure things that are going on. They think if we just have right now most of the surveys we are conducting are in a larger survey or telephone survey. That to me is you know everyone is very expensive. Has resorted get down and people form their own surveys and it is not just an expert like you who understands everything that is going on. That is why i think we can get interesting things going because the end of the day, whats most interesting is not just what we can find once we determine what we want to find but coming up with the questions in the first place which is exactly what you are saying. Which is the questions are most important. What is it like now that we are not asking that this data can get to us . Im just interested, i do not know the answer. What is the answer to that . Do you know . Where do you see a role for this kind of i have done a lot of research on anxiety. And researcher my anxiety is highest in the us. I thought it was going to be new york city and urban intellectuals. But that is not true at all. It is higher in maine and in kentucky and generally rural areas and with low education. Is actually not typical search data that you would see in surveys as well. But then because you have this rich dating can actually see how anxiety changes over time. And so for example it is when two people start for panic attacks . Like 3 00 a. M. But now we basically know through this data, we know how many people are having panic attacks in new york city on any given tuesday evening. And how many people having panic attacks in washington on tuesday evening. And how many people in boston . And we can say what happened in the day . What happened leading up to that that caused this . Is a just random that everyday some people have a panic attack when theres nothing to it . Or was there a clear event a day or two before . I think there are certain situations where it is not true but i think if you wake up it is probably likely to get to google. I think we have to break some of this data down. Because with health conditions, 11 people have criticized my book because they said i search all of these things because im a doctor. For i am a research. I search things but i do not have them. And that is probably true. I think in general research searches are small percentage does not overwhelm the data. But then you can really dig down. On think too many researchers at 3 00 a. M. Are searching for that. I think thats a pretty good idea that a 70 searching, sometimes data is blunt but it could get a lot better. If we got better we can really take of the people who are searching for other reasons. I think that is fair. I just want to go back to you can predict the president ial election within 10 points pretty and accurately based on certain data. And there are things like google and facebook that have much more opportunities to link data within person and generally over time. What kind of use would you see of that kind of data . What would you associate with those databases existing because it is a very different level of quality of the data. Then another search may be. One of the studies i focus on is a study by microsoft. They studied pancreatic cancer. They used data anonymous and identified over time. That the same users are making searches over many months. They say they know who are searching the does not looking at the actual name. Yes, there are links. Then they said okay, so probably has pancreatic cancer if they make a search like just diagnosed with pancreatic cancer. So to get a diagnosis like that you might go to a Search Engine. And then they said okay so here are people who you know anonymous people that we know just got diagnosed with pancreatic cancer in this month. Then i hear from people who are similar to them and never got pancreatic cancer. To have these users over time in the say what were they searching in the months leading up to the diagnosis . What symptoms were they searching . So what searches would predict pancreatic cancer . And they found very subtle things like indigestion. If you are undergoing me will go home and but that is like a really really subtle pattern of symptoms. And i spoke to researchers. Doctors do not know, like our system of diagnosing diseases is not as sophisticated as having a huge group of people and following a time period. So i think that is a revolutionary type of medicine. And with pancreatic cancer if you are told that you have pancreatic cancer, the earlier you are told it can dramatically improve your odds of survival. The ethical question is then what should have been if the Search Engine figures out that you pancreatic cancer. And you have attorneys present cancer 25 percent or 50 percent chance. And i think you can if i have a pattern of symptoms with a risk factor of a disease that can be cured i want to know that. In your case it sounds like youre inflicting most of the symptoms on yourself by reading about them and obviously but this method that you talk about in a variety of you know sort of matching people to most similar cases. Both on facebook and twitter and health records. Why is that not especially with healthcare it seems like an important innovation. Ready think those methods are not coming through as well as they could . Based on the data a lot of it is not linked together. It is kind of all over the place and there is not a huge incentive for people to put it together. A lot of people have been trying to go around the official Healthcare Organization and stuff to try and collect their own data. I think this is one of the things people should keep in mind if they read about this. If a lot of stories about like baseball, politics, mostly because i think it is not that there is no implications outside of those. Those are the three topics i am most interested. I see so you guys use search data when you try to make productions. Sort of. Is that the main source of this type of data that you use . Where do you see these things going . Where have you guys moved beyond survey data . Sure, there is no trade secrets going on. You will have access to the same google unless you all are part of the secret club im not aware of. The way that we use google, at least one of those we certainly is it i will give you an example. Figuring out for instance, what would be effective during the promise of the campaign. Did in fact that harm Hillary Clinton . I found using the data was that in fact, there does seem to be some correlation to when people decided there was that they were going to book for donald trump and when searches for reasonable links in fact jumped up. If anything, if you look at this it might be a stronger correlation then when james comey he said to send his letter over to congress. And i do not think that wikileaks really got asked all that much in terms of after the election. Was at the big effect . I think a lot of my people with concentrated more in james comey. The google data suggested otherwise. That goes back again so i think the question of where americans being asked about wikileaks and their polling . Perhaps not as much as you might have thought. And also, one of the things i think survey data sometimes is more difficult to get out is why are people voting the way that they are . Survey data pulls a very very good at predicting whos going to win and who will lose. Regardless of perhaps what some people might argue. But they are not as good at is assigning reasons for why it is occurring. People often give weird answers. Oh i was thinking that the entire time i was certain. All that had no effect oh blah, blah, blah. So if something did or didnt change was in direct contradiction of the reasons they gave. And again, i think this goes back to the question that both of you were sort of tangling with. I think is necessary that people are lying, is that people honestly dont know what it is that kind of pulls them in one direction or another. Part was finally this one thing that with them over the top. I think that is one example of where we are using google data. I think again, i mentioned this before the other thing where we are using google data is tracking changing minds in realtime. And in general elections most people make up their mind if you month before the election. Very few people are going to change but in fact, in primaries people are much more likely to change their minds and that is where global trend that is specifically useful. The other thing is that there are a lot of races in which we do not have polling data. Were talking about president ial elections right now and there was how many polls 15,000 . It was ridiculous you could beneful is it anything want. Im sure there was one that has Edwin Johnson or other people. But there were many fewer polls for races of the house, senate primaries, make a very good idea. Ive not seen any largescale studies on this. But im very interested to see us more and more people are turning to the internet whether or not these google trends can be applicable to understanding who has momentum in a senate or house primary. Like someone who myself might otherwise be like i have no clue what was going to win. And maybe it was soandso who has a particular chance. For me, those are most of the applications and politics. Again as he said, is just such a small part of the book. To me, there are many more applications and places where perhaps polling isnt as vast. Whether it be sports or health or maybe a topic is interesting to me. Im a huge fan of entertainment i think we all are. Even if we otherwise argue that we are not. But there was this whole story about brad pitt appearing on a magazine cover this past weekend in the story and no brad pitt seems a little sad but i would be interested in seeing whether or not people read that story the same way that the Mainstream Media those numbers of the press read it. I think the particular case we can get outside and i think one of the lessons of the president ial elections that will live in his bubble, where we are all in washington d. C. You cannot get any more elitist than im particularly interested in coming effective this google trend data to figure out directions where the public is thinking that we might not think and we never ask again a question about. As opposed the Current Situation where you only know how bill kristol is doing. Bill kristol has a good sense of humor i will give him that. Understood was a literature which linus arose going to be my engines and just followed ted cruzs campaign chief. I said that i looked that he probably knows where the train will end up going even on their is no sign up. And it was exactly right. I am going to open this up to the audience. Seth, do you want to Say Something before this gets guided another directions . Any and all questions are welcome. Please wait for the microphone and introduce yourself. Ideally questions under that question mark. Hello. My name is julia abraham. I just came out of general interest as a member of the public. I wanted to know, surely there are commercial companies who are developing methods for going through all of this data because they want to know, things to sell. You know they have a strong motivation to make money by tracking peoples behavior. Are there methods that the commercial entities are developing that you all produce to deal with politics and psychology . And other things that may be commercial interests. I think youre former employer is particularly interested in this. You can go ahead. Theres definitely a lot of uses in marketing. I think marketing is one area so ive been talking to people, their stopping using surveys to the east as people like what products theyre going to buy. And they said they dont correlate very strongly with what product they actually buy. So i dont know why that is the case if they correlate with voting. I think this gets back to one of the competing sources of information. In an environment where you are and to who is buying what the maybe a survey will amazon now of course they not so much about who buys what. They know what i buy and so forth. So that creates less of a value could you can just run it directly with people are buying. You need to ask them. For what theyre going to buy. Is always going to be harder to know people are going to do. I think there is good data from amazon and other companies. Of course, most of the ads you see on the internet are based on all of your previous actions that google and organizations are aware of safety are in gmail and you see an ad, there will be comedy as he will be determined partially by the content of your emails and things you have searched for to some extent. And so that i think is the most obviously commercial accusation. I talked about this as well. One thing that they do is rapid experimentation. They basically, i think for now does more experiments than does in the entire year they can very quickly divide the users into a treatment and control group. And they can show different groups different versions of this. So they cant change the typing or change the wording and see which one gets people to be on the same moral click more or do whatever they want more. I think the question of how we can kind of researchers utilize these tools and something that is underutilized and at the game you know there is betterments and academia, you recruit a bunch of people over long period of time, and it takes a month to set up and you do a small experiment like one or two small experiments over 30 people and i think where i think i can move is they can use some additional tools to run a thousand experiments and say this is what we found and i think it would make but i have not seen it done at all. Is over here to middle of the table jonathan. A little more skeptical. Hi, i am jonathan just here for my general verification. Im curious. You talk about weighing google data. How is it highest, obviously theres going to be a classic and agent bias filled in. When youre looking at a google trend for the us, are there any unusual ways we perhaps do not think of that is biased . Again, not presented above the general population. There are a lot of random situations that they just find more and more. One of the things i found is that i was doing all this Research Using trend data and they frequently go with trend data so who searches for what. But if you these over and over again, one thing i found is that he tends to be an outlier. And i think one of the reason is because the residents from d. C. Are different from the people that make searches in dc. They commute to d. C. And make searches and is one of the issues that come up in a data source like this. That i would not have thought about before hand. But it does come up so there really are a lot of issues. I think it is unfortunate i think initially there hasnt been too much Methodological Research on this because i think a lot of survey people are skeptical of the data source but it is starting to change. They just did a beautiful study on using Google Search data. They have these hardcore methodologist that are much more detail oriented and i am and have pages and pages of things with data. So im hoping more and more people, methodologist start studying this to find a lot of these in ways that you can weight it and make it better. I was going to say, as you are basically suggesting on the democratic primary side, you found the data to be pretty much useless because Bernie Sanders was like 9000 trillion searches ahead of Hillary Clinton. Hi, bill former employer and coauthor with seth. I am biased. One comment and one question. I thought this was an incredible book, very well written and interesting, really funny. And really insightful. The question is, baseball and politics are interesting and fine but i think the most interesting part of the book was the stuff on child abuse. Because kids cannot just talk about it. There is sort of a sense that they are going on google and say why does my daddy hit me . And it just seems like an incredibly powerful mechanism. Something you always they cannot get from a survey. He mentioned in the book that you are working for consulting with some governments on that. I would be interested to hear more about what is happening on that. Yeah, there is a disturbing element on searches. And one is that kids obviously they are not like younger kids who dont speak or use the internet but older kids do make searches like my dad hit me or my mom beat me. Like really frightening stuff. You know it is obviously horrifying but i think one of the things that also using the data is that during the Great Recession there was a big drop in official reported cases of child abuse. Which is surprising. You would think that all of everything you know about child abuse is that when people are out of work that is a big risk factor in child abuse. But he did see at the same time the rise in searches, disturbing rise in searches of kids and child it is. And i argue with a lot of data i think it just became harder to report child abuse. Everyone was overworked and Child Protective Service people were on hold forever. So it is disturbing that you see kids making searches and the official data, it was really disturbing. I, yeah im still kind of talking to them and it moves a little slowly and im not exactly sure exactly how to use the data but it is a continuing conversation with some of the organizations and how they incorporate this data. I do not know exactly where it goes or will go. But i think it is really important and it is an area where obviously i think the official data can be misleading. You will not get a survey asking kids this. So i do hope that we will incorporate some of this data. In a similar area, there are things that data comes in more slowly than what we would like which is drug addiction data. I think there should be searches that like to figure out where opiate overdose i think it could be pretty useful. Lets go over here. I am trying to make you run around as much as possible. Good afternoon i am todd wiggins. I am enjoying the presentation. I would like to ask you hypothetically what you see coming down the road in 10 years. I predict that my reading is going to be evergrowing industry. It seems to have developed in my last 20 years from couponing and you talk about advertising to modern couponing which is essentially what we are doing with Search Engines. And intently as we want to know more about how people think nine aspects of advertising the finesse of security or immigration so where do you see yourself in 10 years so i know what to invest in right now as far as stock is concerned. [laughter] i think what youre getting at is that there is a scary element to some data, we can talk about the fun things on important things like child abuse. Is that companies can potentially take advantage of people and one is a set of by columbia professors and you apply for a loan and either get it or dont get it and they have data on everyone applies for loans, what they said on the loan application and when they pay back the loan. In the event you can conduct whether you pay back the loan based on the words that you use. Getting at theres a scary element to some hoff this data. Welcome talk about the fun thing order important things like child abuse but the other area is that companies can use the data to take advantage of people and one study i talk about in the book is a study by columbia professors of peer to peer sites and you apply for a loan, and then either get it or dont get it, and they dat on everybody who applied for loans and what they said on a loan application and whether they paid back the loan, and they found you could predict whether you would pay back a loan based on the worded you use in the loan application, and some of the relationships were just kind of weird, like god was one of the biggest predictors. Using god is a big predictor the same on the government side, i think. I think in the book you emergency the movie minority report. I dont know if you have seen it recently. They had immense processing capabilities in movie and do calculations rapidly to predict murder but dont have the cloud so they are constantly carrying these really large hard drives around. Most of what i think tom crews. Tom cruise obviously governments will be tempted to gain access to this very detailed data on citizens and noncitizens and use them to macfrost and i dont think d make forecast and i dont think theres a of how to predict that. In u. S. We long relied on systems not communicateding with each other to keep the government from using large data sets inch europe theres much more leeway for governments and systems to communicate better. Of course, conversely, europe had much stronger restriction us on what corporations can do with data and much stronger privacy protections. Well see how those two will evolve. Lets go to the left against the wall. Following up on your comment about europe. You mentioned that of course americans tend to be very open on google in terms of expressing honest opinions. Other countries, of course in europe, have a much greater sense of privacy, in asia, of course in japan, you have a certain sense of individual distance and china you have the great fire wall. What kind of how would you characterize responses in countries like china, japan, korea and germany . I havent done as Much Research in other countries, because i dont in the any other language so becomes challenging. A real american. Becomes challenging to do it so i dont know as much. I think it is interesting. The kind of premise of the research in the book is everyone is really honest on google may not stay like that. Someone date paper comparing peoples searches before and after snowdens revelations. They measured whether there was a lowering embarrassing search rate when snowdens rev layings were mid. We found there was a drop. It could be that in the future, people make fewer of these searches. Thats part of the paper. The categorizing increase in the number of searches for snowden . Probably. But that probably the category is embarrassing searches and started asking people how embarrassing is this search and mores of them war child porn or really, real like herpes, like you expert, and one on there that made the list is nickelbeck. And that also dropped apparently after snowden [laughter] lets go over here and i was just wonder i havent read the book so i might be exposing myself for not finishing it but did you look interest the along olines pan treeatic cancer, what too terrorists search for right before the commit the crime and everybody who does a mass shooting searches this but not everybody to searches this does a mass shooting. The government may be doing that. I dont know. Dont have access to individual level searches. Im a little bit skeptical. Just looking at the absolute number is think people make a lot of horrible searches, more than you expect. May just be that the false positive like the false negative ratio is higher than we expect. People have bad thoughts more than we thought, and that, like, the government really shouldnt be intervening, not just because of legal and privacy reasons but just because of data science reasons that a lot of people are in the same group and look horrible and never go through with it. I think this relates to the appoints of questioning whether what someone does that this most embarrassing is their truest self. You could imagine a super google of the future that it wont even need you to search. Just track your thoughts at all times and then you say, well, then what is your true self . Whatever thought you a have the most frequently. Might have to do with, like, going to the bathroom or whatever. Because every once in a while you have to go. I dont question a lot of what you found, but i again getting back to the title of people lying, this idea theres a truth out there which is what people are really like. Think we have to watch out about being sort of identifying truth with the latest technology. I mean, i agree theres different actually came up, tim woo, a colleague of yours at columbia. He said that, like, he he also brought this point. What is the real you . And he is like, one study is you compare searches for gay porn and people who say theyre gay and that in states where its hard to be gay, like mississippi or tennessee, theres lot fewer people who say there are gay but almost as many gay porn searches. So it seems like a lot of men in my reading in places where its hard to be gay, make gay porn searches. Might be married to a woman or tell facebook surveys theyre not gay. My assumption is, that means theyre gay and tim i like, well, does that mean theyre gay . Maybe theyre not gay. I just thought it was obvious in that case that means theyre gay but i see tims point. Well, theres something called the tyranny of measurement or labeling. Its been count it out having the word gay or the expression gay sort of changes things just the way people can think of themselves, its not something i do, its something i am. But also we talked about this earlier, youre example in the book about the efficacy of different schools and you compare them based on how effective they are in raising kids test scores or getting them into top colleges and thoser measurable things, yet people go to school for other reasons than to get into top colleges and to get higher test scores. So sometimes its super clear. Some of the clearest examples through these medical examples but a the outcome is kind of clear. This what you want. Or an election, who is going to win the election other. Cases like obamas speech or the outcome of a school is tougher because its not always clear what you can measure. Of course at a stat statistician i dont say dont measure things but we have to be aware of following in love with what we found. The thing about dat explosion is we can tell richer stories and more complex stories and get much. Measurements itch dont think the available of Google Searches or facebook or twitter will mean we now have fewer things to measure. We now have way more things to measure. Fully agree with you, you have to be careful, not just one measurement. I guess also what youre saying i we have the ability to measure things that are perhaps a smaller part of the population. No matter what state you are most men are straight or prefer to have sex with women than with men but we can now get in on that pick population, that small we help study that in the way that perhaps a larger survey where you poll and people get maybe 10 or 15 or 20 people who match that description and there was no real way to tell x, yoz about them but now we have a real way to get on the specific things. Lets stay away from more specific levels in this area and go to heidi. Him a thank you for the presentation im looking forward reading the book. One observation. One is that the Health Communication stub is supertricky because if we try look at how many people are searching ebola and me might think a lot of people have ebola. And have questions about how representative Google Searches are. Thinking just about the number of americans who have access to internet, 85 of herons have access the internet, particularly 40 of over 65 dont use the internet. Wonder how that might skew your the second question is whether theres a change over time because of the increase of number of time people are using apps rather than google. How does that skew your data . I think the 15 not using the internet, that will get smaller and smaller over time. Just kind of gets to the point that the data is not perfect. Definitely 0 not going to be 100 correlate with the population but let other people worry about that. Yeah. I think the first thing that is striking when you first search google trends and now in the public, is how powerful the patterns are and more it could have been done wouldnt have surprised anyone if Google Searches calm out and it was all this noisy, crazy datament the bible is searched more in, like, new york and least in mississippi. And so kind of the data does tend to work as best we can tell pretty well and will get better over time. Dont know too much about the app stuff. Havent looked into it. But i mean i think changes over time are longterm changed can be tough to measure with this search data. One of the one thing you see is searches for science went down over time, the percent of searches that include the word science, and some people use that as this is showing that americans are losing interest in science, but i think just the earliest observations of google were much more interested in science. Longterm trends can be problematic. I wanted to pick up on your questions nice were kind of critical, and i think criticism is so essential to making all of this work. Not criticizing seths book a few months ago there was something on the internet and somebody who i guess maybe someone wrote a paper and it was passed reasons and said people are most religious in the midwest, not the south, and then there was a lot of, cud chewing about what that meant and someone had used some religion data that was a mashup from several different churches and turn out some churches report attendance in a different they want other churches the data were complete crap would have been better to use google, im sure. What happened, people was, this is hard numbers, lets start explaining them. And its one thing that we sort of lack is a great way is a way to sort of engage with these claims. Way for poo tome take a claim and a claim and say this is kind of interesting could be wrong, lets bounce it around. Student cant die either. Social science is terrible at doing this. Dont know if journalism is better or worse but i think that it what i was getting at earlier, with the rise of Data Journalism or google science or whatever you want to call it, think this is a great opportunity. For us to try to figure out how we can be skeptical without being nihilistic, and i wonder what you think about this. I definitely agree with you. A lot of most of the dat in my book is public and kind of i think a lot of people are we talked andrew and i talked about the peer review versus Data Journalism. I think from my experience, writing a Data Journalism article which with which maybe 100,000 people read is a lot harder than writing an academic paper that five of your rivals buddies read and attack. So i think i get emails from grad students and undergrated grads not infrequently critiquing my assumptions and making sense of them. Think if we can do a Data Analysis that is very public, that may be a better way to move this thing forward and what they got a they put themselves on the line. They say theyre judged afterwards by how good their predictions are. So thats kind of very powerful and their audience is better than the other ones. I will say we get if i Say Something wrong im thrashed for it. My email, my twitter feed, i dont even know what else dish dont income anyone has tried to call me yet or send me a snail mail, although a few people have asked me that was interesting. Agree with you. This type of anything that is is open that allows people to get a look underneath the hood and everyone has an equal opportunity to do so ensures that the data process and what i we claim from data is were all better off for it. Right . If i have my secret little data set im claiming im making x, y and z from, you have almost no way of checking it, even from the public polls the underlying survey data is not released until six months plus and no reason that anyone will give that polly data over, versus this data to tends to be more open and we ick make the determination whether or not this correlation makes any sense, whether or not this data was looked at in the wrong way, and to me thats very important. There are a lot of people these days who have access to the internet and can write a lot of crazy things and make a lot of crazy claims and people will believe them. And it is very important to me that were able to check those. Some cases people will believe no matter what but with this kind of data we can check them right on the spot and anybody who has knowledge who wants to figure out what the truth its, can do so. I agree. The social sciences have sevier weaknesses in speed with review processes were and the small numbers in basically every sub subarea and i dont think the numbers are always small for a good reason. Five people have written bat about a topic, and many more people can a little more transparent than typically is. So i think thats helpful. I got phone calls a few times. Wrote about a case one time and all men doesnt surprise me. From massachusetts called to thank me and call me an american hero. Lets go back to the left here. Second table. Yes. Thank you. Hi, joe schultz with scripps. One thing that came it earlier was being sure to ask the right questions. Another element is asking the right people, and one thing i think is true with polls, with goggle, is that its not necessarily representative, and theres this almost calibration step that we need to make. Think someone mentioned the whole sanders vs. Clinton google trend responses having ahere response from sanders. Absolutely. But after a couple of come calm comcast you get results. Some poll wes had pretty big misses with public polling in wisconsin and ohio. A shame. How do you go bat making sure youre able to calibrated your dat so youre able to kind of validate yourself and make sure youre right Going Forward . Goetz back to the sample. Its challenge with google data. With surveys you can ask people demographic questions and Party Identification and it turned out, as we said during the campaign itself, that a lot of the fluctuations in the polls were clearly attributable to changes in relative nonresponse of gift gift different groups. The google dat extra you dont have peoples demographics. I think the one thing i will say is that i think youre right, that the longer the dat is around the more you can calibrate the models and know how to weight them and stuff. If with the predicting election things, you cant predict an election from the volume of searches people make. They search for trump but doesnt mean they like him. Might mean they hate him. One thing found that reasonably correlated is that you can predict which way people are going good based on the order in which they search the candidates. So, like, 22 of search with the word clinton in it also cloud the word trump so people search for clinton trump polls or debated but if they go clinton they go clinton trump polls compare that to state voting theyre much lisch likely to go clinton, if they good trump clinton, theyre subtle indicators will be predictive and as we do more and more elects we can weight them and calibrate them and figure out whether you can add information to the polls. I think we have time for one last question. A quick one. And then well wrap. Hello. With the possibility of Artificial Intelligence bots do you think that some of these searches could be manipulated . Thats one of the values Google Search data relative to other sources is that google has the smartest people in the world fishing out who is a bot. I think a lot of searches are by bots but google puts a lot of energy into eliminating those searches because advertisers dont want those to be included in the data. I kind of didnt an arms race between google and the hackers but a bigger issue if you take a random search. Going has the smartest bots in the world working on this. Is that right . All right. With that i thing were done. There should be wine and cheese somewhere. Thank you again for coming and have a good evening. [applause] book tapes hundreds of author programs throughout the country all year long. Tuesday, we are at politics and prose bookstore in washington d. C. For the recount of hitlers trial for high treason following his attempted involvement in a coop. Saturday, back to hear sue mcken share her experience reporting on terrorist groups around the world. And thursday, creamer books where Lawrence Goldstein looks at the attack of the submarine. Many of these events are open to the public and look for them to air in the near future in booktv on cspan. You are watching booktv on cspan2. Here is our prime time lineup. At 7 00 p. M. You will hear from former skin head leader, author of romantic violence. Then at 8 30, we sit down with author allen who offers thoughts on skiens and communication. And then Rachel Pierson decribes here experience as a resident physician. At 10 00 p. M. An after words, mike lee recalls the work of the forgotten men and women who fought against a large federal government during the founding. And david kaut mcculla presents ee