Transcripts For CSPAN3 Elliott Morris Strength In Numbers 20221022

Morris argues that the larger purpose of political polls is to improved democracy, not just predict persuasively, argued and deeply Research Strength numbers is an essential guide to. Understanding and embracing one of the most important, overlooked Democratic Institute actions in the united states. And the author is a data journalist for the economist, where he writes about american and elections based here in washington, dc and moderating this evening. Were lucky have jamelle bouie, a columnist, the New York Times and political for cbs news. Bouie covers campaigns, Elections National affairs and culture. Previously, the chief political correspondent, slate magazine and a staff writer at the daily beast and held at the american prospect, the nation magazine. Please me in giving a very warm puppy to Elliott Morris and jamelle bouie, thank you both. Hello. Thank is just on. Yeah, i think it is sounds on the air. Okay, great. Thank you for coming. Hello, elliot, for being here. Oh, my pleasure. Nice to meet you in person. Lets get started, because this is a very interesting book and i was sort of riveted while reading it. And i think theres a lot of interesting stuff to check it out. So im to start and sure, youve been asked this question already. Why did you write book . Why did you write what is . Both a defense of polling. A history of polling, but also sort of an affirmative argument for polling place in democratic. Well, i guess it kind of sounds crazy in hindsight to write a book about polls after 2016 and 2020. But im, you know, im not a pollster. Common misconception, but i read, i guess almost, all the polls that are germane to what im covering and an election forecaster. So all the polls are entering those processes of my work. So after the most recent two elections, when are these, you know, catastrophic of how bad polls performed . You know, my research and polling says well, you should, you know, just expect some of these errors x percent of the time or whatever we can get into how those are made later. And so i wanted to write this book, you originally like 2019 about you people are wrong about about the polls heres science and the art of surveys and the forecasts. And so, you know, doing a bunch of reading from the archives of the American Political Science Association and American Society for Public Opinion research, the Academic Research journals for this stuff and what comes across is not like the statistical take about, oh, polls are going to be wrong this percent of the time. Dont trust them. Every single every single poll you read a real deep connection between or a commitment from the pollsters and the polling academics to democracy. And that is what seemed missing from the broader conversation about polls in the media, least serious, especially from where i was sitting as a data journalist. And you know that thats the thread that i pull on throughout the book that is seemed much more interesting to me. And so thats thats what the book became that ill say that that thats the part of the book thats been that was most interesting to me were reading it sort of not just and internet the details of polling and of sort of what various things meant was interesting kind of from the start. I mean, the book opened with a discussion of ancient athens and athenian democracy and of it opens begins with not just a discussion of polling, but a discussion of Public Opinion and what Public Opinion is and sort of the emergence of Public Opinion as a recognized viable part of political life. And i think that there is i like to hear you talk about that as because i think in a modern and modern america and like modern democratic life, we take for granted that there is a thing in the world called Public Opinion, aggregate views of everyone. But this is this is the thing that this is an idea that had to be developed. And you trace its development which i thought was just a fascinating thing to see. Someone actually put out, you know, chronologically. Yes. I mean, like all this seems so connected. How i first learned about polls and Public Opinion was, you know, in my undergraduate government classes or whatever, which also that long ago and, you know and so when im approaching, you know, how am i going to the question of how am i going to explain polls to regular people or, to very sophisticated consumers of polls, both in a new way and in a way that makes a contribution . The democracy angle seems the natural place to start. So if youre only ever consuming polls as predictions, i guess, of an election far in the future, or even as pollsters like to say snapshots in current time, then youre not thinking the broader implication of what youre doing, which is i mean on a fundamental level, like a microcosm of the democratic process, like every time a pollster takes a poll, at least a fair and a fair poll where theyre doing is democracy at level. So theres you know, there are some old quotes. The president of the American Political Science Association in 1995 who says polls are the most rigorous and egalitarian democratic process there is, again, if theyre, you know, being conducted correctly, because they give every Single Person an equal voice in participating in that sort of micro cosmic democratic process. So that begs the question, like, what . What is the democracy . Were all were all fascinated with or participating in in multiple stages. So so thats where the ancient greek History Lesson comes in. And arguably, you could start a history of polling earlier some early parts of the book talk about the first censuses which might go back as far like the babylonian era or maybe even further back than that in china and. You know, there you have the original counting of people, the original enumerations of populations. You know, its a couple ten tens of thousands of years after that before you have the original all democratic societies and you know, the book doesnt get into the whole, you know maybe there are other democracies in greece we dont really about but well leave those the greek historians. And thats thats the start of any philosophizing about government should match the people want it do even though you know they dont have a particularly egalitarian definition of people what have you. You those like versions of democracy very different than the ones we have now. Right. Theres this saying in american politics right now, especially the right, that were not a republic or a democracy or a republic, whatever are they were direct democracies where indirect ones. But, you know, if you have at the time, right. Aristotle even writing that if the people making up a do not feel represented by their government then its evident its going to collapse into a tyranny or in of itself in the chaos. I dont think were close to chaos in america today, although some people might want that. We might arguably closer toward the other end of that. But if youre looking for something deeper in polls, i think thats a thats a good place to start is the real normative meaning there the real democratic process. Right. Sort of the to go back to the behaviors, the about sense a sense sort of even the the mere act of counting lots of people in that way, trying to figure who exist in the community is constituting that community in a way it is helping create and bring that community into being in in the same way the process of polling sort of like the existence of singular Political Community whose goal you want to represent and part of the book, especially in the earlier chapters, is, is working through this question of should the general will even really be represented because this is sort of the this is the recurring question dilemma of American Government like to what extent should the government should general will should the Public Opinion actually represented in the operation of government and theres there is a portion in the midpoint where you quote rogers is is his name Lindsay Rogers Lindsay Rogers who makes the argument that you know polling is actually quite dangerous because it undermines some of the some of the the deliberate action that the american system is trying to. And so what what i this is a long way of asking have what is your you know what your case for making Public Opinion such an integral part of democratic life like isnt it that we may not we maybe we are a republican democracy but whatever we are deliberations pretty part of the process. So what is what is the relationship of polling and Public Opinion to deliberation so what what ill say is in reading some of the original pollsters George Gallup is a name well all know, but theres others elmo roper, crossley, etc. All of whom you know, are characters in this story of, polling in the book, in reading their work. Lindsay rogers i, i think gives them a you know, misconstrues them pretty heavily. The original pollsters are not saying that the government should do everything that a poll says they are saying in world words that polls should be an advisory referendum and you could think about using them polls as inputs into democratic process. So im a data journalist so think about it and like a model actor where the output is some policy or whatever the inputs are, i guess the quality of certain decisions and the political scientists also say, right, theres this matrix where Interest Groups and activists and Public Opinion and the right decision are all sort of acting in this input into into the into the formula of democracy or government, lets say. And so thats a long way of saying their own. Theyre only one input. Thats thats the position of the first pollsters. And i think the position in my reading and interviewing of pollsters today so you you you know, im certainly not arguing for, again, what sidney verbal would call a government by the survey or by surveys. I thats a horrible idea. Ill put up know that up front the book does not make does not make that case either because that would be particularly to have no deliberation in the process to make decisions based off of what the polls are doing is. Also sucking up a lot of other information. Thats not just what they want or not and not whats good for them. So, right, sometimes arent going to have informed your opinions on things. Sometimes they will have the wrong opinions for them, but other thing that i think we forget polling does is it also gives the government an idea like what policy areas the people need addressing right now so that the questions get are not only, you know, do you support welfare, but are you are you incumbent . Can you afford food . Can you know, does your family have like public transport, get where it needs to go or are you like relying on a broken down car to get places which know would be bad for your quality of life . Do you you need Affordable Health care . If you know, what do you need . So you know, i guess that just raises another question, another point that polls are also not always binary. I mean, the pollsters have what are called open ended responses where people can just, you know, write as much as they want. Thats one of the benefits of modern online polling. And so, you know, all way saying, i think of polls as informative tools that, you know, government leaders could use to better the democratic process in certain ways. But surely rogers has some points that are right, that if you have a government by survey, theres also a lot of danger that comes from that. But i dont think any pollster im certainly not arguing for that. Theres this interesting problem with, the american political system as it currently is, you know, we have only one interest. Theres only one interesting problem. There are there are many interesting problems in china. Interesting. The right word for that. But there are number of problems. And one of them, i think, relates to this, which is that, you know, the u. S. System, the house has been the same size for almost 100 years. The senate has obviously 100 members and population. The us is 330 million plus people. And so its sort of inherent difficult for even the most attentive representatives to really get a good sense of whats going on in their district, whats going on in their state, and certainly for whats going on with the country in the polling can obviously serve this purpose of supplementing for but then you do still get question of how to decide once you have the information and im just sort of are your thoughts about sort of how how unbalances Public Opinion against these other concerns in the interest have an anecdote in the book about lincoln would convene groups of ordinary people in the white house during the civil war to kind of get their idea a vibe check right with with the common man but his Public Opinion backs right yeah i always as a sort of i like all lincoln stories like that because it seems like you seemed like a really dude. Honestly. But thats to say that lincoln would take in this information, but then hed also have to balance whole host of concerns, political concerns, concerns about sort of what he actually are as president are, and then come to a decision to what, you know, im just curious in terms of the actual making process, once you have the polling information. Yeah, well, it might help to think over the counterfactual modern where we dont have polls. So and maybe on a certain policy so to take like gun control right Public Opinion polls today say you know 80 or 90 of the public support universal background checks. And, you know we can have a whole debate about whether or not the polls are representing referendum results or whatever. I think that theyre representing something properly. And 60 or so of the of the country, maybe 5 , depending on how you ask the question again. Right. Its how you do a good pose a thorny subject support banning assault weapons without polling all we really know is that theres never been a proper vote since 2013 on universal background checks and neither a majority or minority in the senate or the house has taken up an assault weapons ban. As you know, full throated supported position since 2004 when the assault weapons ban expired. And so youre left a picture of Public Opinion as informed through the outputs policy that theres no reasonable level of support for these. Two things. Polling tells us that there is so you know. However, legislators are using that information as inputs into their processes are. We can determine that by whatever differences we observe in policy now that counterfactual, i guess, is thats thats one way to think about it. And that would tell us that, um, you know, some are listening to Public Opinion now, i guess, you know, to go back to rogerss criticism and verbose quotes, but we wouldnt expect politician, say, a house member from a district to represent their constituents on every decision, especially if an issue 50. 1 or whatever. Assuming polls can be that precise, which they cant support for back for background checks. But we would would want them to take the publics opinion into account without polls. Guess they can have Public Opinion backs but you cant. Whats the average size of a constituency now . Almost 700,000 people cant have all of them in your office in congress. Right . Rayburns already small enough and. Youre going to get Interest Groups calling your activists, giving you skewed perceptions also. So this is admittedly a hard question to answer. How are leaders actually about this information . But thats because its not its not the only input. So one other anecdote, one of the first examples we have and that the book tells a story of is the 1960 president ial campaign, when John F Kennedy contracts with the first example of a microtargeting here, which is like company that can tell you how different groups are going to vote based on studying Public Opinion data, the actual interviews that they used is by, you know, funny, funny stories that they come from Gallup Survey data that, you know, collects like 20 years earlier. And they do a bunch statistical math that, like, makes conclusions suspect in certain ways and the book tells you why you wouldnt to do that and they are contracted by kennedy and the dnc to come up with this to answer this question which if John F Kennedy makes a speech embracing a some of Civil Rights Act will do better with voters in the northeast black voters and. So they have to, you know, conduct these simulations. They have to semiautomatics being they thats the corporation thats contracted to do the microtargeting they have to take their information from gallup and run it through early computer so theyre like programing every single americans vote and feeling on civil rights on like fortran cards on index cards that you would feed into a computer you dont just type on your laptop back then those computers would fill this whole room and extracting an answer and they tell John F Kennedy yes you should embrace a civil rights in some sort of speech for electoral purposes and you know, at the same time, hes getting canceled every which way from, you know, his brother from other political actors and from his pollster, lou harris, who disagrees. Samuel maddox so even there, what is the Public Opinion at, the time for John F Kennedy on civil rights polls are already sort of offering a muddied signal. So thats a long winded way of saying, you know, we dont we dont expect even right now people to make all the decisions based based on the polls. But it seems that consulting the information, which is what we want, but there are some other probably examples you talk about. Fdr is use of polling and trying to calibrate his message and the nixon who seemed to just really love polling and kind of really used the shape his entire administration. Yeah so before you had Samuel Maddox you had fdr as, a pollster forecaster named emile hirsch. Who was this, you know, or he was the son of a finnish immigrant Family Living in that Upper Peninsula of michigan, which like this in this tiny town, this town, Crystal Falls, only has 1300 people today. So back then, you know, basically no one lived there. And emil irja is a really interesting character. He maybe, like many smart deal bean, just everything. He at one point is both a minor in alaska and also news wires about the mine in alaska and taking other information about, you know, where people had struck gold. Maybe you could go to that mine feeding it to both other news wires and the employees of the mine. There he owns a newspaper in breckenridge, texas, which is at that time a small and this is later or before actually the alaska anecdote at that time, an oil boomtown. And he basically does the same thing there and he calculates how much oil their output gets really indicative sticks. He corresponds some of his finnish immigrants or family, the finnish immigrant Community Back home in Crystal Falls and learns how they actually do the statistics of mining, which is really just a matter of. You hit a rock and if there is or there, then you sampled the area and you mine there. If theres not, you find some new area to sample which is interestingly kind of appalling. This this and then he goes to work on the New York Stock Exchange and he tracks oil and outflows and like early securities trading. I just love this era because its like you, one of your qualifications i dont know, i hit some rocks. I trade on wall street. Yeah, i go to new york. They go ahead, go work for a president. Sure. And then and then he he he gets bored of that and he wants to get into politics and so he, you know, he first pitches the 1928 Democratic National Democratic National committee in 1928, say, can i do electoral math for you . Can i tell you where in the country democrats are, you know, flailing where maybe youd want to put more money there or whatever can address that problem as you want. And you dont have to pay me. Ill do it for free. And they say. No, youre a crackpot. Go away. Thats not how we do politics. In 1928, were going to do what we want, do and. He comes back in 1932 and he says, hey, fdr, let me do this for you. Says that at the post chairman post office. I get you in charge of the post office. This man named james farley, whos also in charge of the dnc the time. So sure, were not going to pay you, but if youre going give us information, i guess well take the information. And hes like wildly successful all he takes polls gallup had been conducting very very early polls and some of the straw polling surveys i might be getting out of myself here from the literary digest is like a famously bad example of polling. And he just the many forecast the election outcome with incredible accuracy. He says fdr is going to win by 7. 7 million votes. Like to put this in perspective, we dont even give forecasts for actual votes now because theres like its not really intelligible, but also a really hard thing to do. We get forecast for percentages instead and wins by 7. 1 million votes. So, you know, like this is the the best soothsayer i could on my staff. You know, i want him in the white house and. So he he provides information to fdr on where where the country sits based on, their approval of the president , but also they feel about like the Works Progress administration a different new deal like the industrial recovery act. So something and and he also just this really also new deal era like 1920 men job thing. Giving out patronage positions in this new new deal bureaucracy the united federal government and only giving them to fdr supporters, people who still early supported them early on or giving them to congressman or other leaders from places that were marginal, that had marginal congressional races so that they could, you know, maybe get someone in their corner to spend more money or more time in the district or campaign for them or whatever the road. So he you know, hes hes like the first election forecaster and hes used not just to forecast elections but to effect the democratic process downstream, which you know, we can think about whether or not a good thing, i think in this case he is in putting the patronage position stuff aside. I mean, theres already problems with the patronage system anyway, handing out jobs. Maybe he like tweaked something here and for marginal efficiency for a for fdr. But you know what what hes doing at the fundamental level is telling the you know president or then candidate Franklin Roosevelt you know these parts these parts of your agenda are really popular, giving people jobs or maybe, hey, theyre not so popular, but people learn about them and they have and they get informed and deliberative, you know, hashtag takes them then then they like it. And so he, you know, he gives speeches about how good the wpa is, about how many its providing, and Public Opinion changes. The wpa like i think in hindsight, we should obviously say wpa is a really good program. Maybe that maybe some people would disagree with that. But empirically i think we should say that and so therefore, i think he has some positive return for democracy during that era. Okay. Well, you also asked about nixon. Yeah, yeah. But he sort of does the same thing with without his own of wizard of washington. Crystal gazer crystal balls, as neal heard, just called at the time. He has what . More like what president s have today, which is an office of public in the white house where you have multiple pollsters you also have interns collecting public data and telling president hey, your Approval Rating went up like 0. 2 points yesterday. Like maybe dont talk about whatever you talked about yesterday or hey, youre in this case of nixon, your your opinions on the Clean Air Act and womens equality actually particularly popular but not among your base. So dont talk about it in red areas. Talk about it in blue press, which is sort of interesting. But you know, Richard Nixons also sort of like a hes narcissistic with the way he consumes Public Opinion data. He really wants to know Approval Rating is and thats about it. So you hear lot from nixon so political scientists have gone back and they find you hear a lot from nixon around the time when hes popular. You dont a lot of speeches from him when hes unpopular, which is kind of you know, thats like the thats one step abstracted from dont talk about things that are unpopular or whatever. Like dont be in public when youre unpopular and show your face when youre popular. I mean, thats like a very narcissistic, you suppose id say you mentioned the bad polls of the literary digest. Theres a portion, the book that is about kind of trouble and polling bad methodology is all of that. And so you i think we you began about the misfires or the the apparent misfires of 2016 of 2020. And i think id be interested to hear more just about sort of what you perceive as the problems in polling and kinds of kind of dangers and mistakes that have been made over the last few years. Oh this is fascinating, but i kind of want to do an exercise for a quick get people if people want to give us a short, quick. So well take a brief break. Polling talks i dont know. Im sort of surprised in general with this to i dont know how many of you follow joe on twitter like religiously like i do he gives reviews of serials which i think is one of your big value ads to society in which usually say like, these are horrible, do not ingest them, especially if youre a child. Its more its more of a bit for people to see me bad stuff. Well i guess we can talk about what other bits of you to but i wanted to pick the next for you to do and obviously you dont have be wedded to this decision like the democratic process but i figure this would be also an interesting exercise to like, you know, do a poll of the people showed up. So i have five serials im going to ask you like a pollster first im going to give you a question. Then youre going to like your hand in order, because are so many of you of the following serials . Um, i weird technical questionnaire glitch, but one of them is gone. So now you know how polling really works. This theres actually only theres only three though, which sort of makes your both of our jobs easier. So of the following three serials which one would you most want to have for breakfast tomorrow . So the options are fruity pebbles. Fruity pebbles basic captain crunch. Oops, all berries, which is all of berries from captain crunch and hersheys kisses cereal. Famously nutritious. So well do it in order and just give raise your hands. Im going to have to count you in order. So its again, fruity pebbles, captain crunch groups, all berries and hersheys gets. Its so fruity pebbles tomorrow morning. Do you eat three . But we have one, two, three of the like fruity pebbles like one, two, three, four, five, six, seven, eight, nine. I see ten people. Okay, this is interesting. Its almost fitting my priors. Captain crunch. Oops. All berries. Yeah i thought so. Okay, thats seven. I see 16. This is also a really way to count how many people showed up tonight. Hersheys kisses. One, two, three, four. This is disgusting. One, one, two, three, four, five, six, seven, eight, nine, ten. I see ten people. I also saw a hand go down which is technically non response base. Okay, its just so the winner with. 44 of the votes and a margin of error of 22 Percentage Points. Captain crunch groups all berries. Ill do it. Okay, so the question is whats wrong with polling . So there are a number of things wrong with that poll. First off, if were trying to represent everyone who would give jamelle an opinion on the cereal tea tomorrow we would want a different audience. This is not a representative audience. We dont have the demographic characteristics. The actual audience or, you know, cereal preferences or whatever is correlated with cereal preferences. Its not the uh, well, maybe we also omitted some, some responses in the question also. So questionnaire design is a really important part of polling. Um and someone put their down, which is a type of polling bias called nonresponse. So the real, the real political world, not the cereal tasting world, the problem, the fundamental problem polling is that you cannot reach everyone you were trying to talk to. So if you take a poll of thousand people to represent the Adult Population of what i think now after this census, around 260 million people, u. S. Citizens, youre missing 259 million of them. So pollsters use statistics to say these are thousand people if they are drawn randomly from the population at large, represent the 260 million. But of course theyre not drawn randomly because maybe not everyone has a cell or a landline phone or is on the internet, always to to do a poll thats thats called coverage where not everyone is the sample maybe. Some people you call dont answer the phone. And if thats predictable off your demographic traits, its not big of an issue because pollsters can figure out, you know, a certain population are responding. So thats thats nonresponse thats one issue theres the measurement error issue, which is like omitting parts of your questionnaire or or, you know, maybe asking people if they support a birth abortion in cases of murder or Something Like possibly bias. Question which is, you know, people ask that question not really so much anymore. And all of those types of errors are are not strictly accounted for in the pollsters. Traditional margin of error. So pollsters do this other calculation, which is okay based on the random chance that ive drawn and an unrepresentative group of a thousand people with all that other stuff put aside, there is this margin of error. Maybe that when i was giving you 22 points or whatever in real life would like three Percentage Points. That doesnt take into account all the other stuff weve talked measurement error, nonresponse error coverage error. And so when, when have gone back and studied this, they find the real margin of error for a survey is like at least two times as wide as the margin error that that we are given. And then i guess to sort of bring in the examples from the book, the literary poll, which in 1948, so what we call a straw poll not a scientific because the way its conducted is the literary digest magazine send you a postcard and ask you for your vote. Or in the magazine, they would print out a ballot that you would cut out and mail back. So already dont have a representative sample. They have literary digest readers or people who are sent the mail. And in the case of 1948, theyre sent mail only if theyre registered as an owner of a telephone or an automobile, which is not that many people. In 1940, especially not people would be more likely to vote for Franklin Roosevelt. So you end up or if youre literary digest, you end up with a poll that says Franklin Roosevelt is going to lose 24 points or lose by 12 Percentage Points from the republican challenger, alf landon, when in reality he wins by 24 Percentage Points. So theyre off by 38 Percentage Points. Thats thats the literary digest. Now, pollsters figured out since 1948, because we have computers and actual statistics now, how to control for some of those problems pollsters use this called weighting, which makes sure that the percent of white and black, you know, nonCollege Educated and College Educated americans would have what have you matches whats on the census. So that way theyre talking to a representative portrait of at least those who were interviewed the census. But that only fixes issues of nonresponse since if you put the precise variables the statistical calculations so not really a fixed problem as you know as i guess reporting on polls that just say x is, you know, support for that candidate is like ten Percentage Points would have you believe its an estimate that has a margin of error and can be severely biased by the process thats generating that number. And in 2016, you dont have a lot of responses from nonCollege Educated white americans. So pollsters end up underestimating support for donald trump. And we have sort of downstream forecasting errors, too. And then in 2020, many pollsters say, okay, well, we learned our lesson. Lets wait by college education. Well make sure we and the interaction of College Educated and race so they make sure they have the right share of College Educated black and nonCollege Educated white americans in samples and so therefore, if everyones being honest about who theyre going to vote for, it should roughly match election results. But 2020, you have a worse polling error. And thats because nonresponse has shifted from that nonCollege Educated group to people who are more prone to support donald trump. So if you look surveys from from the states in 2020, youll find bigger misfires in polling averages and election forecasts in areas with more donald trump supporters. And then if you actually look at polls that interview people off of the voter file, youll find if you want to like predict response to that poll because they know if picked up your phone and you didnt pick up your phone, thats also correlated with support for democrats. So now after 2020, were sitting here like, okay, well, we dont want to repeat the literary digest, so well use statistics. All right . We dont want to repeat the errors of 2016 and 2020. So we will wait on something. Theres no of the number of donald trump, joe biden supporters in america at any one time. Thats like accurate to a decimal point or something. So pollsters cant solve that problem with their traditional tools. So they go back to the drawing board. So in the book we tell the story. I tell the story of pollsters trying to fix their methods after 2020 and they want to design better polls. So they want to interview people not over the phone or online but by mail. Some of them are trying face to face polls now as a way to increase Response Rates among those groups that arent responding to polls via other methods, some polls are recruiting with with mail surveys, with Response Rates of almost 30 . Now. So this comparison, a telephone pole today gets the Response Rate of like 1 . So so 30 is pretty good. And maybe, maybe, maybe thatll work. But we wont know until we observe next election or we have another benchmark survey from a big Polling Group that says, you know, okay, our pollsters arent representing people who own refrigerators or something. Thats a government benchmark that pollsters its important. I heard a snicker, but its important. Its important tool and and and so so youre left with this youre left with this tool that is like this of the democratic process that is used by government leaders, by election forecasters and, you know, News Coverage by unelected leaders targeting policy outputs, stories of the book on there in that regard too. And by Interest Groups and maybe theyll be more accurate next time. But you dont really know because the process thats generating those numbers is breaking down in ways that we dont have, that pollsters dont have solutions for. So that seems that seems very bleak like your polling is not broken, but you know, what we call in statistics, this data generating process is breaking down in certain ways that could lead to higher errors in the future. And you dont know until that happens. But on the other hand, polls are theyre still giving us a pretty good read of both. You know Election Voter intention and issue preferences. So in other studies of issue preferences have found that, you know, Public Opinion on, say, abortion what have you is more accurate perhaps than preelection polls. So we can sort of trust those signals a little bit more. But the average error there be, you know, one or two points on a question that really have figured out that doesnt have a lot of measurement error or, you know, ten or 12 points on something that has a lot of measurement error. And thats really correlated with partizan response. So, you know, youre youre left with this tool that is really important thats breaking down, which, you know, were trying to push actors to like do what the people want, but, you know, theres, you know, theres no surefire solution. So i think what leaders should do is say, okay, this, this poll im getting from x Advocacy Organization or from why New York Times report is a reasonably good indicator. But im not going to make all my decisions off of it. And not going to blasted out in like a Campaign Fundraising email or whatever to try to shore up support based off that like 1 over the 50 mark because it might not be real, but if you have something, a policy thats by 80 or 90 of the people, thats definitely above 50 . If its if its asked correctly before we get to audience questions, i wanted to ask you one last thing, and that is that in the conclusion the book sort of, you know, granting all of the all the problems, you do make a case for sort of greater use of polling in public life. So if you could just say a few words about about that case, but why we maybe should be using a little more than we even do a few words. So so one way this one way this could work to elevate the voice of the people through polls in washington, d. C. Or and statehouses is to have into Interest Groups that are like dedicated to trying new polling methods and testing their assumptions and, their processes, and then giving those signals to leaders. Right. We have pollsters that do this today. The Pew Research Center is a really other pollsters also do and they publish, you know, four or five reports every single day about what the want. According to the pew research polls. And then those get use those get circulated by the press, by the political establishment, an elite in washington, elsewhere. And they get used to steer public debates and maybe to put pressure on lawmakers. But we can sort of imagine in an environment where you dont just have like one group doing this, you have four or five and maybe they come different parts of the ideological spectrum. And all of those errors that you might expect from ideological biases are eliminated by that sort of scattering along the spectrum. And then maybe you also get better polling methods because you have more people trying this their reputations and maybe the fundraising dollars they get are tied to accuracy or to the number and subject. The surveys theyre putting out. And thats, you know. Thats like one thing we could try. We dont have that. So if were wondering about things can do maybe looking for new industries to create or areas to target with money and a and a sort of response from the pollsters is one thing is one thing you could do. But i think the sort of more fundamental problem is the news, News Coverage and i guess conventional wisdom and pub, you know, peoples the publics priors level about what theyre expecting from surveys is wrong there. You know, we were expecting hyper before elections. Well, read the book. Youll very quickly be assuaged of that expectation that there you know, theyre expecting maybe election forecasts with like 99 probabilities or whatever for their preferred candidate. Well, you know, read the book and. We understand why those probabilities arent so instructive, but theyre also damaging to this broader purpose of polling, which is to elevate issue, not preelection polls. And thats as i as i interviewed pollsters after 2016, in 2020, thats what they kept coming back to me and telling me for every poll that they do or every question they ask on preelection vote intention, they do ten or 15 other polls on issue attitudes. And, and thats what pollsters are interested in. So, you know, i think the book, if i may, is important in saying like change channel in how youre consuming Public Opinion polls. Think about them as tools for democracy, something they can do than just predicting an election. And, you know, thats not new. Thats thats, you know, pollster stuff. And seeing that for for a while but to the extent that there are voices and breaking through i think its obviously not and i think i think the people would be better served if they sort of changed what they wanted from the polls also. I mean, thats going to be really hard because. Theres an intense interest. I think theres a weird human or societal interest in knowing future, especially whos going to govern you. So were obviously going to have preelection polls, but if we can change how were covering them from the press, aside but then from the sort of putting my susan on how im ingesting that information and making decisions based on that information and calling my legislators based on information whatever, i think that also puts polls on an on a on a better footing for the future. Right. So if you have a theres a microphone here, so just come on up and, ask a question. And if its not a question out, you. Hi. I had a question about like the role of ama pollster, but i have a about about the role of in the discussion about the book and i think in general, polls are sort of elevated to a position of sort of. That they are polls are elevated from sort of the dirty work of politics. Right. Polls are some source of truth. They sort set the groundwork for the political. And then you can use polling help inform your decision making. Im curious your thoughts on weaponizing polling, the role the poll has, polling has been weaponized politically . Jamal, you said that polling in some way constitutes the like, helps communities sort of constitute themselves, sort of painted the picture. I think we probably grant too much accurate like too much authority to polling but im curious for your thoughts reaction on that and how like polling is used and weaponized by by organizations in politics. I think the easy way out of the of the question is if someones weaponizing polls they shouldnt be weaponizing them you should be using for good or whatever. I mean thats pretty cheesy, but you can you know, theres other sources, information that are weaponized by these people to its not just the polls. Its like, you know, maybe you take an isolated news incident and you use that for your report or your conventional wisdom narrative writing or for your campaigning or whatever. So, i mean, its i think to figure out if polls have been weaponized more than other sources of information or if they, you know, carry more authority in driving x, y or z outcomes. You know, if the alternative is we shouldnt have polls all obviously for that, you know, as per previous hour, i think thats i think thats worse than weaponizing it isolated incidents of weaponizing polls. See so i was actually going to ask this question before the very end, but i think you maybe got to it a little bit, which was distinguishing between horse race polling and issue polling. But i was a staffer on the 2020 Warren Campaign and we would get responses all the time from voters about how we love the candidate we love what youre talking about but we dont think you can win. And therefore were not going to vote for you in a way that almost became selffulfilling. And i think a lot of people some some people have pointed to a specific New York Times poll in october that maybe made a Significant Impact there do you think in general and we can expand this from horse race polling to issue polling as. But do you think in general there is a conservative tendency, small c conservative tendency in polling to crystallize Public Opinion around certain options. Are you even talked about like limiting political imagination a way that is dangerous tendency . Do you think that is outweighed by factors or do you think that that exists . Maybe its a force moderation and is actually something that a service that polls give to democracy. Yeah. I mean if if youre a democrat in 2020 or a republican in 2020 or 2016 or whatever, if if youre partizan and you want your side to win, then a Public Opinion poll is giving you about valuable information and maybe we dont maybe dont we dont like the outcome or maybe people give too certainty to that one poll and we shouldnt reporting on one single poll from the New York Times and like every other outlet or whatever, just like we shouldnt be reporting like a Campaign Poll in every single and treating it as truth as there not being any room for debate about electability of. Warren in 2020 versus biden or the other candidates or or what have you. So, so so i would say if its if that true and democrats want an electable candidate then then its a valuable its a valuable indicator. But i would say theres other data that you could use to figure whos an unelectable candidate. But then before polling. Right. Theyre rationalizing and basically the same process without the data. So you can imagine if Elizabeth Warren, if you have a 2020 election and Elizabeth Warren is sort on the fringe, maybe ideologically or whatever were thinking, whatever the narrative and you dont have the polls that say shes down by four or whatever in october and. I think like working class whites in wisconsin dont want vote for. I think this is the story. And then that maybe you know maybe she does even because like they dont they dont her or they think shes too ideologically but i think theyre i think people are making the same partizans and in washington are making the same rationalizations without the polls anyway, i can just give a quick thought. It sort of its interesting to think of in terms of like the structure of the political system right, sort of when you have a less democratic primary system or less democratic for choosing candidates, then theres a certain amount of Public Opinion still happening. Should have like, you know. Does this candidate play well in this kind of environment, etc. , etc. But there are its much less central. Its sort its value judgments based off of, you know, interest and and and managing and that kind of thing. But in world of of of democratic primaries, lowercase d voters are actually choosing the candidates that will run on the banner of the party then polls do take on a much greater significance and perhaps in perhaps that the solutions to the dilemma here is to you know maybe maybe the candidate Selection Process has gone too far in one direction versus another right and so the way to solve it is to sort of actually reduce the input of voters and increase the input of elites, i dont know. But its just like im thinking of it in terms of sort of what what what does the actual structure of decision, what kind information is that . Does that take it . Does that sort of put out a premium . You might be also like think that the two party system is putting those pressures on people to want someone whos electable, who will defeat the other candidate at all costs in some other, you know, parliamentary or other multiparty democratic system. Then you would have other people would have other pressures, preferences in, the Selection Process. And, you know, polls would serve some better purpose there. So its a maybe like a psychological thing that politics. Can i ask a very quick follow up . So on . And we dont have ton of time so i can sorry, but you can email me i love high you mentioned you talked about like waiting people who respond polls as a way to kind of match up make sure that they represent the population is representative of the population as a whole so the samples are positive. So how does how does that work when you have a 2020 census . That was all kinds of different reasons, probably significantly less representative of the population, doesnt america . Yeah i was i was wondering like, how are they accounting for that . So there are there are already polls coming out that are weighted to the results of the 2020 census. But because we know it under sample this is admitted by the Census Bureau obviously you know theyre statisticians not like partizans or whatever that that the census under surveyed voters in certain states and especially latinos then whatever composition metrics have are going to be biased against those groups. Theres also this huge change which is sort of not really talked about at all so far in the way that the census asked the race question and giving more response for nonwhite categories, which increases the sort of voters of color percentage. But in unpredictable ways, pollsters like to distinguish between people of color as hispanics, blacks, api, something. But the census is sort doing that and they add another category. But pollsters havent added the other category. So whatever metrics they are being weighed upon or weighted on are going to are kind of going to be wrong. Like theres going to be an extra margin of error there. So what pollsters should is probably be honest about this. Say we dont really know the demographic, you know, percentages of the population, these metrics or whatever, and try to control for ask the Census Bureau. Hey whats your of the margin of error for the benchmarks and then add that in in what youre what youre going to add up with this pulse that are really i mean pollsters be honest in telling you this the margin error further survey of a 5545 race includes. Like 5248 in the other direction because of all the extra error or whatever. I mean thats i think more honest it my position after writing this book is maybe we can use polls still that have large margins of error. We have lots of them and especially were more honest about this instead of treating the individual surveys as truth or whatever the answer is, we have no, we have no idea how wrong how wrong they are, how biased they are. Theyre not broken or whatever. Not catastrophically wrong, but we dont know how biased they are. A sincere thank you on behalf of politics and prose. To elliot morris, jamelle bouie, you both so much for being here this evening. Thank you thank you for coming. Thanks. Thanks to everybody. Coming out d good evening, everyone, and welcome to politics and prose. Im brad