Good morning. Im adam segal, correct the digital at Cybersecurity Program has the council just want to say a few words of thanks and a few words of welcome. First let me thank shannon and lauren and alex for helping put this together and making sure that it runs smoothly. We want to thank pwc which is provided the funding to help us run this as as a number of othr programs in the digital cyberspace policy program. Like many of the other think tanks were doing more and more work in this space, on Information Operations and russian Information Operations in particular. This month we published a cyber brief giving some policy recommendations to liberal democracies and how they can counter russian Information Operations. If you happen to have seen that please take a look. Its available on the website. Also last month we rolled out a Cyber Operations tracker. Thats the website you might is seen when you walk in. It is a list of publicly revealed state Cyber Operations. So espionage, more disruptive and destructive attacks, goes back to 2005 we have approximately 200 known incidents, and the plan is to add more as they happen and as they become more known to us. Please check that out. Its updated every quarter. If you have an incident we dont know about please let us know. We will go ahead and add it. Please find me today if you have ideas and suggestions about how cfr can be helpful in this space, what we should be doing. If we do something we shouldnt be doing or if there thinks we should be doing that we are not, please come and find the and thanks to spend the day with us today. I feel be a great discussion. Good morning, everybody. Im sam feist, and were going to the conversation this morning on hacking into our election systems. The next panel, the canadian panel, is going to focus on hackers and other compound hackers and other mischief makers are trying to influence Public Opinion and influencing the public and how that might change boats. Votes. This panel is going to focus on how mischief makers but actually try to change votes. And hacked into our election systems. We are going to talk about the scourge of our election system, Voter Registration system, tabulation systems. And if you think back, think they might be able to hack here or there but can do, in fact, an election . We can just look to the election that happened last night in atlanta. If you havent, if youre not familiar with what happened, it is yet undecided the democrats had at this moment by 729 votes in one of americas most largest cities. Any mischief could affect any election. In fact, one of our panelists who happens to be the secretary of state of you get told about race this morning in a state that was decided by just one vote. So these things do matter. Id like to start by introducing our panel. I mention connie lawson, the secretary of state of indiana and destroy the president of the National Association of secretaries of state. To her right is matt blaze, an associate professor of computer and Information Science at university of pennsylvania, recently helped organize the defcon voting machine hacking experiment to test the vulnerability of our election Voting Systems. To my right is michael sulmeyer, the director of the cybersecurity project at the Harvard Belfer center. Michael previously worked as the director of plans and operations for cyber policy in the office of secretary of the significance of thank you for joining us. We will spend about half an hour visiting and having conversation, then at about 9 1d have you all ask questions for our panel and we will continue the conversation until about 9 45. I want to start with connie. Connie is responsible for the Voting Systems in the state of indiana but also works with secretaries of state across the country. Would you describe our Voting System, in your opinion, as currently safe from hackers . And mischief makers, or are you particularly concerned . Will do you fall in the continuum . For small, obviously people in your severed this before. There is no evidence that any thoughts were tampered with in the 2016 election. I think that Election Security has always been a priority of sectors the state, and i think that the email that every chief election official received in august or september of 2016 changed the way we do business. So were making especially cybersecurity a priority, and we done a number of things we can with department of homeland security, the fbi and, to become, to make sure that we get the information we need. So the number one activity since the 2016 election for the National Association of sectors a state at the sectors state has been to improve the key medication between the intelligence agencies in the United States, and we as chief Election Officials so that we can get information we need in order to prevent or react quickly if there should be a cyber attack. Just going to put you on the spot. Are you comfortable at this point know that the system is perfect, but are you comfortable that we have done as much as we can do, are you comfortable if there were an election tomorrow in indiana it would be safe . About the time you say youre comfortable thats when you should be worried. Im never going to send comfortable about it but im always good to say that im going to be very vigilant. But but i do believe that were doing everything we possibly can in india to make sure our elections are safe. I am very fortunate not every state has the support from the general assemble. My General Assembly appropriated 1. 4 million so that we could make sure our system is secure. We migrated our data. We have done a number of things to secure our outward facing west side. We certify our Voting Machines in indiana. We will recall the voting Voting System effective Oversight Program so we know where every machine, every type of machine, every serial number, every tabulating machine, we know where it is and when it is in use. So i feel good about what were doing. Weve been told by dhs and multistate information sharing analysis and we were doing the right things. Matt, do you feel good . Well, you know, i feel good that connie is doing the best that can be done. But whatever is the best that can be done is almost certainly either not good enough today or the honeymoon is going to end very, very quickly. A little bit of background from my perspective, im a computer scientist. In 2007 i led teams that were contracted by states of california and ohio to do a talk about and review of the election system technology, including the Voting Systems in the back in systems from the vendors used in those states which turned out to be the same vendors used in the other 49 states. What we discovered in 2007 was that these systems were riddled from top to bottom with exploitable security vulnerabilities in virtually every component of the system. And some of the vulnerabilities were sort of coding errors, bugs in the programs that could be fixed. Some are more architectural, particularly in the socalled dre systems, direct recording electronic Voting Systems, a touchscreen Voting Machines that record voter selections electronically in their internal memory and the systems that process those. Interestingly, we know that those can be exploited and in many cases they can be exploited with no more physical access then you would need as a voter or a poll worker at a precinct here but theres been no evidence that they actually have been exploited in any election. We have to kind of walk a fine line between saying look, this technology very desperately needs to be improved, and falsely telling people that our elections are illegitimate. So i dont want to say that our elections are illegitimate but i dont know how to prove that they are not. In some of the cases that technology where using doesnt really tell us. That concerns me greatly. What do you think the biggest vulnerability of our Voting System is . First, thanks to the council for putting this on a for having us here. I was that deities i never feel good about anything. I never feel comfortable. The challenge is one that strikes me at least of risk reduction, not elimination. So you have to set your standard, your objective in se way thats reasonable. So you are always going to have some level of uncertainty. The challenge and the opportunity to reduce that risk is much as possible and i figure the General Assembly in indiana wants to help you do that with some appropriation. The challenge that every seat is it doesnt take much to have an effect. You dont need to have National Wide intrusions. Reducing the risk of gain unauthorized access, thats what the risk were trying to do re, gaining unauthorized access. You do that in a couple key jurisdictions, get the timing right, you can change count. You can make things a lot more difficult for the folks were trying to make sure that our elections are conducted in a way thats High Integrity as possible. You can really complicate compd effort in just a couple key ways. Thats my perspective on it from my experience. It doesnt take much and we have to reduce that risk is much as possible. I just want to make sure everybody understands that the last election that we question was the 2000 election when we were virtually using paper and punchcards. If you think about the way we do elections today, ive been a county clerk and i been on the ground and a bright elections. I did that for eight years. I will tell you that there are security measures that are local Election Administrators take that make it literally, well, it can make it very impractical for some to get to our Voting Machines. First of all, these machines are kept under lock and key. Most of them have a visual scanning of the facility. We know who comes and goes. They use logins. So we know again comes and goes. We do public tests once the public tests run before an election, we know that the votes are recorded properly and that there are no votes that would be present on election day before some actually comes to vote, those machines are sealed. When a Bipartisan Team arrives on election morning they cannot the seal from the machine and a record the number. One of the first things the Election Administrators do at night when they get the results from the precinct level for the vote center level is they look to make sure that the serial number on the lock that was cut off machine is actually the serial number those place on the machine after the public test. And so and a Bipartisan Team, again delivers these results. So is it possible . Yesterday is a practical . I would say no. There are many physical aspects of these Voting Machines and tabulation machines that take place and have taken place for years that it just seems, i mean we dont put them out in the middle of the courthouse and say have added. So i covered the 2011 election, 35 days ill never get back, but as a result of the florida recount the federal government and state governments, the federal government spent millions of dollars to help replace many of our election machines. The florida system used punchcard paper ballots that punchcards at work come as we know, sometimes not always easy to read and that was one of the issues. We replace them with, to a large extent, these electronic touchscreen bows that didnt message should that have papers at all. There were completely electronic. In indiana what percentage right now are those machines . We have 92 counties, and theres 50 plus the use the dre. However, the dre that we used to have an audit trail, paper audit trail inside the machine. Its a mirror image of the ballot. Its not a voter verifiable paper trail but there is a paper trail. Did the florida debacle make things worse, matt . They made them different. It essentially shifted as from a system in which you could have very vulnerable to smallscale retail mishaps. To one in which smallscale retail mishaps probably have become less critical since the help america vote act. But we paid for that by exposing ourselves to catastrophic failure in ways that we previously worked. Our elections are far more dependent on the integrity of software, and thats something that we simply dont know how to do. So it we are all the money in the world to design our system today, what sort of equipment, machines, system, what would you come if youre in charge of voting and the United States of america, how would you have americans vote to get as the safest possible outcome so that at the end of the day, the day after the week after elections the losing candidate or anybody else cant come in question is a this note is not right . I would hire matt. Okay. [laughing] thats the fine idea. [laughing] the two things i would you say is youve got the paperback of some way to have an audit trail on every machine, and yet cant wait to turn off the win some of these devices. Im with you and physical access. I dont have concerns about anybody rolling in the courthouse and having at it. But wireless access to a network is a problem. Some of the machines would look at the report called hacking jabs found it could even turn off the wireless. It was not possible to turn it off. Thats the security problem. Connie, if the legislature gave you 20 million, 200 million, what would you buy . I have no idea. I need the experts but i would certainly be doing a lot of research. I would say the most important thing is education for our local Election Officials. A number of states are governors have said come have set up cybersecurity councils. We have one in indiana and were working with a local elected officials. Were running phishing email campaigns so we can educate them on what to notice, what not to notice of what to click on, what to notice before they do that. We are working on multifactor access so that the passwords are stronger and so those are the things that we are doing in the state of indiana and i think most secretaries are doing that as well. I i wish you say that very first election i ran as an Election Administrators was 1989 in hendricks county, indiana. We use lab machines which are pretty lever there is probably one in this use them now but im dating myself but i will tell you that it would make you feel very well if you saw the way those results were taken in. We would get a written total from the precinct, and you have a tally sheet. I remember sitting on the floor with his huge tally sheet and numbers get transposed and youre adding all these up. It was a disaster. It really was. We finally ended up with the results that was there and correct, but two, 3 00 in the morning you are still working on this paper towels. People are not that patient today. The worst thing we could do with tape we would have to go back to all paper. What we need to do is take about how we can make our Technology Work the way we need for it to work. If you had billions of dollars what would you do . Its funny, im in the one branch of Computer Science, but have most of my time is spent pointing out how terrible Computer Science is building reliable things do we really are truly terrible at building Reliable Software systems. It is literally the first problem of Computer Science. We dont know how to build programs that dont have bugs in them. That may at some point in the future, there may be some breakthrough that makes that less of a problem but it has not yet happened. Arguably this problem is getting worse rather than better as we build larger, more complex systems. Whats the solution . Well, the best solution that anyone has come up with for elections as a concept invented by professor ron at mit called Software Independence. That is to say, were going to use software to get his all sorts of benefits to have computerized election systems, but we dont want the integrity of the election to depend on the integrity of the software. Because that is simply a herculean task. So the technology that exists today that has this property of Software Independence is a combination of two existing things that we can do today. What is whats called precinct counted optimal scan ballots. That is, ballots where the voter marks a ballot or maybe use about parking device to create a paper optical scan ballots that is fed into a reader at the polling place that records the selection and keeps the tally, and then captures the physical ballot and scores it stores it in a lockbox. That technology has advantage that it maintains an artifact of the voters choice that the voter actually marked. The second thing you do is make sure that the software that is doing the tallying has not been tempered with our doesnt have bugs in it. That can be achieved with a technique called risk limiting audits where you get statistical sample of the polling places, do a manual count of the paper ballots and ensure that matches the electronically recorded results, if it matches, great pic if it doesnt match and you know have a problem and you have to do more of the recounts. The combination of being both of those things properly gives you this Software Independence property that just eliminates a wide swath of potential vulnerabilities that are really hard to counter in any other way. But why i watched the British Elections this summer, okay . Reasonably well developed nation, United Kingdom held an election for parliament this summer. They used paper ballots. They are tightly at each constituency. The people who voted for candidate x, theres a pile there. There are few all watchers checking. They count, recount, the write them all that and so it stands at a microphone and read off the results without ever touching a computer. The only one who seems to add them up, the Television Network back in london where they literally add them up and do the arithmetic but thats it. Whats wrong with that . Is in the foolproof . Why do we have to get all fancy . Seriously. Theres nothing wrong with that but the United States has i would just a patient . We are. We are americans and we arent inpatient people. But more serious problem is u. S. Elections are the most logistically complex in the world. We vote on more contests on a single ballot. We have more different ballots. We have School Board Elections and the dogcatcher election and referenda, depending on where you are, bond issues and so on. In england they are voting for, its a parliamentary democracy. They are voting for a single representative in general in these elections, or maybe one or two issues. Here i vote on about 20 Different Things in philadelphi philadelphia. Michael, you worked at the department of defense. The word went not said russian on this panel. But thats the backdrop for this, at least right now. Do you believe that the russians or any of the bad actors but we would use the russians for this, tried to act out elections . Want to hack our elections . Are actively trying to break through all of mans fancy systems . Or is his religious something that, they probably are overstating . Do i believe that before Intelligence Services would love to gain unauthorized access or hack into systems that would reveal information . Absolutely. Wood for Intelligence Services loved to be able to gain access to system to try to change tallies . I think in their dreams they would love that ability. Its hard for me to see a proposal being discussed in the kremlin and the Security Services and they say, no, les let that one go. We are not trying to hack the election, are we . Who knows. [laughing] the point is i think that they may want to be able to achieve these outcomes as predictable and understandable. Being able to see the causation from intends to actually being able to realize an objective, thats the tricky part. Its not always theres a doctor evil plan hatched and then everything falls perfectly into place. It usually lets see what happens if we try moving some pieces around the chessboard, right . Sent a bunch of phishing emails, sue the reviews. What you gain unauthorized access to one system, what does that open up . A lot of times you have the play on the line of scrimmage and audible and it doesnt always work according to the playbook. The Voting Machines we spent most of her time talking about the Voting Machines but it just want connie to walk through for those of us who dont count votes at a local and state level, just walk me through when it i am voting in indianapolis and 90 voter, i go into a polling station, i push my vote on a machine. What happens between the time i vote and the time that the second estates website reports the total . Just walk me through who is in control of those numbers and how does the vote, information about that vote move, my thing all the way on . Once the vote is cast, obviously it up to the Election Officials, a Bipartisan Team of Election Officials at the precinct level to bring the results back to the county level. And the county totals how do they do that . They will do that depending on the type of machine. D. O. E. It would be a recording device. I believe on optical scan it is as well. Im not as familiar with those but they bring those results back. Is it a key fob or do they write a numbered out . Its some sort of Electronic Device that they bring back. It is run through a tabulation machine. Theres a machine that reads a device, the usb port or whatever. And then the precincts are total together and then the counties call the results, we call the county so they get its not connected to the state anyway. So a million, 200,000 votes in this county and somebody confesses not but anyway thats a lot. Someone calls your office on a landline . Yes, but the results are not final for ten days. He has remember folks are able to cast provisional ballot when they got to the polling place, for example, if they forgot their photo id. They have ten days to go to the Clerks Office on a county level, take the id and provisional ballot somebody on the telephone back in your office heres the vote and the type that into a computer and the computer does the math and then it gets published on the website . Thats right. Thats how the world knows about it. But again the counties have the opportunity to do their audits. They make sure the results are final and they dont actually certify the results to the state for ten days. So matt, if michaels friends in these intelligence foreign are trying to make mischief what are the points of failure, we talked with electronic voting machine, but what are the points of failure if it is in any secretary of states system . I worry less about the secretary of state statewide position than the county for a 3000 counties in the United States, roughly. About 2500 of them have responsibility for running elections. Which means we have some are in the neighborhood of 25003000 different local Election Administrators. Some of them are quite good at protecting their systems. Some of them are less good. Theres pretty wide variance among them. This has nothing to do with intentions or goodwill. This is simply a matter of very widely different capabilities. To the extent that our Voting Systems have been secured, and weve seen horrible, we would look we see really horrible exploitable vulnerabilities. Its a threat of conventional corruption. Somebody trying to get themselves elected mayor, or sell votes or what have you. Nationstate adversaries were not even in the threat model of the systems, have been designed against. And so when you think about the capabilities of the National Intelligence service like the gru or really of any country, they had capabilities but certainly include everything that a current candidate may want to do but also they are going to have additional capabilities. Theyre going to potentially do supply chain attacks with equipment that get shipped to be tampered with maybe before its receipt. They may do attacks against infrastructure that is being used. So they have Additional Resources and capabilities but thats not the most serious problem. The most serious problem is they have an easy problem to solve than someone wants to cause the results to go a particular way. State adversary adscs satisfied with simply disrupting an election, casting doubt on the legitimacy of the results, causing chaos on election day. And that is significantly easier than causing a a determined result. They both have more capability and wider range of things that may satisfy their goals. So michael, elections are run by states. They are run by counties. They are run in towns. Is this a policy issue for the federal government in the same after the 2000 recount mess, congress got involved in with billions of dollars appropriated for new election machines that are now causing another problem, but is this a a federal issue . If so, what should the federal government be doing to address this before breast next president ial election in three years . The federalism questions are thorny, no doubt about it. I think that for the federal government to say, federalism is too difficult so we are out, good luck to the states and locals, i dont think the federal government can take a pass. What i would like to see some sort of a playbook come that the federal government be able to put together for best practices and counsel. My colleagues at harvard for to get a playbook for campaigns. I see no reason why federal government couldnt provide an update a playbook for state and local authorities on these issues as well. All of those agencies are involved and we are talking about all sorts of things. For example, communication. There are seven pilot states that have msi, multi state Center International auto of new york is giving the states monitors so that they can be monitoring the Internet Activity on our election system. Not all election systems are on the state system and so it becomes a little more, gated than what you might think. We are doing the seven pilot and hopefully by the primary in 2018 every state will have a monitor on their internet activities so we can be informed. Give a sense of trauma and ministration is making this a priority and are they doing what they need to do or how would you grade their efforts so far. Knowing that we are three years away from a president ial election. It is hard to tell on an Administration Level about whether hearing some of the specific dhs people talk about how they want to helpful and those folks who are in the specific office they see that as a priority in the question about how fast they can pursue clearances i would rather they be able to do a drug deal with the intelligence and declassify rather than working through an entire process to give clearances but life isnt. How would you grade federal efforts and much of the federal government been doing now will be have time . First of all were three beers away from a president ial election and were 11 months away from Midterm Election so there is not a lot of practice time for the significant election and there are extremely capable people in dhs and obviously i cant speak to the administration posture on this but certainly there are very, very capable people who need to be empowered to assist. You think they are empowered . I have no opinion. Okay. On that note i want to open up to questions from everybody here so raise your hand and wait for a microphone will come your way and then also say your name and your affiliation. Hello, russell with the hoover institution. I have a policy question that is probably better for a later panel but since we have everyone appear im curious given the ubiquitous nature and vulnerabilities it seems to be that deterrence from defense is not feasible and so this question is more for you, michael. What is a good policy that says adversaries us elections are sick or sent and the risk is too high for another nation involved . The challenge is if youre going to leave bags of money on the lawn overnight and then try to deter and talk tough about dont take that money and then your stunned the next morning when the money is gone the deterrent is not quite the model there and i think there is a lot there that we can bring across the board to bring it inside and forget locking the door for a minute. No defense is perfect and i completely agree but for you talk about deterring and imposing cost you do have to look at are we really not wanting to put more resources into architecture and very unsexy thing that would make it much harder. Im the ceo foreignpolicy interrupted and we havent talked about what happened and weve been talking about the actual voting but what we havent talked about was before the voting and in particular facebook and social media and how people are influenced by that is the next panel. This panel is focused on Voting Systems and such and the but i have a question for connie because not only on the facebook level of social media level but what are states doing and are they looking at this and is this something that states are getting involved in . In the information in how elections are being influenced. No. I dont think the states are involved in that. Obviously we have known for years that foreign nations have tried to influence peoples opinion here in the United States regarding candidates and how they should vote and obviously we do voter outreach and we encourage the accessibility of our Voter Registration and accessibility and i think indiana may be the only state in the country that has an app called who are your elected officials and you can find out from School Boards and present how to contact your elected officials so we are doing everything we possibly can for people to get a correct information but i dont have control over facebook or twitter or someplace like that the puts out the wrong information. Yes, in the third row. Please wait for the microphone. Mike. In addition to security the other election issue going on at the moment is oppression and to what degree does the risk of technology or technology impose risks in terms of wiping people off registration rolls and some of these other things that seem to be going on at the moment. In other words before people even get to the voting booths we have not talked about we talked about the Voter Registration rolls so lets talk about that for a moment. Go ahead. How safe are the systems that you happen to be here and how safe are the systems so that when someone walks into the voting booth their name is on the role and they are actually allowed to vote that a mischief maker has any race my name before i get there. I have no idea whats going on in indiana and you seem great and im sure your systems are terrific but this is definitely a point of vulnerability particularly for our nationstate interested in disruption. In many states and many jurisdictions the pull book is the polling place in the Electronic Device and those devices are often have on security weaknesses in your name is in there and at best going to be casting a provisional ballot. How often are either of you is there a paper backup. If i walk in even though the poll workers in the uses the ipad to check my name is he or she have a book under the table in case everything goes haywire is that likely or not likely . There are 2500 different answers to the question in different counties. Let me just say that when we are notified last fall before the election that there were two ip addresses that had been responsible for getting into the illinois Voter Registration system and getting into a small county in arizona that had actually allowed that ip address to have access to arizonas statewide Voter Registration system. We checked our Voter Registration system from januarn indiana has 92 counties. We have 6. 7 and i see professor here and we have 6. 7 million residents in the state of indiana, 4. 8 million registered voters and we checked 15,500,000 logins into our system and the reason we had to check that many was because that was how busy the counties were. They were looking at petition signatures, they were registering voters, candidates were filing their decorations. All the absentee ballots and everything was going on so that is how busy these systems are and that is why states are looking at things like multifactor access to the statewide Voter Registration system. We are winning a time system. If its after midnight maybe it is just supervisors of the election that have access and all those things we are looking at. If i were to go before and present but in indianapolis and their ipads quit working is there a paper book that i keep going back to paper. Yes, they would have a pull list. Yes data breaches on a large scale our daily events and they dont even get reported unless there on the aqua facts or Office Personnel management and the only reason we havent seen is the of not yet seen a largescale data breach of Voter Registration databases and its because no one is seriously trite. The individual states and counties their best efforts are are going to be no better than what the office of Personnel Management or aqua facts or any of the long lists of incrementally complex systems that have been catastrophically breached our and we may be in a honeymoon where it hasnt happened yet but its only a matter of time. On that. On that company no, lets take another question. Please wait for your microphone. Can you tell me how many states common to independent system that you talk about or how many are working on it and conversely which states in your experience are the most vulnerable . There are a few states that are using both exclusively precinct countering optical stan plus risk limiting audits. Virginia decertified all their machines and they have risk limiting audits that i learned recently they havent after the certification. Rather than before so there is some adjustment that needs to be made. Colorado is a significant one there so there are a few states that are starting to pick up on this but there are the exception rather than the rule. I will say that the National Association of secretaries of state have a winter meeting and we will be talking about risk limiting audits and weve got [inaudible] coming to talk to us about paper talk exercises and Incident Response and all of those things are on the table. Its not like anybody is ignoring those so i want you to know that those have been a definite priority of cheap Election Officials prior to 2015 but its been a high priority since the time. When you go to an event like that for example what is it whats the most important thing you are telling the secretary of state and what are you saying to either that you have to do what youre trying to scare the daylights out of them and what is the message. Fear is always a great motivator for anything but in this case theres enough Awareness Among the secretary of state the dont need to do that and you got to give news you can use. Everybody now is geared up to the reality of what is at stake so the most helpful thing i have seen that my colleagues bring to the table is Something Like a playbook or implementable actionable practice that the secretaries and other colleagues to use. Esther, second row. Adam getty with bionic security. Ive spent the better part of six years of my life doing nothing but Data Security and data integrity for high clients including the federal government. One thing weve been discussing all morning is the security of the Voting Systems in the voting process. The intent should be that the constituencies of our democracy trust the results. Ultimately that is the goal. While there is a lot that can be done it is being done to ensure the security of the system and security of the process what i havent heard discussed this morning is there a way to have a common private, only voter verifiable and reconstructive audit overlay on top of the results that can ensure the trust and integrity of the outcome of all of the things such that any of the lack of integrity of the rest of it may not necessarily cause the outcome that nationstate emissary might be speaking, voter disruption or cheating of the candidate. The audit is independently verifiable and i know in your case, i didnt hear you mention anything about [inaudible] overlays or the random function overlays to some of these things and im interested as to why . The basic problem is the systems that do that are extraordinarily complex and essentially they make our elections more dependent on the integrity of Underlying Software systems in particular is part of the vote casting process so you may have the effect when you look at the overall usability of decreasing the confidence in our elections rather than increasing them. The other problem is that the heavily [inaudible] problem we want elections to have transparency and we also want them to have a secret ballot. We wanted to be impossible for someone to learn how someone else noted and we wanted to be impossible to prove how you voted because we dont want people to be able to coerced into revealing. That is a pretty difficult set of things to achieve. Instead i think we have to to rely on making systems simple and publicly audible with processes that include the train of custody, a public ritual where we do the risk limiting audits and so forth. In practice that is likely to do much better than any fancy photography and i say that is a fancy photographer. You use the term risk limiting audits so for those of us who dont live and breathe this stuff just explain that a little bit. The basic idea is that once you captured the paper ballots and electronically counted them you want to make sure that the software that counted those ballots hasnt been tampered with or hasnt bugs in it that is reporting incorrect results. So you sample the precincts in a various races to a manual recount in every race and are they statistically significant sample and verify that what you hand count matches what you have and do it every time. And you do it every time before the votes are certified but state . Thats right. If you discover a discrepancy up to do more and count. Is this happening in indiana and if so how does this happen and what triggers it . We dont do risk limiting audits are now but we have an Oversight Program working on that right now. Weve had conversations with colorado and i just heard on the call this week that new mexico is working on that as well. We are going to be doing that and that something i deadly support and i think we should. I will say that after the 2016 elections we had a Congressional District in the state senate race both were in a recount and we recounted an entire Congressional District and the results were the results. Exactly the same. Exactly the same. Atlanta might be calling you for more advice. Yes, maam, in the very back. Austrian, from google. When you talk about desperate using the United States is doing their best to just train those in what. Michael, do you want to take a step on that. Your mr. Policy guide. Great question. No. The government is not doing i dont think government is doing enough. Its not that there is ignorance about it but i think the biggest problem thing is theres no singular step up and set of skills that will solve everything or equip everyone to do everything technical. Theres a lot of this kind of information that you can learn effectively. There are even say free youtube videos to just learn about how different systems work. You said that for her exactly, yes. There are some type of skills that require a large type of federal investments to steer people towards but that actually doesnt need to be the way to solve the stem crisis as a whole. Question. Yes, sir. Allen, sydney, austin. First to comment on the federalism on the constraints of the federal government responsibility here. I would note that before the constitution obligates the federal government to guarantee a republican form of government to every state that the elections are compromised that would be called into question. But my question concerns the discussion has centered around whether theres any evidence of impact on us elections in terms of hacking the voting results. Where do we see internationally . Are there other a lot of cells that are subject to electronic procedures as well the patient state adversaries may not have chaos and uncertainty and maybe even in installing particular candidates do we see evidence around the world is relevant to us . Probably the largest country that uses electronic Voting System is in india. They have a customdesigned voting machine and questions have been raised about the security and integrity of the design they are using. It is a paperless dre system. But to a large extent the us has been on the leading and leading edge of ruling out computerized Election Technology post, help a mocha vote act and i think we are seeing and we have to look inward as well as outward to see what is going on. I will also say that the situation very much reminds me of Internet Security in the 1990s when technologists were basically warning that the systems that we have on the internet in general are going to be and are insecure and will be attacked and for a while people were saying you are just chicken little thing the sky is falling and sure enough the sky fell. Really havent been the same since. The situation with electronic Voting Systems is remarkably reminiscent of the situation with general Internet Security of the 1990s. The example you mentioned that there were issues in illinois and in the last election did we ever actually learn who the hackers were and who was trying to penetrate the systems and is that actually been discovered or announced . I have information on that but i dont think i can say. Oh you dont have your clear that. Yes. The city to whoever is watching in russia. No, i cant say. You had me at article four of the constitution and then in terms of where else you could look at where its vulnerable and you can look at an attractive target for people who want to be in this business and this is eastern europe. Russians have an interest in so not surprising that they would be poking around. I think if there is some aspiring grad students were watching and a quick masters thesis would be doing comparative studies of all states looking at ukraine and recent elections and to get travel support money and investigate. We are not done yet but i have yet to be convinced that any of these systems are superior to paper so far. As we have been talking for the last 45 minutes paper sounds better and better and the idea that many or all of the systems in india are in the Worlds Largest democracy and anyway i have trouble buying this. That is neither here or there. Yes, sir, in the middle. Thank you. Fred from a drone safety and security company. Id like to ask the panel a question. Are there are other similarities for the similar markets and cyber precautions we take there or the coming transportation attentiveness transportation markets that we are going to be looking at. Are there any Lessons Learned from those that can be applied to the voting cyber issues . Well, for my money the Financial Sector has invested the most for the longest one of time because they realize they had money to lose so they took it upon themselves to defend themselves and i think the federal government has the best type of relationship with an Information Sharing Analysis Organization that is managed by the Financial Sector. There is definitely good lessons to be learned there. The question though we start getting across Different Industries is what has been the governments role in requiring different types of transparency, reporting about intrusions. Defense contractors are required to report to the military when there have been certain kinds of intrusions we can start to think about how transparency and reporting can lead to better practices going forward. Id also point out that in the financial industry theres a straightforward Feedback Mechanism that tells you how much you should be spending on security. We know how much we stand to lose and we can do pretty straightforward risk calculations that tell you what your budget given exposure is. In the case of the integrity of elections that Feedback System doesnt really exist. Unfortunately we spend far more on election campaigns then we spend running elections themselves. Most election operations are in counties and the budget for Voting Machines and running elections competes with the budget for fixing roads and building fire stations. Probably the most important lesson we can take from this is that we need to think about how much we value the integrity of election systems and understanding how much work to put into this. You mentioned that they have invested money recently which is i presume a good thing. Do you sense that the other 49 secretaries of state are in a similar situation or are more of them underfunded or not funded either at the state of the county level and is this where the rubber meets the road . I would say im very fortunate as secretary of state of indiana to have the support of the General Assembly and i dont know its common practice across the state to get the appropriation that i was able to get to modernize the system. I think as the attention continues to be drawn to these issues that the states will step up and fund but obviously there are some folks would like to see the federal government step up as far as the funding goes, as well. Yes, sir. Strategic insight group. This is a question for matt. To what extent would we know if these things have occurred and there are cases of intrusions hiding for 500 days in the corporate sector and that is question number one in question number two if we put in all the various things weve been doing are you convinced that that would secure against this sort of thing . The answer to the first question is it depends. Unfortunately a lot of the systems the audit trails are just as vulnerable as the other aspects of the system so there may not be good forensic investments on this intrusion and in other cases there may be signs. In 2016 certainly there were indications of attempt and i would say that with the current design we cannot be universally confident that it hasnt happened. It is probably only a matter of time before it will. The combination of risk limiting audits and an optical scan and paper artifact of the voters record gives us a pretty good assurance within a statistical certainty that the count of votes cast is accurate. It doesnt help us with the other piece of that which is the disruption piece, Voter Registration systems and so on. Those we have to do the same hard thing that we do with any other on my system. Training, resources into it, keep systems uptodate, and so on and have to love them enough to pay enough attention. Yes, in the back. Microphone is right beside you. Thank you. Tim white, spectrum group. Is there a Critical Mass of public concern that the issues that you have been explaining this morning or in fact does the public generally either not care or believe that the tradition of Electoral Systems in this country have always kind have been doubled around the edges and this is just another way to do it because without that Critical Mass of Public Awareness not press awareness and not awareness on those involved professionally in the process but people care that much . What you think, michael . No, i dont. I wish there was a broader mass and its stunning i know to most were at harvard and the rest of the world is like cambridge, massachusetts. There everyone is focused on it. But im concerned also about the leadership deficit on this topic. If no one wins political points by talking about this this is a partisan thing but you need a leadership level of it to generate that mass and it can be bottomup but it could also be topdown. To talk about how an election is rigged before the election is difficult and when youre trying to improve peoples confidence and get it on i think you need to come to the internet indiana section of state and answer the telephone and you would understand that people are concerned about it. We get calls every day. What are they saying . What is going on, i just read in the paper about the russians hacking the state of illinois and is indiana good . Is there Voter Registration fraud in indiana . You know, just whatever is in the news. We get calls and a number of calls. I can report every week on the types of calls that come into the office and this is an Election Year and i would say i have had just as many calls regarding the security of our election this year this off Election Year as i have ever had. Ive had more but just as many. Autry, american university. Following up on that question is there not a sense that theres a politicalization of this issue where someone feels that someone concerned about elections reflect more upon a particular type of election of our latest president ial election or particular election locally and how can we truly remove this issue from the partisanship and the polarization right now that we have domestically . I think were in a rare moment where its more bipartisan than it has been in the past. Every election theres a most recent loser and the risk is that it looks like the people complaining about it are simply upset about their vanity of losing. After the 2000 election we saw bipartisan interest that led to the passage of the help america vote act. Arguably, it had significant problems and led to that technology but it was at least a bipartisan effort. I am optimistic which is rare for me but i think we are at another one of those moments we are approaching another one of those moments. You have someone over here thats been raising his hand for a long time. I wanted to go back to the comment that was made a moment earlier about the great vulnerability of data breaches. Are these just information, theft issues, in other words is the risk just a risk of loss of privacy or does in the data breach do you also get an ability to manipulate or disru disrupt . The goal of a nationstate ever terry who attempts to breach a system one with Voter Registration information would be disruption and so they are likely to want to first of all ensure their future and continued access to the system and they may try to delete they add serious records to make it appear that theres been widespread registration fraud, disenfranchised, selected voters, and in some way that advantages them or simply cause havoc and delete everything or cost systems not to be ready on election day. Particularly when we look at this from the perspective of a rival Intelligence Service attempting Information Operation against us we have to look at a very Broad Spectrum beyond merely linking data. I would like to add that in june at a Us Senate Intelligence committee one of the under secretarys of dhs stated that 21 states had been targeted and so that was a surprise to all 50 states because we had not been told that we been targeted. The language that we use is so important because what its targeted means and this just this last week, i believe, chris from dhs said that you need to remember that just because someone targeted or tried to get in a state doesnt mean that there was an actual breach and that when we use the word targeting it really meant they were scamming or trying to get in and it made us all very concerned. Some states and indiana was not a state that was targeted but it took three or four months after that to find out the 21 states that had been targeted or scanned but we know of no additional states besides illinois that had gone the breach. Yes, sir. David martinez, state department. I agree with the moderator that i failed to be convinced thus far that the safe and most secure way of guaranteeing the integrity of the results is paper ballots. I also worked briefly as a government, my home state of new mexico, where i was able to observe pulling and practice and saw that that also presents vulnerabilities but i was wondering if connie you could expand a bit on what some of the cost, political, logistical or financial of return to the rebels would be. I thank you made of one point about in a 24 hour news cycle we dont have the patience to wait for that account so that might be a political cost but what are some of the other costs and return to such a model . I cant imagine. Obviously if its a balancing act. We need the technology but we also need to verify what we are doing and i couldnt answer the exact question regarding the Financial Impact that it would be a large one, i would think. I know the hours with me and my staff to count the first election that i ever administered in indiana and that was a small special election so i cant imagine what it would be in a county like marion county, indiana, our largest county in minneapolis i can imagine what it would be. They count their absentee balance and they have over 300 teams of people by person teams who count just the absentee ballots so can you imagine what it would take in addition to that to count the actual paper ballots. At the end of the day paper ballots may make it more competent for the russians but they dont necessarily lead to a more accurate count or do they . When you say paper ballots going back to the stone age. So youre talking about one 100 counted so i think everyone in the Election Security world is pretty uniform in advocating paper ballots whether the question that were talking about is whether or not they are completely handcounted. As a as opposed to the optic skin. Right, optical scan has the benefit that you get some assurance that your ballot is being fed into the system and a record of it is being made as soon as you submit it and thats an increase in integrity but the problem is we are also now dependent on software and so that is why you need to risk limiting audit backup behind that. Were just about out of time and where the council of formal relations where i want to end with a simple question and is close to a oneword answer as possible. Ill even give you your choices. Looking ahead to 2018 and 2020 are you optimistic or pessimistic that the system will be materially safer than it was in 2016 . Ill start with matt and ill come down. Im convinced that the actors will be emboldened. Im optimistic. Pessimistic. Well, all right. On that note thank you very much. You for coming today. [applause] the next panel if your joining starts at 10 00 oclock. Thank you very much. [inaudible conversations] we have everyone sit down please thank