Transcripts For CSPAN3 National 20240702

Front lines of our nation and bringing us together in this important fight. It was an honor to be with you today and thank you for the honor of joining us today. So good to be here. Good morning, everyone, i hope you are able to get caffeine in your systems, get your blood flowing. I do want to acknowledge all the media that has been with us the last four days. We appreciate all your coverage and everything you guys have done for us, so thank you guys for being here. We would like to follow that fireside conversation with the honorable and newberger by taking an indepth look at the future of cyber. For our next session, 2020 four and beyond, the hard road ahead, we have invited a distinguished group of experts to look at the topics through multiple listings. Moderating this discussion will be the former cio nasa. Our featured speaker for this conversation on the road ahead are dave frederick, assistant Deputy Director for china, National Security agency. Eric goldstein, executive assistant director for cybersecurity. Retired Lieutenant General steve. Senior executive advisor. Andrew, advisor, technologies, formally coo, cia. And david richardson, vp of product endpoint and security. Look out. Please join me in applause and welcoming our speakers. [applause] wow, good morning, everybody. I hope you have shaken off the malaise of being in a conference all week and talking to people in person. I hope you are using your devices as we are at a cybersecurity conference, thats always a good thing. We connected those devices and inserted the chinese rfid, correct. We got that at the front desk. Good morning, everybody and its great to see you. Given the Rapid Advances in technology in the ever shifting Political Economic and environmental landscape, the future of cyber threat world is sure to change. This group of experts will explore how this future will likely impact cybersecurity and things to think about in order to keep up. We might go back to some of those oldfashioned human problems that will lie ahead in the future as they are here with us today. So, gentlemen, this is an all play, so everybody gets to answer, and hopefully will you will sort yourselves out. If not, i will help you if necessary. Looking beyond cybersecurity, what big shifts are happening in the world that will change how we should be thinking about the Digital World and protected protect it . Who would like to start . I will just run the board. First of all, its a pleasure to be here at the conference, and then with this group of professionals. I think as we look at the way ahead, and im forecasting out a little bit, the things that concern me the most are democratic shifts, particularly in the global south. Lots of opportunity, but lots of challenges there. I think supply chains are increasingly becoming in issue. If you think about restrictions that weve placed on transferring technology to the p. R. C. , there is going to be a titfortat there. I think we have had the opportunity to purchase a lot of raw material from those. I think those are things that are very vital from things moving to electric vehicles to building self, so those supply chains are going to have to be reconstituted and that will require a significant investment as we move forward. And then, i think its really the combination of things we cant predict. Social economic, political, and factors will drive it. We have to really watch it. If you go back to and newbergers comments this morning, or you start with the comments its the key of partnerships. Its foreign partners, interagency partners within the u. S. Government, its commercial partners, its partnership with academia that gives us the year early warning. We never seem to predict these things exactly right, but that is going to be the critical aspect of getting through this. To build on to steves answer, i think one of the things that we should look up thats happening in real time as we are watching a digitally enabled army to take on an analog army, and we are seeing the digital army winning. We dont know how this is going to end and the overwhelming force may win out in the end, but i think theres a lot of lessons here beyond just the battlefield lessons about what a 21stcentury capability can do against it. Clearly a 20th century capability. Beyond us improving and becoming better, i think theres also others who are paying attention to this and see david can take on goliath and maybe win. I think it changes the defense dynamic broadly in the National Security dynamic broadly as people watch this unfold. Its wonderful to be here. Its really just a privilege to be on the stage. If we look at the cybersecurity ecosystem, i think at this point we could say it is defined by a constellation of actors who seek shared goals, share norms in cyberspace, freedom, openness, security, freedom and some entities that seek the inverse, and then theres the middle. Theres organizations and countries that are undecided about what these futures of the internet and the Global Digital comments should look like. So, one of our broad challenges, which applies to cybersecurity, but not only cybersecurity, is how do we in the United States convey that positive, affirmative message of the internet, of message technologies as being an ecosystem that enables growth, enables prosperity, enables freedom of expression, and that cybersecurity is an enabler of the affirmative positive vision, because right now there are too many places around the world, and the general managed just mentioned the issue of supply chain, there are countries attempting to make in roads and entering into commercial agreements, entering into supply chain dominance in a way that undermines our collective global interest in seeking a world where small d Democratic Values are the norm and are enabled by security and resilience at scale, and its up to all of us in this community to convey cybersecurity, not only as an approach to protection, but as a positive vision to advance the values in the future that we all seek. Just to remind folks, he represents the Critical Infrastructure journey, so those were important remarks. Based on the dependency of lets say, electricity and clean water. Anyway, dave. Dave its great to be back. When i was here last year, i was in Cyber Command is the executive director, now Prime Minister a new role as the assistant Deputy Director focused on china. Erica and i did not coordinate on our comments, but i want to build out on key points that he made, but starting with this focus on the p. R. C. , we really assess that the prc and the competition between the prc and the United States and our allies and partners will be the issue for the next generation. And what we are seeing happen today with the p. R. C. Is, they are exporting a digital governance model designed to improve sashes support authoritarianism and support the global inflow. That something our u. S. Partners will have to work very hard at the counter and provide some positive options, especially the global south and other regions. Another area where i think the competition between the new United States would be important as technology standards. China sees technology as the main battlefield based on some of their official statements, and weve got to work very closely in close cooperation with our foreign partners to effectively engage in standards. Those of the base, telecommunication and Cyber Security standards that we are focused on today. But we need to be focused on the emerging issues of Artificial Intelligence and other emerging technologies. Glaxos are excellent points about china where we arent in a trade war with them, and what does that mean for supply chains that we are also accustomed to having on the lessexpensive side, what does that mean and how do you make shifts. We are talking about a huge economic impact, both on the Positive Side and may be on the negative side as well. Many thoughtful remarks, thank you for that. I want to build on what youre talking about around Artificial Intelligence. Obviously there has been a lot of discussion here already around Artificial Intelligence, but that is essentially lowering the skill gap for good actors and bad actors, great, it become so simple these days to generate compelling, automated attacks, whether it be fishing emails, phishing websites, those types of things or sophisticated xplay chains or those kinds of things. Where that used to be something that required highly skilled individual to be put in a very dedicated effort. These days you can buy fish and get it online. You can buy these things online very easily, but its becoming even simpler than that with the rise of Artificial Intelligence. But theres the flip side of it, this is also a tool that could be used for good, that could be used for sorting through massive amounts of data, find the anomalies and things like that. Something that needs to be embraced by organization, as it was mentioned earlier, in attacker only needs to be right ones, and the defender, you need to be right all the time in order to successfully defend your organization. So i think Artificial Intelligence is a big one. The other one, i would say is post quantum encryption. If you get better world whether thats five days away, 40 years away or a month away, who knows these days, there will be a world where encrypted data stores will all be given enough time and enough money event of encrypted data that could be broken. Organizations need to think about where your data lives, who has access to it, even if its encrypted because theres a clock on that. Someday somebody will be able to get access to that data. When we were in the green room preparing here, and that you had mentioned theres a pretty important vulnerability thats out there right now that perhaps our audience may benefit from hearing about. Absolutely. If you havent seen the news late last night, citizen lab send out a vulnerability disclosure around a new exploit chain found in the wild. It affects pretty much every apple device out there. And this is a pretty scary exploit of basically what can happen. Someone can send you, through i message, what whats called a file thats like your boarding pass. Your flight later, or Something Like that. And your phone automatically parses that when it arrives, to generate an image, generate a pretty thumbnail, and the act of parsing that can export your device remotely. So you could receive an item from an unknown number, you dont even have to look at it, you dont have to see it or know that it even happened. Then your device can become compromised remotely and infected with advanced spyware like pegasus. They put out a patch last night for all apple devices so iphones, ipads, imax, etc. , you should get that updated soon as possible. Its very, very important, and these are the modern kinds of threats that exists the stays where your phone can be in your pocket, you can get a text message and its now compromise, and the attacker, first thing they do, delete the text message, delete the notification, you dont even know at happened. Somebody is living in your phone, somebody is watching everything thats happening in there. Thank you so much for that dr. Also remember, a friend is not going to send you tickets to taylor swift. So just keep that straight. And that is, also this is audience participation, make a note, not now, not until were done in 20 minutes and 40 seconds, dont do that update, Pay Attention to these guys. They are talking about beyond now. Exactly. There seems to be an emerging conflict between developing technologies focused on decentralization and traditional comp traditional political and Economic Industries wanting to Leverage Technology for control. How does this play out . Steve, why dont you get us started. Steve this is nothing new, its not emerging. For those who are working or have been on the government side and worked the government side. And now the commercial side, its actually a very interesting viewpoint. And we have to realize, you are working toward the same thing, but the value systems, the value chains may be a little different. If you think about, what i think a lot of people look at is, web three. Oh, or web three blockchain. So, that becomes this popular discussion and i kinda fall on the molly white viewpoint, which is, its a bunch of scammers, a bunch people that are out there, and whats the value proposition, so, thats one example of where you could be, so i think there are other examples for use of ai. Certainly generative ai. Quantum, computing for power or for encryption to protect encryption. In the Technology Come and go, i think it was said very well, in 2018, the issues, the concerns they were facing, they are not the same ones they are facing today. Part of that is a change in technology. So, where im at on this right now, that tension can be useful. I think its not just government, its not just big business and the little guys that are out there, there are a variety of factors well beyond the technology. Sometimes we jump right into the technology. I think its important to look at the other factors. They are social factors. There are economic factors. There are political factors. There are cultural factors. Either discussions we are having in the u. S. Or you might have in the european union, you might have in another place of the world may be very different than the discussion you may have in china or in russia. So, this tension between freedom and oversight, or compliance, or regulation actually, i think, is very valuable. You will have some people that will be pushing the limits, and sometimes they get themselves in trouble very, very quickly. They always either want to look for someone to bail them out, or there is another group that is exploiting gaps in regulation and oversight and compliance. So, theres a role for both the government to be involved in this space. Theres a role on the commercial side, and certainly, theres going to be a very allowed actually, increasingly more political role in this. I think where we have to be at the end of this is clear communication, very Good Partnership between the people who are deploying the technologies. The government who can provide some oversight. I think the other challenge, frankly, for the government, is getting expertise to understand the technologies, understand the effects. You think most recently the deployment of chatgpt and some of the other ai language models just creates this firestorm. Im sure the Big Companies that are deploying the capabilities, they probably did not foresee what that reaction, that visceral reaction was going to be. I know the government is still trying to sort out what is the role in this space. So, the question predicts the outcome, . 10 always going to be there, and thats actually ok, is what im saying. Thank you, steve. We are in the middle of a really interesting and dangerous time in the cyber Threat Landscape. What i characterizes both a deepening and a broadening of cybersecurity. On the broadening, we talked for a while about the democratization of the abilities manifested most tangibly as a Service Ecosystem where you are able to rent the infrastructure without any training and launch attacks on victims of your choosing and access in order to execute your malicious intent. If we combine an ecosystem of that nature with the increasing ubiquity of generative ai, we are further reducing the floor to launch damaging intrusions, we are further democratizing the availability of the abilities to actors who have malicious intent, but no capability. All you need is the intent and probably a little bit of might. Combine that with what we also see as the deepening of sophistication of some of our apex to adversaries and i will call out to this context the advisory that our partners released on our p. R. C. Web techniques, which is extraordinary challenging where we now use traditional malware untraditional control that is detectable by the cybersecurity tools that we all know and love. That wont work anymore for actors using these techniques. And you actually need to understand the activity on your network to such a degree that you can detect anomalies that indicate the adversaries using legitimate tools, Network Management tools used by the administrators, but malicious intent to gain. We see this at an intersection, democratization of capability and advancing in sophistication, what is that mean . In the first instance, a focus on resilience. The fact that our goal is detect, prevent, respond in every context. We are never going to succeed because we are never going to keep every adversary out of every Network Every time. We can do is make the investments today to make sure that when our adversaries gain access, we ideally evict them quickly but limit their ability to cause harm on the American People that it is a cultural shift because it takes us outside of the traditional cybersecurity i. T. Box, and it becomes much more of a issue than a continuity issue and thats much more where we need to be. What we would encourage is, we are preaching to the converted, but thats try to get out into the broader world and speak with the business community, the Resilience Community about how we can make these investments before they are needed. Thank you so much. How about you . Excuse me, for a second. Dave a lot to cover here. I think the part i would reinforce going off his point, one is, in area, a trend that has been very positive is the relate nurse just relationship and partnership between government. There is still a lot of room to get better, but when i kind of reflect back on or when i first came back on the intelligent side and started focusing on cyber and cybersecurity around the 2016 timeframe, it has improved so much. In continuing to work on that, continuing to mind the work of private sector firms in the cybersecurity degree in government support, i think is going to be critical to deal with these longerterm trends. The other pieces, how do we have an effective conversation with Business Leaders about cybersecurity . Theres economic uncertainty, theres always a of folks pulling back, so how can the cybersecurity have conversations about Business Leaders with risk , make sure we arent too much in the geek speak mode and we are trying to make the case and put the foot on the gas in terms of the deeper improvements 10 network hygiene, to resiliency through investments. Recognizing it as a cost center. How do we get through that. I dont live in that world. I dont envy the folks i have to make those cases to boards that are wrestling with the cost. But i do think the work like we are doing with these is helpful in the sense that we try to communicate some of these risks to a broad audience. We are not just keeping it in the family anymore. I see that as a major change across the interagency and with our foreign partners. It has been very positive for the whole community. I think that sounds great. I think i want to remind folks that as we talk about our external threats, you have to think your internal threats and the more complex environments these are, your Insider Threats are just as complex and hard to catch and monitor with some of these things we talked about can be turned inside to find behaviors downloading at 3 00 a. M. Wednesday will when you are on the east coast time. You just say, ok, i think thats a little awkward there on that one. As we move towards Renewable Energy sources, they need to attach more devices to our connected Digital Space open zamora area of attack, more surface area to attack. How do we think better about security at the edge . I will start with you. It is, i think, a truism that securing an enterprise environment is just getting more complex by the day. Whether it is bring your own device, whether its mobility, whether its cloud and hybrid, the complexity organization is really challenging, and for a small organization, is practically impossible. Where we need to focus as a community, how do we help organizations simplify, how do we help organizations prioritize . We have seen really exciting work in the Vulnerability Management space to help them say, if you try to address every vulnerability or misconfiguration on your network, you are not going to succeed if you focused only on cvs s critical vulnerabilities, you will probably reallocate resources. How can we identify the smallest numbers of vulnerabilities to focus on, the smallest numbers of deploy to achieve the broadest benefit. We tell the organization, here are 200 controls and 10,000 vulnerabilities, they are not going to achieve the goal. We are looking at this in a few ways. We can either focus on our cybersecuritys goals as being a list of steps that a small to Medium Organization can take to achieve the most risk production per ties by impact, but also help the organizations focus on what are those vulnerabilities that are actually being exploited in the wild with a likelihood of being reported in the wild. Lets start there, and then build out. Its reasonable to note, that mature organizations have a Vulnerability Management program where they are able to triage and enterprise risk. The vast majority of organizations dont. So how can we make their lives easier, have them relieve them of some of the burden and focus on what matters most. Thats an excellent point of having this management, those closest to it can tie foreigner ability management. Should there be a freeze on your network updates. Since i worked on nasa launches and stuff like that. Having that program, you know also when you can do it, not just if you can do it, but when, because sometimes when is just as important as doing it yourself. We are talking about at the edge. I actually first want to build on that Vulnerability Management point. I was talking about this vulnerability. These kinds of vulnerabilities, when they first come out, the nationstate level, actors know how to exploit them. But months later, anyone knows how to exploit them. So theres a timeline on these vulnerabilities. Give you a concrete example here. Prior wet kit exploit change called trident had been used to infect the devices with the pegasus malware. It was a series of three exploits and web kits to compromise the device from a single click. That same chain is if you look up a Youtube Video for how to jailbreak your nintendo switch, its the same. That exploit change went from nationstate to script kiddie, to literal kids using it to pirate games on their nintendo switch. In the course of nine months. So this is its important that you also think about the timeline on former abilities, and getting those patched in understanding what needs to be done day zero, wet needs to be done day 30, and thats all of these things that are remotely exploitable. They need to get caught up in patched. On the point of edge, coming back to that, security has always been this pendulum that swings to secure the network, secure the edge, centralized, decentralized, swings back and forth, i think the right answer is of course all of the above in defense and all of the good best practices that we all know. These days they talk about this idea of cybersecurity mesh architecture. The idea that these things need to be in communication with one another, and your edge devices need to understand their own stake, but you also need to understand all the egress from your network, and if youre thinking about Renewable Energy in these modern sources, and electric sources are a sickly data centers on wheels. They probably have more in common, its like a tablet that drives her car. They have more in common with an ipad than they do probably a Combustion Engine vehicle that you wouldve bought a decade ago. So, it is a completely different model these days, and you need to think about how do i secure that device from all angles. Just a workforce top tip i heard about the nintendo switch. If the thirdgraders spell fishing with a ph, that is not a spelling error. Ai and Machine Learning sapir to be creating a new environment for such things as over trusting technology, and increased potential for deception and misinformation. What are some of the implications you see as ai begins to reach its full potential . I just want to remind you guys that ai and ml are already on mars in the rovers set her up there right now, we have already put it on another planet, lets talk about earth and how we can really capitalize on ai here. So dave, why dont you get us started. Dave i did want to know i was a little worried once it brought up this exploits that we were going to lose the whole audience, so im glad you asked everybody to hang in there for 27 minutes. Lexa takes about 27 minutes to do the update. Im sure for folks that have attended this conference the whole week, theyve heard a lot about ai. And i will kinda pivot back to my main focus, which is the prc, with both an example, a recent example, and then a look to the future. As one example, in 2022, on a major u. S. Platform, it was able to run, basically a completely fake news entity through a number of ai generated videos called wolf news, which, unsurprisingly was prochina communist party. So i think as an example, if we have an adversary thats able to take advantage of a major platform for some amount of time, that is kind of the tip of the iceberg on where this is going to go. The work i think of the future, on reglet tory frameworks, how do we handle this. How is the u. S. Going to look at setting some standards on dealing with this medium . Certainly innovation and detection of deepfakes and often, content. I think it will open a whole a whole range of challenges. It will enable both more effective maligned information, disinformation operation. Its also going to enable our adversaries to be over propaganda more effectively across multiple anguish barriers. So if you think about chinas efforts to really shape the information environment globally, you certainly would expect many things to take advantage of these strong capabilities to be more effective in their propaganda campaigns. So we will be watching that very closely, i think theres a lot of work ahead in the whole community on really grappling with this challenge of ai, getting the right framework in place, principles based on markets strengthening and follows our values, but gives some boundaries for the technologies today. Just to build on that, when i joined cia, we spent a lot of time and effort trying to understand soviet denial and deception, and they were very good at it. But thats childs play compared to what exists today. So thats going to be an enormous effort. Being able to tell real from fake, which can be cutesy about your favorite celebrity, but when you are making National Security decisions based on information that you have to be able to tell real from fake, now youre in a different ballpark. On the ai prince, just look at analysts, your favorite three letter agency, trying to make sense of the world for a policymaker. What we use computers for for the longest time was to be this and we put a lot of information, shake it around, and it would allow me to look up the stuff i need to see so i could help inform a policymaker. That doesnt work anymore because that is still a mountain of information. So now i have to rely on something that i dont understand, which is true for most people, to draw conclusions and show me the conclusion so that i can figure out how to make sense of the world. And so that puts the onus on someone whos doing this to trying to make sense of the world for a policymaker to understand how that thing works. Because otherwise, do you trust it or not trust it . At the same time you have an adversary whos trying to fool it by inserting information and it cant detect its fake and offering it up to you. So it becomes the spy versus spy game with an everchanging environment in a constant reason to keep turning on ai. Your ai has to get better than the ai thats there to fool you, and as someone is trying to make sense of the world, you will have to Say Something is wrong with this ai. Something is not working right because im getting odd conclusions. First of all, the idea that its going to take over. Its mindnumbing to me that really smart people believe this is going to take over. Its never going to replace a human being. I just dont see that ever happening. But its a very valuable tool and i dont think we could go forward without using it as a tool. It sort of saying, i dont trust this thing called automobile, so i will continue to ride my horse. Its not going to work for very long, so we have to move to this ai enabled world, but i think its going to require almost an Education Campaign for people to understand what it can and cant do, where it could go wrong, how can it go wrong, so it opens up a whole another realm for people to understand as they try to make sense of the world. Excellent, thank you for that. Just to add onto that a little bit, another tactic people need to think about when it comes to something thats publicly available, generative lee ai tool, how are they training this data and is a training on your data or on your organizations data . Do you have the right policies and those things in place and controls in place to make sure you are not accidentally feeding and information that its learning on and continuing to get better on that is actually proprietary information that shouldnt get out there . Thats one factor that you should think about. The other is sort of the inverse of this deepfake thing is, a world where you dont trust anything. Where you can just accuse evidence of being fake, anything that you see is being fake, and whether theres an audio, phone call video view committing a crime, Something Like that, its all fake, everything can be fake at some point, so i think the authenticity verification has got to go on both sides. We are going to need tools to build authentic, verifiable video. Authentic verifiable photo. Authentic and verifiable that this came from a legitimate source, which is going to mean more data more metadata, it will need more Additional Details and other factors that adversaries will also try to fake and fill all of this in to try to get us to a place where we can try to sift out whats real and whats fake. So, im also worried about that flip side where anything can be accused of being fake at the end of the day. Thats an excellent point, we are coming to the end, which i thought went incredibly, rapidly. There were so many nuggets, including a spelling opportunity for you that you provided here, and a psa for your apple product, which you can do upon conclusion of this event. But i want to summarize what are some of the key points in our connected world, and that is, resilience, being resilient is deeper now, and far more broad than it was even probably just a couple of years ago. Partnerships are key, whether its a partnership with a cyber professional, with your mission or your business entities, partnerships with your boards and other in the private sector. Partnership between the private sector in the United States government in terms of keeping this awesome country safe. Cyber is an enabler, its not a prevention. Its your to protect you. It enables the business and enables the business and helps you address reputational risk. You dont want to be that product that corrupts your phone that you live by. Supply chain is far more complex than it was, and as we move towards Renewable Energy, those materials that we need, theres a geopolitical complexity associated with the modern world in the world thats about to come. Ai, great opportunities left, people like me will still have jobs. Thank you, andy. Its still going to stay in the loop here. And that it is on other planets, so we have already done that. We will see what happens there, and if they find water and creatures, we will definitely see what happens there. And it being protected in a post wanted encryption world. That is definitely coming in 2024 and beyond. Remember, threat actors, the big apex hunters are still out there, the big sharks and lions. But the hunters, its a lot easier to get into the game to attack you, your business and your government. With that, remember to update your apple products and have a safe journey. [applause] fantastic panel. Thank you. It is becoming increasingly audience at the Cyber Defenses can be better informed by leveraging those who have taken a more proactive approach to countering the threat. In our next session, how spectrum, Cyber Operations enable each other. We hope to offer uas in which the u. S. Government is working to enable more direct action between the two groups. Moderating this panel is the senior vice president. Joining them on stage for this discussion is timothy, the senior director of cyber offense, defense expert raytheon. The former director for the center for cyber intelligence, cia. And the deputy to the commander Cyber National Mission Force, please join me in applause in welcoming this panel to the stage. [applause] thank you to our panelists for being here, and thanks to those who are still here on friday morning. Nick and i were just talking, i think we are on cspan, so quick shout out to your mom, my husband. So, just a great discussion to have. I wanted to start with the statement, a hypothesis statement thats really going to undergird our discussion today. I will read it, its to prevail in conflict, competition, the whole of our Community Must execute fullspectrum Cyber Operations, faster and more effectively. You can imagine, when we all got together on the phone, one of the first requests was, lets define fullspectrum. A lot of ways that it shouldnt and isnt allowed to in this environment. So the panel has defined fullspectrum to mean the full capabilities and partnerships required to address all the threats across the full spectrum of National Power. So, still a pretty broad definition that we can work with. So, jumping into things, in the National Cyber strategy, we have identified the need for more integration to disrupt the actors. We will use all instruments of National Power. Military, intel, law enforcement. So i wanted to start the first set of questions, really nick and andy to you, just to get your point of view on how we are thinking about these requirements. We think about whats in motion today in the department of defense, in the Intelligence Community, to achieve the vision laid out, where are we, how are we thinking about the requirements. Tim, we would love to hear from an industry point of view as well. Great. First off, thank you very much to the great team here for having us and giving me the opportunity to talk about their great work that the great men and women of the Cyber National does every day. We engage adversaries to defend the nation against Cyber Threats. I think its really about partnerships. When we look at the scope and the scale of the threats we face , in all donate domains, but particularly in cyberspace, you cant do it alone. Take strong partnerships between the department of defense, the Intelligence Community, up in arms of the u. S. Government and with industry. Industry maintains billions of endpoints and networks all around the world and thats where the threat surfaces, and thats where the attacks are for our adversaries. So, how do we Work Together with industry to take what we know and what we are doing in the government, share it with industry, learn from what they are doing, adopt best practices and share Threat Intelligence back and forth so that together we can really confront and defeat these adversaries. I like to use the metaphor on how we executed the war on terror since 9 11, because sometimes we look at cyber just as a mysterious, new environment. In a lot of ways, its not. After 9 11 it took a couple years to get the u. S. Government Community Integrated with sense that was relevant for countering the terrorist threat globally. When it all started, the very Intelligence Community and special ops were kind of doing their own thing. But over time we integrated. I think we are much further along on that then we were 23 years ago on the ct fight. Where the Intel Community, the title x dod community, Cyber Command in particular, are very well integrated. Pillar two, and you made reference to the Cyber Strategies disrupting this mantle. Again, in a very different context, disrupting and dismantling cyber threat is different than disrupting and dismantling a terrorist threat. But a lot of things that came through for National Power apply. Pillar five, Building International partnerships to defeat the threat. Again, something in the context of cia we been doing since 1947, Building International partnerships, but sharing, sometime cia has better access to our international partners, we sure that across the board when it comes to threats with the state department, with the rest of the Intel Community, with dod. Since he obviously has in International Portfolio as well, but not as many folks overseas, we would integrate that entire community. So, you made reference to the private sector, i think thats the one thing thats vastly different than the counterterrorism play. The private sector owns the industry, owns the cyber infrastructure, so we have to integrate that and that is a major difference. I think just to go on what both nick and andy have said, its really a partnership, and the underlying tone of these responses is, we are focused on mission. So the more that industry can understand the missions, we dont do this for sport. We are developing capabilities that will see operational use in conflict, and other uses that we want to make sure that the more we are closely partnered with the government, the better we are able to develop those capabilities and make them relevant for use in operations. We will come back to the capabilities in a minute. First i want to take the conversation we are having and put it through the lens of what we have seen in the conflict in ukraine. Just from your point of view, what has the conflict taught us about the integration of the offense of cyber during wartime . Thats an operational example of the strategy we are talking about in the first question. We collect intelligence along with other evidence in the Intelligence Community that then informs operational and the rest of the government. We cant do that alone. We made reference to the private sector but we cant do it alone, we need our international partners, and weve had some ukrainian officials here in the conference this week and they have done an extraordinary job utilizing all the tools and National Powers that they have in ukraine to defend against the russian onslaught. We have been integral in that. The general, first day of the conference, made reference to utilizing intelligence and sharing that intelligence across the nato partnerships, but also with ukrainians. I think that has been decisive for where we have been since february of 2022. But whats also been decisive is the engagement from American Companies and other International Companies helping the ukrainians and not just since february 2022, but also where the ukrainians learned a great deal of how to protect their Critical Infrastructure, and protect all their government systems. So i think we in the United States and across the world should learn from our ukrainian allies and the enormous capacity they have built since 2017 and prior to that, in the Human Capital that theyve built to defend their own network. So, none of that can be accomplished without the integration that weve had across the community both in the u. S. Government side, the private sector and our international partners. Let me ask you, through the lens of Cyber Command, but how learning has been looking at ukraine and looking up the roles of offensive and defense in cyber during wartime. A couple of things. One, weve had teams on the ground in ukraine in the runup to the russian invasion of the very last days. Working alongside ukrainian partners to defend ukraine and networks against cyber attacks. And to discover threats that we are seeing on ukraine networks, bringing those bad, share those with Network Defenders inside the United States, across the government and across industry to really learn about how we can defend together against these threats, and learn how we can optimize both Government Allied partner systems and privatesector systems to defend against the threats and share information bidirectionally. We found things on ukrainian networks that we shared with industry, industry a rich that data in to defend networks here at home. And then share that information back with ukrainians. The other thing that we learned is that partnerships are built on trust. And you cant surge trust. You have to build trust over time by working together, by working sidebyside in a common goal. So its really an investment in building those partnerships and deploying teams to go forward with allies, and the Amazing Things we can accomplish when we work with our allies backed by industry and backed by the rest of the government. It has been a really big lesson that we have learned. If i could just add on, i frequently find myself explaining what Cyber Command does across the spectrum to some of my colleagues in the Intel Community who are less familiar with that. We they may say, we have an allied partner who needs help defending their network. We need to hire a company to help them do that. I say, we already have that in the u. S. Government. That gets to your point on how we integrate. I honestly think that the hunt for teams are one of the greatest tools we have in our arsenal on the Cyber Defense front, and building that trust and partnership across the globe. Nicholas it is not Training Service that we provide. It is really our operators learning how to Work Together to identify and defeat common threats. Moderator i wanted to move and talk a little bit about do you feel that the Intelligence Community has the right data to truly be interoperable . I heard you say we are making a lot of progress, but are there areas of progress we are still working on . Nicholas i think we do. The problem we have is that there is so much data. The attacks surface is growing every day. Our adversaries are adapting. The Threat Landscape is changing at the speed of cyber. Where we used to hunt for exquisite tools and look for indicators of compromise with highend malware, now we are seeing piercy state p. R. C. States. And that was very hard to unpack because if you do not know what your network looks like, you are never going to see that activity. And you cannot run an Antivirus Program to find it. I think the challenge we have and this is whether it is opportunity, to baseline networks and make sense of the data and normal behavior and some networks and help identify what is truly anomalous. Even if it would to look to a casual observer as something that would look normal. That is the challenge we are confronting now. Andrew we are swimming in threat data and we have two different systems. We have an enterprise system at the classified level that does not connect to the data lakes on some of the Cyber Threats the private sector has. Integrating those is a heavy lift. I think the nsa Security Director has given us a roadmap and over the past five years it has been extraordinary. The Intelligence Community should follow that model on how we can securely integrate those two streams of data. If we do not, we will miss a major portion of the cyber threat picture. Moderator we have been in the full spectrum spanish for a while space for a while. What do you think about the solutions that are being brought to the government five years ago versus now . How should we think about the right way to achieve speed, agility from a solutions standpoint. Maybe we can talk about what the Government Point of view is. Moderator it is really two pieces. We want our offensive and defensive teams to both believe they are at the top of their game. Having those teams Work Together and inform the solutions that each of them are developing is critical as we move forward. The attack is expanding daily so we need to be thinking differently and develop new systems with the nationstate adversary in mind. For industry, to answer the second part of the question, i think we can look at these things from me contractual point of view. We talk about speed and agility. Industry and government go faster to get these capabilities. Government needs to hold industry accountable in terms of the contract in vehicles. We need to get to more of a results based contracting model so that Industries Held accountable to deliver capability that is ready to see the battlefield. We are not doing this for sport. We want these things to see operation and our government partners need them to succeed. Moderator absolutely. Andrew the private sector leaders represented here, i think they understand thoroughly the Intel Community requirements. The talent pool here is extraordinary. We just do not have enough. We need to make more of them. As far as integration, acrosstheboard we are more fully integrated. I do agree, i manage my workforce with results oriented philosophy. I think the private sector has the same philosophy. That makes it easier to integrate the private there into the mission. Because if everybody is in dedicated to the mission then everybody has the same job satisfaction. I think we are there for the most part. Nicholas i think it is all about partnerships. It is about Building Trust through common goals, common objectives. And building habitual relationships. We talked about the cybersecurity collaboration center. Cyber command has a program where we collaborate with Cybersecurity Companies and they exchange of information in an exchange of information where we bring things we learn from our interaction with other government partners. And bounds things in real time off of cybersecurity professionals on a really collaborative basis, where we look at data and go off and use the data to defend our networks. Moderator we have a little bit of time left. Look at full Spectrum Operations , if you could wave a magic wand and create new capability, new partnership, what would that be . If you would be able to prioritize one thing that would help the full spectrum operational success, what would that be . Andrew if i had a magic wand, i would be more fun at parties. Nicholas as we look at cyber actors increasingly live enough the land using native commands to do their exploitation, there is tremendous opportunity there for smarter tools that cannot query across commercial data and classified data. To get those pooled together in a way we can really find these threats together, because it is not just Government Networks being threatened. It is allied networks, partner networks. The other thing is as we increasingly defend forward, our Allies Networks are getting more complicated as well. Our tools need to keep pace with the evolving Threat Landscape and increasingly complex networks. That is about integrating Machine Learning at the front end, so rather than just collect them at the front and and pushing it forward, we can do more advanced processing to find these more advanced techniques at the front end. And initiate Immediate Response action rather than having a delay. I gave you two things. Moderator that is fine. Andrew if i had a magic wand, we would have 10 ats of the human medical Human Capital coming out of schools with the skill set we need. We would higher extraordinary people, computer scientists, electrical engineers, but not just that. People who have an understanding of technology, but may be humanities majors and can write and translate technology into operationally relevant discussions at the director cia level. We have a Phenomenal Group of people, but not enough. I think we have the academic programs to develop, but i do think we need to do a better job in the u. S. Government of marketing the opportunities in the u. S. Government for people with stem degrees. With an nonstem degrees but an interest in cyber technology, because if we do not, we are not going to have enough human beings to deal with this. Timothy i would say our industry has always been shortstaffed. I do not see it changing but we are trying to come up with innovative ways to solve the problem. Get better Training Programs that are more action oriented around exercises and better preparing people for this career field, but also creating an environment where people want to come to work every day and try to solve problems that are not necessarily meant to be solved. We have to create that environment to attract and retain talent. The second piece is leverage the new technology, whether it is automation or ai to better the workforce we have to be more effective. If i had a magic wand, until we can solve the problem, if we could simplify this problem. There are 15 steps or whatever it is, if we could have 14 steps and focus better on the 14, i think we as a collective industry could be more effective. Moderator thank you all. Any Closing Remarks . Nicholas thanks for hosting us. We have to figure out how we can attract and bring out the best in every part of american society. Bring in, get them the training they need, give them the meaningful work they deserve. And figure out how do we let people go and bring them back . Part of being a good partner is learning what your organization looks like from the outside and learning how other Government Agencies work. One of the things were thinking about in Cyber Command is how do we leverage our Alumni Networks and make it easier for people to come back, how do we leverage the tremendous talent that exists in the National Guard and the reserve components. It is really great to see that happen and to see the couple of 1000 young people we have in the Cyber National Mission Force come to work every day. Andrew i would just say thank you again. Having a venue like the billington summit to get our message out, for many years the cia in particular is kind of a black hole. We have been doing this in other venues as well trying to get the message out that the cia should be considered an employer of choice for technologists. We just have to keep getting the message of across. So that we can get a healthy cycle coming out of school directly or in the private sector already who want to spend some time in the National Security arena. We are trying to accelerate security clearance process to enable that. We are not there yet. We are getting there. It is critical or we are not going to get where we need to be. Timothy i would just say thank you. It has been an honor to share the stage with you. The theme that has come out of this panel is the message of partnership, whether it is government to government or industry to government. This problem is not getting any easier. It is going to take all of us working together to solve it. There is a lot of opportunity in front of us. Moderator thank you all for the time and thank you to our audience. [applause] every day, thousands of cyber personnel work to ensure our nation is secure. With our next panel, emerging cyber leaders, we wanted to provide you with the uptick of what this looks like at operational level. Our panelists make decisions every day based on the real world events they are addressing every day. Moderating this panel will be byron love, associate director for Program Management at raytheon. Joining him on stage are lakshmi raman, director of ai, cia. Kenneth chew, unit chief cyber division, fbi. Jay bhalodia, microsoft federal. And Lieutenant Colonel stephen, joint task force commander, Cyber National Mission Force. Please join me in applause in welcoming our speakers to the stage. [applause] good morning, everyone. My name is byron love and i am the Cybersecurity Program director at raytheon. I had been looking forward to this conversation all week. As you have heard, there has been many great speakers here at the billington conference. Cybersecurity looks different from the trenches. This morning we will have the opportunity to hear unique perspective on leadership from law enforcement, the Intelligence Community, the military and from industry. A great way to start is with introductions. I would like for lt. Col. Stephen hudak to begin. Lt. Col. Stephen hudak, United States army, Cyber Workforce officer. Ive been in the army for 16 years now. I have been serving in the Cyber National Mission Force. Masters degree in computer science. Getting to do the job i love in the army. In the Cyber National Mission Force, lead the team up 150 individuals across the army and navy to help defend the nation. Jay bhalodia, managing director for Customer Success at microsoft. I lead a team of amazing technologists that help our federal customers achieve security outcomes with microsoft solutions. I have supported some form of regulated industry throughout my entire career. Focus on the federal government in a lot of different capacities, including operating programs. The mission is what calls to me and it is what i try to instill in my team as well. Really thankful for this panel. Really excited about being on stage with these amazing speakers. Director of ai at the cia. Started at 21 years old. Feels like yesterday. Work my way through is the developer, program manager. I ran Enterprise Data science until i moved into this great job which is about driving ai strategy and strategic implementation across the enterprise. [indiscernible] who did not hear me . [laughter] do i need to do it again . Director of ai at the cia. Started 21 years ago as a developer. Move my way through Program Management, running enterprise i. T. Projects. Enterprise data science to include the entire management of the data science cadre. And finally into my current position as the first director of ai for the agency, which is about driving the ai strategy and strategic implementation. My name is kenneth chew. I am a unit chief in the fbis cyber division. I have been with the fbi for 18 years. I have supported a couple of missions. The counterterrorism division. I moved to support the fbis Cyber Security mission in 2016. I am really happy to be on this panel to talk about things. We have some great experience and talent on this panel today. Lets get started with our first topic, cyber challenges. Cybersecurity environments face many challenges, from Personnel Shortages to tools fatigue. What are the Biggest Challenges you face when it comes to meeting the requirements for your job . Who would like to go first . I will start it off. I think cyber is an inherently challenging domain. What you will probably hear is challenges across the entire Cyber Enterprise are not too dissimilar. When i think about advances in the last decade, it is a rapidly evolving environment, not just for us, but for adversaries as well. At the Cyber National Mission Force, no shortage of people. A lot of people who want to join , want to be a part of the mission and defend the mission. Training those people and retaining them is an opportunity for us. Getting people to stay at, getting them exposed to the mission is something we are able to do and help drive people to stay. Same thing for me. It is a humane challenge. There is a challenge of shortage. When we build our organization, we hire for adding to our culture. We hire people that are passionate about the craft, the industry, customer outcomes. When we have a situation where we have less capacity than we have demand, it creates a tense situation. When you have passionate individuals, we are not going to let the past us. Let capacities stop us. Shortage and capacity was good yesterdays problem is exhaustion. That is a big area. I think recognizing our people, but managers get exhausted too. I agree with that, we are all running at 1000 miles per hour. It is important that we are thinking about the people who work for us and their wellness and how we can support them, but also get the most for our mission. When i think about the ai cyber nexus, where thinking about Data Availability and quality. Working with industry partners, how are we going to integrate any potential solution they might have been to our systems, which are very unique from a security requirements perspective. We also need the right talent. Like everybody in this room, we are all trying to find those Artificial Intelligence practitioners who know how to do this work. And we need to be able to do it with ethics and legal implications in mind. Trying to pull all of that in place takes time. I cannot agree more with everything. Human Resource Management is a real challenge. In the fbi, we are always trying to focus on over the horizon threats. Anticipatory intelligence means trying to do more with the same amount of resources. While also paying attention to our consumers, which includes high levels of government of the private sector and also the public in our messaging. With that in mind, burnout is a real concern. I tried to take into account that if my people want to build their career, i want to help them. The fbi has a wide mission set and i will support them because it tends to keep them on my team longer, building the expertise wider. It seems the cybersecurity challenges are around people and we have heard a lot about the shortage of personnel that have the talent to operate in cyber, opening up paths for individuals to come into cyber. And taking care of them once they are there. The positive work culture of making cybersecurity a place where people want to work. The pace of cybersecurity threats is overwhelming. Threat actors are now using ai. How do you build a positive work culture during stressful times when our Technical Resources are short and you have more need that money to get things done . I am happy to start. In the fbi, we try to focus on the mission. In order to keep my people focused on the mission, they have to feel like they are involved. Last year, i had the opportunity to deploy one of my intelligence analysts to albania in response to a cyber attack. In the world of cyber intelligence, and to explain to them so that anyone could understand it. She had the opportunity to brief National Security council, high levels of u. S. Government, and it is those opportunities that keep people coming back because they are exciting, they are new and they are career building. I also think when you are leaving an organization, the people are looking to you to create that culture. It is important that you were communicating your values and your ethos to them and then you are living it. Not only talking about it, but you are acting it out and they see it. I also think that the people in your Organization Need to feel valued. They need to feel that what they are doing matters and they need to feel rewarded. If they are seeing the fruits of their labor, that starts to build the positive work culture. I will build on that. It is the same thing. It is three things for me. First, it begins with communications. That means listening to what your team has to say. A means communicating transparently. When you do those two things and they see you are acting on what you told them, a builds trust. You are executing on what we told you and you are honest with us. Think about leaders you followed where you felt supported. You could run through a brick wall for that person. Clarity, you mentioned shared purpose. There is clarity for the individual as to what role they play in the mission, but there was also clarity for how that fits into the overall organization. And then recognition. I would love to say i have infinite pockets of money where i can give out rewards, but you also want to recognize for impact. You want to recognize when there is positive impact. On my team, we commissioned a coin and our Leadership Team gave out a coin to a bunch of people on our team. It lets the entire team see what you value. I got a call from one of my team members a month later. They said i want to get one of those coins. There are different ways you can motivate your team through recognition. When i think about building a positive culture, those are the types of things that come to mind, having a unit identity, coins, something that people can rally behind and help them identify their mission set to give more credence to what they are doing. People first, mission always. It is more than just words. We care about the people. As long as you take care of the people in your organization, than the mission will follow. Before we talked about the challenge of overwork. There is also the challenge of under work. I dont hands sometimes can drive a bit i dont hands can sometimes drive a bad culture. I am retired air force myself and we learned Integrity First and excellence in all we do. That is current with me throughout my career because leadership becomes a part of our pattern of life that our adversaries leadership as part of our pattern of life as well and it is observable. I applaud you all for setting those examples for people who follow you. You all are effective leaders which is the next topic. Each of you represent different sectors. As a result, you bring a unique perspective to cybersecurity leadership. What is one positive thing you believe has helped you to become an effective leader. I am going to lead from the front. It is the way i am wired. Whenever i started my career, i wanted to do every phase. When i began my role in security, i wanted to do the jobs. Ed is being able to engage with the team at every step of the process, but not take away from them. Helping guide and direct, set the guardrails for are we still achieving our purpose . That is the thing for me. This is extended into the team culture a little bit. I have this phrase i say. This phrase upset in the seat. Of sat in the seat. Whether it is someone who has performed that capability, it builds residence. We are a Customer Service organization so having the perspective has become a part of our culture. I love what he said about you guide and direct. I think the ability to know how to build the team and bring the right people to your team that can fill your gaps. None of us can do this on our own. So the team you build is so important. To help you fill the right gaps, both from a subject matter expertise perspective and from a culture perspective because you wanted to be a positive place to work. What helps every leader is just having a Current Society curiosity. Being interested in what you are doing is positive so that you yourself are building some acumen and leveraging your team in the spaces you need to. The importance of having team identity. It is important so they understand what their role is in the mission. In that, identify what a winning team looks like. Identifying what type of personnel you need to fill those gaps. So that you can build a team that volunteers for more things and take on career opportunities. I do not think i would have been in this position if i had said no to a lot of opportunities. I said yes to a lot of things i was not prepared for, but it has been beneficial. It is important to bring that spirit to the teambuilding aspect. I have two things i want to talk about. The first positive thing is having had leaders who allowed me to fail, to try Different Things and not be upset when they did not work. They trusted i could get past the initial failure. If we did not fail, we probably were not trying hard enough. I appreciated that as a positive aspect. The other is when i was starting up my career, learning the value of teamwork and what is capable. When i was given jobs, i could easily accomplish those tasks, but i had a leader say the success of the unit is not whether or not you accomplish your tasks, it is whether everybody does. Now being a leader, looking to the junior members and helping them field the team can do better when we Work Together, and having them believe in that concept. The trust that you mentioned, that has be earned. You have to listen to your people and take action on that. You want team leaders that are supporting us to have the same type of values. I like what you said about the delegation part of it. I had a general teach us a group of young officers that if something comes across your desk that is fun, than it is not for you to do. I agree. Everybody needs a chance to shine. They have to have their opportunity space. Absolutely. We talked about the one thing we would do in a positive perspective. What one thing that you did while perhaps not overly positive so helped shape how you make decisions. I think the thing that helps me make decisions are the diversity of experiences i have had. I think it is really important. I tell people who ask me, how do you do what you do . You have to build your toolbox. You have to get that diversity of experience because at a certain point you are making decisions with limited data points. And you are going to be called on to make those decisions. What fills in that data are the experiences you have had. I think it is important to have that diversity of experience to help inform your decisionmaking. For me, absolute clarity on what this one is. When i first moved into management, i really wanted to be successful. I wanted my team to be successful. What i found is that there is a difference between being accountable for your team success and empowering them to be accountable for their own success. It was a hard lesson because it started off with, i was hoping making sure our team achieved the objective and the best way they could. It took a mentor of mine punching me in the gut to tell me you are spending more time helping than they are spending working. That was a big lesson for me. It is rolled into how to lead my team right now. I will set early deadlines because, not to stress them out, but to get ideas on the table early. So that you can be accountable for your own success, not i am fixing something at the end. A lesson for me, i was recruited into the Cyber National Mission Force after winning a hacking competition. When i first showed up, i was a little bit arrogant. I had some Senior Leaders who helped me realize the value of being humble and did it in a positive way. I viewed it on a negative on myself. I was bragging about these things and there are a lot of people like that. The opportunity for mentorship there and the value of remaining humble. Whether it is your people on your team, staying true to what you are actually doing. I think it is so important to understand my role on the team. Understand my role on the team. He talked about being punched in the gut. Listening from different perspectives. Even though it might be hard to hear, it tends to make the team better. It is a natural part of organizations to have conflict. How do we motivate, the next topic. What are the key motivators for you, your team and your people as you strive to serve in your everyday job . I came to this question and i did not really have a premium for it. Frame for a. The analogy i look at is our team is like an orchestra and we are like the conductor. What that means is understanding each individuals unique passion , what makes them tick. Our role is to align their passion to our organizations objective to the mission and the community and the people we serve. That is our role. I love to say this is like the lego movie, but everything is not all some always and there is some general discord. What happens in those situations is there are a couple of people who mention this. Really focusing on that persons passion. Finding a way to get them to move to their passion. Sometimes that means you take the pillar of your organization and move them to Something Else they are passionate about. That is kind of scary. What ive seen when we do that is they get so much energy and motivation, they bring it to the mission and the other people they work with feed off of that. My sixyearold is always watching me. Other people see you are willing to invest in your people and find them an opportunity to do what they love. That is retention. Knowing your people and genuinely knowing what motivates them. It is different for a lot of people. There are people i work with that Cyber Command and just seeing the facilities is motivating on its own. One example, i had a Young Software developer working in a room developing software for the Cyber National Mission Force, and he never got to see it used. Bringing him on the mission to see when it was used motivated that individual. He got to be a part of the team and it got him to stick around a lot longer than he planned to. He got to see firsthand the impacts. People cared about his role in the organization. I could not agree more about personal motivations, understanding your team, figuring out what they like, whether it is travel, getting up in front of a crowd to brief, sitting sidebyside with the mission partner, you can start to map peoples personal motivations to things that will reward the organization. I also feel like i get a weirdly free pass for motivators. The motivation of mission is threaded through every officer that works there. What motivates is the mission. People in National Security feel like they are a part of something bigger than themselves. It is very special. I think giving people the opportunity to solve unique problems is a really powerful motivator. They want to come work for us because we have cool problems. That you cannot talk about. Passion and motivation is key in the fbi too. Getting to know my people, if they want to work on ai, if they want to work more private sector, it all feeds into the mission. Just like in the cia, people want to join the fbi from a very early age. It helps that we have a lot of tv shows that motivate people. It sells itself. Getting them to understand the impact of their work. To see the product that was briefed at the National Security council or briefed by the president. That is really inspiring. We are getting close to the end of time here. I will take 30 seconds and talk about driving production productive change innovation. What does it take to drive productive change and innovation . I touch on it a little bit. Not being afraid to fail and not accepting no for an answer. When you are working something for the first time i might be scary to hire leadership. Settling those fears and talking through it and why do you need to innovate to work the mission. The risktaking. I am hyper passionate about this one. Our adversary is creative. When we find ways to mitigate them, they find ways around that. We need to be innovative to combat that. We need more diversity. It is when we bring many voices together, listen to those voices that we get the best ideas and bet creates innovation. And that creates innovation. I think it is i agree with all of those perspectives. It is a convergence of top down and bottom up. It is the people who are boots on the ground who understand the problems to be able to innovate against those problems. From the top down, we need to give them the space and the ability and the resources to perform that innovation. It is important that those two things converge. Discipline is key. It is always scary to upend the apple card and try a new skill. It is important to understand your team. It is not just my unit or my division. It is pulling together resources so we know what all of our options are. As the general said, at the end of the day, leadership matters. These emerging leaders before you hear matter to the mission of Cyber Defense. I want to thank you for your commitment and contributing to this conference. Please joining join me in giving these leaders a round of applause. [applause]