Taking our stuff. We have a mix of commercial Sector Companies working with the government and a mix of government folks that are focused on Important Missions and critical to our national security. In my opinion and given my experience i think offense always wins. With that mindset i want to look at what we can do today and Lessons Learned and front line knowledge things you can take back to your organization and implement. This gets to the heart of the issue, scaling. We have lolts of systems, we are certainly dealing with a scaled attack surface and dealing with adversaries getting more and more advanced and theyre more and more capable over the past five years. You can see here were focussed on i. T. Modernization, spending 95 billion. We have cyber spending going up close to 10 billion. Is it actually working . And we also have a growing talent gap. With these things, i want to put the panelists on the spot early this morning and kick it off. Everybody in two to five sentences or less, what is one way youre scaling your Security Programs to address the growing threats in light of the Cyber Security talent problem . Okay. Ill go first. Theres two aspects to this. Ill try to be within two to five sentences. The first aspect is how can we better leverage a. I. And ml to growth with the data coming at us. The second is how can i get on board the right Technical Skills that can help me with the mission . Being in government, i cannot match the salaries of industry, so i have to work some unique ways. I have to appeal to their sense of mission. Our mission is interesting. I have to appeal to their patriotism. We also have to look at other means to increase their pay. Were doing cyber retention incentive pay. We can bump their salaries up and we base it upon performance, certifications and the job youre in. Were looking at a new hiring management process, Cyber Talent Management system. Angie bailey, the cchico for dhs is beginning to put that in place. That should help us create more automation in how we hire things. We have a robust Cyber Internship Program which we ran this summer for ten individuals. We kept them away from data entry and gave them cyber kinds of problems. So theres a couple ways you do it. One, you have to look at helping the team deal with the growth in data. You also have to make sure you have a team and face up to the unique problem government has in hiring. Rachael . I would say that its much the same. Right . Were dedicated focus around Artificial Intelligence of course. I think part of that is also i would couple that with automation. What are the things we can do to take some of the tedious tasks off our instant Response Teams . Off of our analysts so they can focus on the bigger problems. I think thats been a big one. The other thing i would say is really trying to better couple our i. T. Modernization efforts with our security efforts. I would say that in both directions. We talk about how security isnt built in from the beginning. The flip side is true. When our security engineers are building something, they arent thinking about User Experience either. Thats not something theyve been tasked to do. So i think coupling those we find we have stronger solutions. Absolutely. I think one of the aspects that we really take a look at is how do we quantify the risk and how do we value the investments . Rather than looking at a particular a. I. Tool or product or specific supply Chain Solution or crowd source penetration model, it was like, how can we better do the hard question of what was the value and what was the Risk Reduction . A lot of our focus right now is lets have those hard conversations to put a number, to put a value on it so i can really look at the thousands of unsolicited emails i get every month from vendors to say which ones are giving us a return on investment. How do i look at replacing an infrastructure that is aging . How do i make sure i have the tools in the cloud necessary for monitoring, and really kind of revalidating our risk discussions from a qualitative approach to a quantitative approach and making better decisions and i think thats our ultimate se ultimate end is to make better decisions regardless of the tool, the technology, the ai, the supply chain were using in the universe and making sure that we know the right questions to ask and we can give reasonable thought out questions on risk rather than it was red and now its less than red. A technical focus and technical career what we had to do was scaling meant reaching out beyond the i. T. Experts and beyond the cyber experts to people that affected their mission and they really werent familiar with what we were doing so we had to devote resources to upscaling them on what the threats were and why this had to be a vital part of getting their mission done, even though they were not the i. T. Team or not the cyber team. The another part s we had to devote ourselves to devote ourselves to the ruthless annihilation of the legacy systems and that patience was such a weakness. It was a security problem that we could not tolerate. We want patience when bringing people on board to understand the importance of cyber and i. T but absolute fullon devotion of annihilation to the legacy systems was key. Good morning. I would focus on two specific areas. One from a People Perspective and trying to source new people in to our business and to provide them back to our customers and source them from the perspective of diversity of skills, so build an seans point. Not just engineers and computer scientists but an analyst and an operator perspective, skill sets to leverage into that part of the business. The second part is leveraging diversity of geographic reach. So we typically focused on major work centers in the d. C. Metropolitan area, for example, where we have clusters of customers but we have the ability to reach out throughout the United States and globally as a provider and we are finding better ways to utilize that talent back into the business. The flipside is application of advanced technologies to take that diverse skill set and allow them to do more than just whatever their primary career field was. If they were an intel analyst, for example, that had nothing to do with cyber, how do you take them and then reapply them in the cyber field to give them the tools to convert their mind, the way they think, into usable capabilities to better the cyber posture defense of an organization. Sounds like a growth and efficiency but how do we measure that effectiveness . Emery, you touched on this a little bit. We spend a lot of money on new tools, systems, people, training. How do you kick the tires and know whether your Security Program is actually working and youre actually getting more resistant to attack . I think for us personally we are really focusing on how do we change our risk methodologies to capture that value literally in dollars and cents in terms of activity. So for example crowd source pent testing. What is the value of those what did we defer by finding the critical and high vulnerabilities that could be exploited on the internet . How can we bring that to a value statement versus spending another 100 million or 1 million on the next new tool versus bringing subject Matter Experts to the table. We can have those conversations. The methodologies are there. We are very cautious because people worry that you have to have a perfect answer. For valuing the new tool or the new technology, and i think what people forget is we are really making the decisions. We are making the decisions today, theyre not always the most informed decision today but we can make a slightly better decision today and well make a slightly better decision the next day. So if we are at least trying to, you know, have a meaningful conversation, change the way we are talking about risk and valuing these tools and options, you know, another example would be, you know, how are we valuing the return on investment for authorizing official training . Like, there are theres a long history of measuring education outcomes. People in cyber, you know, suffer from the same problem that we other i. T. Do. I. T. Is like, we dont need to involve security. Well do it at the end. We suffer the same thing. We decide to have authorizing official or enduser training. Do we bring in the educators to come up with learning outcomes an objectives and how to measure it . Theres a long history of that. So we suffer the same thing and it is really just thinking outside the box and saying, you know, we shouldnt fall into the same trap others do. When theyre talking about cyber. We need to bring the partners in now, so when i train and authorizing official, we can actually measure did they learn it . Could they apply it . And activities like that. But things are much more dynamic now, too. Something you said i want to highlight is continuous. Right . It has to be continuous. I know john at dhs, we have the cdm program and theyll talk about that later in a panel this morning, but how rachel, i think you would be great to bring into this, too. How are we leveraging cdm to do Continuous Monitoring and evolution of our risk analysis and what datas feeding that . There is no equation for risk. I wish there was came out that dollars and cents and gave you a very clean answer. Some day. Im going to wrap cdm into my answer. We have spent a lot of time over the last two years looking at our socs. We have 17 socs in dhs, security operation centers. We started a long road here, crawl, walk, run strategy and were beginning to get into our walk phase, and this involves three aspects looking at the tools or cdm. Looking at policies and procedures and then looking at the contract aspect. On the contract aspect, i gave it to the secret services to work out how we would move to a single contract for manning of our socs. Knock on wood, well have an rfp eventually out this fall. Moves on to policies and procedures, the iciso, they have the job to figure out the dod csc pmanual, squeeze out the dod centric things and push in the dhs things. The objective was to develop a program where we can inspect our socs and bring them up to a standard, in other words raise the bar, and we did our first one this june. The Chandler Isoc was passed. They got a threeyear sign. Thats a big deal. Well do tsa probably in december, january. We are raising the bar. To get to the last piece on tools, the cbp had the lead on tools and the plan is to leverage as much cdm as we can to bring into the socs and the idea is not the same tools in throughout all of dhs because some started on different paths. The real question is, how do we integrate things and roll it up to a dashboard to give us the insight of whats happening. Further downstream we look at how do we consolidate . My objective is to take the enterprise soc and move it into the cbp soc and looking at other alternatives in how we could shrink our footprint. As a major provider of cdm services we internally, internal to bruce allen, we try to drink our own champagne as best we can, so leverageing the same models so understand what our highvalue assets are and what the risk posture looks like on a continuous basis. The piece i would add to that on top of some of the metrics that emery mentioned is also looking at the success of simulated intrusions on a regular basis. So when youre looking at how am i measuring the success of the things implemented, im continuously monitoring. Im looking at trying to make best guesses around the metrics of risk. We should be evaluating whats the success of something coming in and making regular simulations and exercises a part of that strategy. Right. Chris, you know, when you have this soc training going on, how do you recommend we emulate the adversary . So the best approach is to train as you fight. Fight as you train. Emulating from a threat emulation perspective it needs to be laid into the soc operations. Not an external outboard capability, so integrating the types of training exercises, in situ, similar to other military command and control systems. If you want to teach someone how to defend against a missile, you simulate the missiles in the system. You dont pull them out of the system and put them in a closed room and a synthetic environment and teach them how to defend themselves against missiles. So laying that training in and then emulation is correct. And sean, you know, at cia i imagine you guys are like nsa putting a lot of systems into denied areas. How do you simulate that type of environment thats super malicious and make sure that your systems are remaining secure and such a threatening environment . Well, one tremendous advantage in the new organization four years ago was that we had the teams that conduct the Cyber Mission and are monitoring and doing Cyber Intelligence reporting in the same team with the folks who are providing these remote systems and so the first thing is to build an integrated team and really spend a lot of time on the way they Exchange Data and we had red team and blue team but got to be purple teaming. We measured how fast to go in identifying for any potential weakness in a very wide sea of systems, what it took to identify where it was and how we could get to it remotely and so it changed what happened before we went to the field and it changed also what happened in the field and we deployed more of these folks overseas, and very, very quickly the demand was send more, send more. So a big part of it was people and giving them the tools to integrate the data together. One thing thats not there yet is, despite all the progress on the monitoring tools, on the back end, really having advanced analyt analytics, and it was mentioned before, with an ai component, that gives you more of a time advantage is an investment that we are trying to make and trying to find the best vendors for. So on that ai, chris, how are you guys fusing ai in your solutions that youre delivering for the government . Our approach for Artificial Intelligence is close to seans need, which is around the concept of providing decision aids. So if youre trying to make a decision about an action to take or trying to inform the development of a playbook, for example, if youre an instant handler and you want to automate a process, you need that Artificial Intelligence running underneath in the background to help prompt those users and or even upscale them from people who are not cyber experts or cyber ninjas, so you can draw upon those skill bases. We do a lot of human in the loop ai, where you reach a point where you need a human decision, and in plugging that human in, using that to feed supervialsed Machine Learning models can help evolve things. John, are you building that into your soc plan in these agencies . We are but more broadly looking at ai and how to use different parts of it across the dhs organization. We have appointed recently an interim chief data officer, brian tequa who works for me. And the taesing is to develop a federated data strategy, in other words, how we implement some of this stuff and it will touch the cyber side of the house and figure out what does a cdo office lock like in dhs . What are the authorities and what are your resource . We are working through the practical things of it as he had his first cdo Council Meeting which is really important when you start bringing together these different pieces. It is important and we spend time thinking about how we can bring better tools into the process that leverage the capabilities that ai can bring to it. You raise a very good point about the human in the loop. Some point the human has to come in the loop because right now data is overwhelming the human. Of course, of course. Were not able to fully automate everything. We cant ever take the human out of the loop but one other thing to talk about is comply chain security. This seems one of the most daunting task, especially in military or in Defense Systems with systems of systems. Think of the number of things in a fighter jet, for example. Emery, whats your take . Is that a solvable problem . Are we grasping at straws here . I cant speak to fighter jets but i can talk about energy sector. From that perspective, i think there are answers that are evolving. For example we just issued a supply chain as a Service Contract for the entire department to be looking at a lot of those vendors who are providing unique technologies in the energy sector, from power distribution, things like that. And there are solutions that are out there now to look and aggregate that information, look at the results in terms of something thats meaningful to the mission delivery. Whats the impact of reliability . Whats the impact of these other capabilities, so i think there is stuff out there. I dont think its fully fleshed out. I think if you start looking at supply chain as only testing products, you will never catch up and youll never get done. But you can look at the reliability of the vendors, manufacturing processes, the risks that they introduce into the process, their history and security over time to help influence and at least get a better understanding of what the risk posture is. Even though there might not be a definitive answer that this person is safe and that person is not. Right. It would be nice if theres a definitive answer to the problem because there is and its not possible for an organization to literally boil the ocean and get to that answer. It really comes back to Risk Management at the end of the day and understanding your supply chain, the vendors, what you put into your contracts, all those pieces that come to play so you can at least begin to comprehend what youre dealing with. Vendors play a significant role, right, and chris, rebecca, whats your take . How are you making sure you deliver cyber hardened systems to your federal customers . My perspective is in a world where you try to strive for perfection for everything, everything becomes equally as important, and thats a dangerous place to be because when all things are equally important, you really dont und