Taking our stuff. We have a mix of commercial Sector Companies working with the government and a mix of government folks that are focused on Important Missions and critical to our national security. In my opinion and given my experience i think offense always wins. With that mindset i want to look at what we can do today and Lessons Learned and front line knowledge things you can take back to your organization and implement. This gets to the heart of the issue, scaling. We have lolts of systems, we are certainly dealing with a scaled attack surface and dealing with adversaries getting more and more advanced and theyre more and more capable over the past five years. You can see here were focussed on i. T. Modernization, spending 95 billion. We have cyber spending going up close to 10 billion. Is it actually working . And we also have a growing talent gap. With these things, i want to put the panelists on the spot early this morning and kick it off. Everybody in two to five sentences or less, what is one way youre scaling your Security Programs to address the growing threats in light of the Cyber Security talent problem . Okay. Ill go first. Theres two aspects to this. Ill try to be within two to five sentences. The first aspect is how can we better leverage a. I. And ml to growth with the data coming at us. The second is how can i get on board the right Technical Skills that can help me with the mission . Being in government, i cannot match the salaries of industry, so i have to work some unique ways. I have to appeal to their sense of mission. Our mission is interesting. I have to appeal to their patriotism. We also have to look at other means to increase their pay. Were doing cyber retention incentive pay. We can bump their salaries up and we base it upon performance, certifications and the job youre in. Were looking at a new hiring management process, Cyber Talent Management system. Angie bailey, the cchico for dhs is beginning to put that in place. That should help us create more automation in how we hire things. We have a robust Cyber Internship Program which we ran this summer for ten individuals. We kept them away from data entry and gave them cyber kinds of problems. So theres a couple ways you do it. One, you have to look at helping the team deal with the growth in data. You also have to make sure you have a team and face up to the unique problem government has in hiring. Rachael . I would say that its much the same. Right . Were dedicated focus around Artificial Intelligence of course. I think part of that is also i would couple that with automation. What are the things we can do to take some of the tedious tasks off our instant Response Teams . Off of our analysts so they can focus on the bigger problems. I think thats been a big one. The other thing i would say is really trying to better couple our i. T. Modernization efforts with our security efforts. I would say that in both directions. We talk about how security isnt built in from the beginning. The flip side is true. When our security engineers are building something, they arent thinking about User Experience either. Thats not something theyve been tasked to do. So i think coupling those we find we have stronger solutions. Absolutely. I think one of the aspects that we really take a look at is how do we quantify the risk and how do we value the investments . Rather than looking at a particular a. I. Tool or product or specific supply Chain Solution or crowd source penetration model, it was like, how can we better do the hard question of what was the value and what was the Risk Reduction . A lot of our focus right now is lets have those hard conversations to put a number, to put a value on it so i can really look at the thousands of unsolicited emails i get every month from vendors to say which ones are giving us a return on investment. How do i look at replacing an infrastructure that is aging . How do i make sure i have the tools in the cloud necessary for monitoring, and really kind of revalidating our risk discussions from a qualitative approach to a quantitative approach and making better decisions and i think thats our ultimate se ultimate end is to make better decisions regardless of the tool, the technology, the ai, the supply chain were using in the universe and making sure that we know the right questions to ask and we can give reasonable thought out questions on risk rather than it was red and now its less than red. A technical focus and technical career what we had to do was scaling meant reaching out beyond the i. T. Experts and beyond the cyber experts to people that affected their mission and they really werent familiar with what we were doing so we had to devote resources to upscaling them on what the threats were and why this had to be a vital part of getting their mission done, even though they were not the i. T. Team or not the cyber team. The another part s we had to devote ourselves to devote ourselves to the ruthless annihilation of the legacy systems and that patience was such a weakness. It was a security problem that we could not tolerate. We want patience when bringing people on board to understand the importance of cyber and i. T but absolute fullon devotion of annihilation to the legacy systems was key. Good morning. I would focus on two specific areas. One from a People Perspective and trying to source new people in to our business and to provide them back to our customers and source them from the perspective of diversity of skills, so build an seans point. Not just engineers and computer scientists but an analyst and an operator perspective, skill sets to leverage into that part of the business. The second part is leveraging diversity of geographic reach. So we typically focused on major work centers in the d. C. Metropolitan area, for example, where we have clusters of customers but we have the ability to reach out throughout the United States and globally as a provider and we are finding better ways to utilize that talent back into the business. The flipside is application of advanced technologies to take that diverse skill set and allow them to do more than just whatever their primary career field was. If they were an intel analyst, for example, that had nothing to do with cyber, how do you take them and then reapply them in the cyber field to give them the tools to convert their mind, the way they think, into usable capabilities to better the cyber posture defense of an organization. Sounds like a growth and efficiency but how do we measure that effectiveness . Emery, you touched on this a little bit. We spend a lot of money on new tools, systems, people, training. How do you kick the tires and know whether your Security Program is actually working and youre actually getting more resistant to attack . I think for us personally we are really focusing on how do we change our risk methodologies to capture that value literally in dollars and cents in terms of activity. So for example crowd source pent testing. What is the value of those what did we defer by finding the critical and high vulnerabilities that could be exploited on the internet . How can we bring that to a value statement versus spending another 100 million or 1 million on the next new tool versus bringing subject Matter Experts to the table. We can have those conversations. The methodologies are there. We are very cautious because people worry that you have to have a perfect answer. For valuing the new tool or the new technology, and i think what people forget is we are really making the decisions. We are making the decisions today, theyre not always the most informed decision today but we can make a slightly better decision today and well make a slightly better decision the next day. So if we are at least trying to, you know, have a meaningful conversation, change the way we are talking about risk and valuing these tools and options, you know, another example would be, you know, how are we valuing the return on investment for authorizing official training . Like, there are theres a long history of measuring education outcomes. People in cyber, you know, suffer from the same problem that we other i. T. Do. I. T. Is like, we dont need to involve security. Well do it at the end. We suffer the same thing. We decide to have authorizing official or enduser training. Do we bring in the educators to come up with learning outcomes an objectives and how to measure it . Theres a long history of that. So we suffer the same thing and it is really just thinking outside the box and saying, you know, we shouldnt fall into the same trap others do. When theyre talking about cyber. We need to bring the partners in now, so when i train and authorizing official, we can actually measure did they learn it . Could they apply it . And activities like that. But things are much more dynamic now, too. Something you said i want to highlight is continuous. Right . It has to be continuous. I know john at dhs, we have the cdm program and theyll talk about that later in a panel this morning, but how rachel, i think you would be great to bring into this, too. How are we leveraging cdm to do Continuous Monitoring and evolution of our risk analysis and what datas feeding that . There is no equation for risk. I wish there was came out that dollars and cents and gave you a very clean answer. Some day. Im going to wrap cdm into my answer. We have spent a lot of time over the last two years looking at our socs. We have 17 socs in dhs, security operation centers. We started a long road here, crawl, walk, run strategy and were beginning to get into our walk phase, and this involves three aspects looking at the tools or cdm. Looking at policies and procedures and then looking at the contract aspect. On the contract aspect, i gave it to the secret services to work out how we would move to a single contract for manning of our socs. Knock on wood, well have an rfp eventually out this fall. Moves on to policies and procedures, the iciso, they have the job to figure out the dod csc pmanual, squeeze out the dod centric things and push in the dhs things. The objective was to develop a program where we can inspect our socs and bring them up to a standard, in other words raise the bar, and we did our first one this june. The Chandler Isoc was passed. They got a threeyear sign. Thats a big deal. Well do tsa probably in december, january. We are raising the bar. To get to the last piece on tools, the cbp had the lead on tools and the plan is to leverage as much cdm as we can to bring into the socs and the idea is not the same tools in throughout all of dhs because some started on different paths. The real question is, how do we integrate things and roll it up to a dashboard to give us the insight of whats happening. Further downstream we look at how do we consolidate . My objective is to take the enterprise soc and move it into the cbp soc and looking at other alternatives in how we could shrink our footprint. As a major provider of cdm services we internally, internal to bruce allen, we try to drink our own champagne as best we can, so leverageing the same models so understand what our highvalue assets are and what the risk posture looks like on a continuous basis. The piece i would add to that on top of some of the metrics that emery mentioned is also looking at the success of simulated intrusions on a regular basis. So when youre looking at how am i measuring the success of the things implemented, im continuously monitoring. Im looking at trying to make best guesses around the metrics of risk. We should be evaluating whats the success of something coming in and making regular simulations and exercises a part of that strategy. Right. Chris, you know, when you have this soc training going on, how do you recommend we emulate the adversary . So the best approach is to train as you fight. Fight as you train. Emulating from a threat emulation perspective it needs to be laid into the soc operations. Not an external outboard capability, so integrating the types of training exercises, in situ, similar to other military command and control systems. If you want to teach someone how to defend against a missile, you simulate the missiles in the system. You dont pull them out of the system and put them in a closed room and a synthetic environment and teach them how to defend themselves against missiles. So laying that training in and then emulation is correct. And sean, you know, at cia i imagine you guys are like nsa putting a lot of systems into denied areas. How do you simulate that type of environment thats super malicious and make sure that your systems are remaining secure and such a threatening environment . Well, one tremendous advantage in the new organization four years ago was that we had the teams that conduct the Cyber Mission and are monitoring and doing Cyber Intelligence reporting in the same team with the folks who are providing these remote systems and so the first thing is to build an integrated team and really spend a lot of time on the way they Exchange Data and we had red team and blue team but got to be purple teaming. We measured how fast to go in identifying for any potential weakness in a very wide sea of systems, what it took to identify where it was and how we could get to it remotely and so it changed what happened before we went to the field and it changed also what happened in the field and we deployed more of these folks overseas, and very, very quickly the demand was send more, send more. So a big part of it was people and giving them the tools to integrate the data together. One thing thats not there yet is, despite all the progress on the monitoring tools, on the back end, really having advanced analyt analytics, and it was mentioned before, with an ai component, that gives you more of a time advantage is an investment that we are trying to make and trying to find the best vendors for. So on that ai, chris, how are you guys fusing ai in your solutions that youre delivering for the government . Our approach for Artificial Intelligence is close to seans need, which is around the concept of providing decision aids. So if youre trying to make a decision about an action to take or trying to inform the development of a playbook, for example, if youre an instant handler and you want to automate a process, you need that Artificial Intelligence running underneath in the background to help prompt those users and or even upscale them from people who are not cyber experts or cyber ninjas, so you can draw upon those skill bases. We do a lot of human in the loop ai, where you reach a point where you need a human decision, and in plugging that human in, using that to feed supervialsed Machine Learning models can help evolve things. John, are you building that into your soc plan in these agencies . We are but more broadly looking at ai and how to use different parts of it across the dhs organization. We have appointed recently an interim chief data officer, brian tequa who works for me. And the taesing is to develop a federated data strategy, in other words, how we implement some of this stuff and it will touch the cyber side of the house and figure out what does a cdo office lock like in dhs . What are the authorities and what are your resource . We are working through the practical things of it as he had his first cdo Council Meeting which is really important when you start bringing together these different pieces. It is important and we spend time thinking about how we can bring better tools into the process that leverage the capabilities that ai can bring to it. You raise a very good point about the human in the loop. Some point the human has to come in the loop because right now data is overwhelming the human. Of course, of course. Were not able to fully automate everything. We cant ever take the human out of the loop but one other thing to talk about is comply chain security. This seems one of the most daunting task, especially in military or in Defense Systems with systems of systems. Think of the number of things in a fighter jet, for example. Emery, whats your take . Is that a solvable problem . Are we grasping at straws here . I cant speak to fighter jets but i can talk about energy sector. From that perspective, i think there are answers that are evolving. For example we just issued a supply chain as a Service Contract for the entire department to be looking at a lot of those vendors who are providing unique technologies in the energy sector, from power distribution, things like that. And there are solutions that are out there now to look and aggregate that information, look at the results in terms of something thats meaningful to the mission delivery. Whats the impact of reliability . Whats the impact of these other capabilities, so i think there is stuff out there. I dont think its fully fleshed out. I think if you start looking at supply chain as only testing products, you will never catch up and youll never get done. But you can look at the reliability of the vendors, manufacturing processes, the risks that they introduce into the process, their history and security over time to help influence and at least get a better understanding of what the risk posture is. Even though there might not be a definitive answer that this person is safe and that person is not. Right. It would be nice if theres a definitive answer to the problem because there is and its not possible for an organization to literally boil the ocean and get to that answer. It really comes back to Risk Management at the end of the day and understanding your supply chain, the vendors, what you put into your contracts, all those pieces that come to play so you can at least begin to comprehend what youre dealing with. Vendors play a significant role, right, and chris, rebecca, whats your take . How are you making sure you deliver cyber hardened systems to your federal customers . My perspective is in a world where you try to strive for perfection for everything, everything becomes equally as important, and thats a dangerous place to be because when all things are equally important, you really dont understand whats important. So from my perspective, it goes back to engineering 101. Engineering basics and we have lost a lot of that along the way in this agile world of just trying to shut out as much capability as we can with commercial technology. Its in the docker container. Yeah, dont worry about it, thats explodable. And so, we lose that. Right . You lose that essence of what pieces and parts of the system are more important than others and then where do i apply varying leveling of security and Risk Management against the components. I dont know that i could say it any better, is how do we make it scalable . It is identifying the things, the crown jewels we need to care 100 about, and the things to accept more risk with and then i think as part of the supply chain our responsibility to look at the own supply chain and as rapid way as possible and offer up the solutions identified for leverage outsource services, to do evaluations and then bring that back to government. A major part is typically cloud migration, too. A lot of people are moving to data center or moving to hybrid cloud. Rebecca, what is your take . Whats a Lesson Learned on a secure cloud migration . Cloud migration . Yes, we went cloud first some time ago now, and we were early adopters of office 365 and a number of other claude Saas Applications in addition to building a lot of stuff in aws and azure. I would say in the same way we talk of earlier of security think about the experience of the user or the developer, in addition to the developer thinking about building in security, cloud became the same thing. Right . A lot of early mistakes lifting and shifting what you had in on Prem Data Centers to the cloud and that didnt work. We needed to be building based on native cloud ability to achieve the security benefits. So thats a big part of it. The other part of it was around governance. Folks wanted to be the the ability to be autonomous and very rapidly build in the cloud. And so we needed to put measures in very quickly to say when you stand up an account, we now will automatically require these sets of things. From web application firewall to whatever it might be and well automate putting those in place and then give you access to that account but we had to take some time to dedicate around how do with build that, so that folks have the flexibility and the speed they needed to build what they build for clients. Super important, that governance. Sean, you mentioned ruthlessly shutting down legacy systems but i imagine a lot of those things are in production and used every day and so how do you do this migration while keeping the Mission Going . It is interesting, though. When you look at how long some have been around and when the claims are how much theyre used you find that its like the stuff you have stored in the garage thats been there a really long time and a last job you want to do to clean it out. Those things are dated to the point where the vulnerabilities cant be addressed. Last time i was there we were using lotus notes or something. Right . Too soon. Sorry. You know, just but we have the problem of we have to overcome the function fixidents. This is the way i use it or someone might be upset. We have to wait until the workforce changes enough and that usually means enough people in this area retire or leave, and then well change it over. These excuses are fully unacceptable. Running hardware where it is no longer maintainable, it needs to be put down, and what we found and what we will find is when you shut things off, those calls that were going to come, very few of them come, and the ones that do come, you often have a better solution and they say i dont want to be in the i. T. Business. Im using this capability. What you gave me is better reliable. By the way, doesnt cost as much . So i just i cant say it enough. I have no sympathy for the excuses to stay on legacy. I just havent found one that was justified. The other example is, if you have a 16yearold driver, the only thing was to give him the clunker car, right, the beat up car, oldest one. Buy him the newest car. The price point on a cheap relatively entry level car, the safety features on that car so far outweigh that legacy vehicle, and thats the education we have to do with the folks who are clinging to legacy. Yeah. Sometimes you dont have the work force for that, right . How do you develop that workforce that knows the latest cutting edge . So, john, what are the Workforce Development programs you have launched . How did you know what i wanted to talk about . Getting there. But the other panelists have really echoed the right things from rebecca talking about automation and all that but heres the key thing. I have an existing workforce. Im not going to be able to change them out rapidly. How do i train them . We have done and we have established sort of a Cloud Steering Group within the organization, among my direct reports, to develop how we go forward. We call it the cloud cyber cloud center of excellence, excuse me. And what we have done over the last couple of months is set up training days, a sort of one on one kind of course. We allow the folks to telework, to spend their day training, an their day training. We had about a 97 accomplishment rate on that. That phase, i thought it was very important for folks who work in my Budget Office to begin to have an understanding of what cloud is and understand the terminology. The next phase of this, were going to start looking at a smaller group. The focus there will be more on the security aspects. Again, this is about raising the bar on knowledge. And the last phase is when we look at the smaller cadre of folks. The dhs strategy on cloud is hybrid and multi. We are agnostic as to what Cloud Service providing we go to. The components within dhs had already started their move. For us to say dont go there, go here, is a not starter. How we put in the right security into the process. So were looking across everything. How do we build the capability within the workforce . How do i build the capability within my network to manage this. Theres no doubt the Workforce Needs to grow. Where do you see the future of that security Talent Pipeline and how do you ramp that up . We started a journey several years ago at the Fouryear College level, a particular program we have at the university of maryland. That was built around multidisciplinary intake. Our first objective was we were not recruiting the top 12 of the classes in this particular career field. So we needed a way to go after that so we did that. The secondary is the twoyear colleges and vocational trades and creating apprenticeship programs around those students to bring them into the organization. Not everybody need a Computer Science degree to perform a cyber role. And then the third area is the academy construct of the organization. Cspan 3 is back live at the 10th annual billington Cyber Security summit. Up next well hear from chief Information Security offices from the justice department, Homeland Security and the export Homeland Security and the export im so i used to be the cio at the department of interior