Schools. Not all of them. But thats just an example that came to mind of supporting the contexts that you want to put into place. All right, thank you both very much. Lets give them another big round of applause. [ applause ] be part of cspans campaign 2014 coverage. Follow us on twitter and like us on facebook. To get debate schedules, video clips of key moments, debate previews from our politics team. Cspan is bringing you over 100 senate, house and governor debates and you can instantly share your reactions to what the candidates are saying. The battle for control of congress. Stay in touch and engage by following us on twitter at cspan and liking us on facebook at facebook. Com cspan. Our special presentation of the National Bullying prevention summit continues. This next panel looks at specific examples of successful antibullying programs in local schools including efforts to support gay, lesbian and transgender students. This annual summit is part of an effort by the federal government to create a National Bullying prevention strategy. This is about 40 minutes. Since im between you and lunch, were going to try to move as fast as we can, but wanting to make sure that you have as much information as possible. This is a very important session where were actually going to talk a little bit about some of the state education and local education perspectives. One from dr. Bradshaw who has been working with maryland and with a lot of what theyre doing in the same areas that youve heard about the social Emotional Learning and also a little bit of the multitier network, framework, rather. And also from all the way from los angeles, sarah train is here. And im just going to read briefly her bio. You did hear a little bit about dr. Bradshaw earlier, who participated in an earlier plenary session. But i do want to introduce you also to sara train. And im not going to two over everything in her bio, which you all will have. Sarah is the coordinator of the Los Angeles Lgbt centers project spin, Suicide Prevention intervention now program. A coalition strengthening collaboration among many programs and Services Designed to support lgbtq youth throughout los angeles county. Under her leadership the coalition has directly reached over a hundred student, parents and school staff and administrators with trainings on Suicide Prevention and crisis and care to create inclusive schools. She assists the schools on building pass to respond and implement inclusive policies in accordance with changing legislation. So we have some great speakers today. And again, we want to what weve asked them to address is some of the Lessons Learned for implementing bullying prevention programs in districts and in the classroom and some of the promising things that theyre seeing that youth are responding to. We feel that this perspective kind of goes along with our theme of implementation and being able to learn in what is actually going on in the field. So well have dr. Bradshaw speak first and then followed by sara. Thank you. [ applause ] well, its exciting to be back up here. Im going to kind of shift gears a little bit. Because as a researcher and part of a partnership that is based in the state of maryland and is really focused on scaling up positive behavior support and multitier Systems Support and its largely led by our partners at the department of education, alexander and mike ford were first invited to be on this panel. But unfortunately they had an obligation back at the state department so they were unable to attend today and asked me if i could come. The third leg of our triangle is shepherd prath system which is a Mental Health provider across the state of maryland. While they provide a range of support services, anything for the largest support services for kids with autism across the state and in treatment and out treatment, Mental Health services they have a strong commitment to prevention of behab yourle a and Mental Health problems. The partnership actually began in 1989 through shepherd prath and the Maryland State Department of education and johns hopkins, my colleague and i joined around 2001. So weve been working in partnership to do research as well as scale up positive behavior support and use it generally as a framework and not necessarily as a program, really a framework for delivering other evidencebased models. So were building tremendous work that renee talked about from the national t. A. Center which is led by george sigai and rob horner and tim lewis, so we always feel like were standing on the shoulders of giants just peeking over the corner here trying to see what we can do next and how we can actually scale this work up. So much of this effort really comes out of a grant that we received out of the shop that is focused on safe and Supportive Schools and the comment that came up about transformation grant and is pbis really the panacea for that, i dont think that thats necessarily what was the intent but its certainly one tool that can be used and one tool that weve been using to try to address School Climate. So you can think about it as a potential tool or a framework. Thats how weve been interpreting it and how weve been using it. Im sure there are other ways we can think about that. But when the safe and Supportive Schools grant mechanism came down from the u. S. Department of education providing an opportunity to do research about evidencebased practices, frankly, it wasnt a Research Grant but we kind of embedded that in it to promote School Climate. Our Partnership Came together and said, lets focus on this particular initiative. It was geared high schools. So i want to zoom in on this particular project because its something were just wrapping it now. Its also a High School Model where we embedded a randomizedp framework in the setting. Whereas our partnership had done two trial trials to support the Elementary School level. One it just that tier one and the second trial it tier two plus tier one and we have published our findings based on. That not going to talk about those today. Ill be happy to share if you have questions. I want to share Lessons Learned from doing this work at the high school level. The high schools are a whole different ball of wax. You hear a lot about the frame works and can you get the incentives and systems in Elementary Schools but what about high schools . We really found the School Climate framework is one that helped us connect positive behavior support around different kinds of change that we wanted to effect in the schools to reduce bullying and to reduce behavior problems and Substance Abuse problems. So its really just a framework to guide the kind of effort were doing. We built a lot of some of the literature around School Climate and i work with the National Education association and we have partners here from the nea today. Theyve been doing a lot of work around bullying prevention and School Climate. They have a Research Briefs here. This is an xeexcerpt of the research. Whether we get to measurement of School Climate, there is a big piece of the safe and supported schools is to develop a sustainable system for a a. Sesise for assessing the climate and to zrif the Decision Making and put that data back in the hands of the administrators and teachers and staff to decide what kind of interventions they want to use. So we worked together with our state partners to develop measure and through that process we were looking at several different existing measures and some were very behaviorally focused and some are just perceptions. We wanted to cover the behavior. Behavior and kids perceptions of the environment. You might say arent you flipping around . Isnt climate supposed to impact those things . You can think if you have a lot of kids bullying each other in the School Building that, is going to negatively impact the climate. This is the cycle and the per accept shoal process data. We want to unpack that a little bit more. And we built directly off of the u. S. Department of educations model of School Climate because whether youre writing a grant, nice to use that groups framework. And so we also thought why dont we actually try to validate the model . We have a paper that is forthcoming in the journal of School Health where we validated this model. So we collected data on over 25,000 High School Students across multiple periods of time and then through analytic approaches and addressing a variety of things about ethnicity and gender and grade level differences and were actually able to provide general support for this particular model with a couple of tweaks. Youll see we have a strong theme around connectedness in the middle. This really gets at the theme thats mark hit on about the relationship component. In fact, as were unpacking our data, thats where were seeing most of the action in terms of mechanism of change of kids feeling connected to others. Especially our africanamerican students feeling cared for by adults. Its more important than just a global perception that the School Environment is pretty positive. So that caring and connection dimension is really something thats coming out as a strong lesson for us about what we want to focus on. And certainly maps on to a lot of the research that we know from health and Michael Resnick about the important of the connections to other people. So maryland being one of the 11 states that was funded under this safe and Supportive Schools grant so thats why we call it md for maryland s3 for the safe and Supportive Schools. That is the name of our project. And were just now entering a no cost extension of that project so this is our fifth and final year of it. And we worked very closely to develop a statewide system for assessing School Climate and looked at several different measures that were available and barrowed very heavily from existing validated tools and really created this system and i get calls from School District saying can we use your system rather than us going knock, knock, knock, do you want to collect data for us . And so, in fact, we already have School Districts that adopted this model, 117 schools that are going to be in one of our districts that are scaling up the School Climate survey system. It is something to really be able to sustain the effort. They see the value in having the data to drive the Decision Making process. We enrolled 58 high schools and were going to work with over a period of four years. And we did use a random assignme assignment. We could look at our fidelity kinds of questions. It was true random assignment based on the 12 School Districts. And they are geographically dispersed. Theyre all over the state. Maryland is a funny state, we reach all the way up to near West Virginia and pennsylvania to the Eastern Shore and so then we have more urban areas toward the middle of the state near baltimore. So we do really have a nice microcosm, i think, for the United States a little bit of diversity across the state. So we use the framework to blenld together the data base Decision Making pieces as well as to bring in other evidencebased practices. So in addition to doing the tiered framework, we wanted to plug in other evidencebased practices. We would give the schools a coach. The coach worked with them around the data and provided training on a menu of different Evidence Based Practice thats they could choose. So this is the list of the Evidence Based Practices. Were not testing ovais, were checking test and connect. Were testing prot ses by which they select Evidence Base practices and implement them. It was fun putting together this initial list of programs. We did it based on our partner feedback. Everyone was hot for ovais, for example. The first year everybody signed up for ovais training. By the second year, we were down to just one school that was actually trying to implement the elements because of some aspects about the level of commitment that was needed. The elements most attractive to schools and what schools choose, frankly, the tier two supports around check and connect and check in check out, those were our most popular efforts. Similarly whether we got to cbit, everybody is saying i want something at tier 3. They sent folks to get trained and they were like whoa, i dont know that we can pull this off. I know this came out of ucla unified. But its interesting when you put out this menu and they get all excited about things then you see what do they actually going to pick up and run with and what are they going to implement . We had a lot prove ses daof pro well. Were selecting and implementing other Evidence Based interventions. We have fidelity measures across the different programs. So these are just a snapshot of the fidelity tools we actually send in outside observers unaware of the schools Intervention Convention or what programs they chose from the menu to assess the fidelity the different dimensions including the positive behavior support framework and the other Evidence Based programs that they may have close en. And then we are looking at observations. We actually got a supplemental grant from the william t. Grant foundation to bring in observers. We thought weve got all the kids reporting climate. We actually are getting parents and teachers, too. Can you actually observe School Climate . What if we send outsiders in . What about the built environment of the schools . How much does that interface with the way kids are interacting with each other and connecting with each other . So we sent an observers that use different types of tools and ill talk about those. Were collecting the data to supplement the data being collected by self report. The self report data from students is one of our chief outcome of interest. So this is just giving you a little bit of an overview of how we select the students. Some schools do want to assess all the students. They feel strongly about that. Its nonidentifiable. Its annan mus in that framework. It allows us to capture the data electronically through the on line system and most importantly, it spits it right back out to administrators. They can see the data in real time and be able to sort it and create different types of charts. So we built in a lot of elements around youth involvement and youth voice. In fact, we had Youth Advisory Committee that came up with our tag line that youll see here. What kind of school do you want your school to be . I would love to say these are real children but they arent. Theyre just too perfect and beautiful. We wanted to put together a campaign that we rolled out across the schools and you can see its really about getting youth voice and youth involvement. And so they were involved from the very beginning. And then we actually got real live children from the state of maryland and one of our schools to do a video that is a lead in to the survey system so that way its kids talking to other kids and adults about why collecting this data is helpful and why its important. And so that shows that youth voice and that youth connection. And we actually drafted a script for them and sent it over to them. Theyre like yeah, this isnt any good. They marked it and came back. They said this is so great f we just hired actors or something, it wouldnt have gone well. We had kids in the project talking about the experiences and why its important to have that dat yachlt this is just a little bit of the data that we collected. And only to open your eyes to actually looking at some of these indicators and we used the three dimensions of School Climate, Safety Engagement and environment. Things that jumped out to our school staff, some get the point that mark brought up about emotions. So nearly 22 of the students feeling sad. And most of our teachers are like i never thought about them feeling sad. Theyre quiet. Theyre not acting up. Theyre not in trouble. I dont think about them being sad. And so really helped them get in better touch with the emotional situation in the classrooms. So is one that came out. So the equity piece, only 61 of our students are feeling that issues of equity or appropriate within their school. And so this is clearly we have a pretty diverse sample. Its about half white and half other minority africanamerican being the largest minority within the state. But we can see even when we weighted the data to reflect sample diversity that just over 60 are feeling issues of equity are appropriately addressed within their school. Then other issues about students needing support. 68 of students that are feeling theyre able to get the kind of support they need within their school. So these are the kind of data that we try to provide back to the schools. A little bit of a wakeup call and help them understand and we also have student data paired with parent and teachers. So we can show the similarities and discrepancies across them. As i said all this data is reported back to the schools. We actually had another grant previous to this where we developed all these reporting tools and met with principles about what data you want. People can get overwhelmed with data. We created all the different templates for data support tools to help them use the data. And, in fact, have actually seen that even in our control schools that only got access to the survey, theyre actually improving just by getting access to the data. But we are seeing greater gains in our schools that got the interventions as well. This is a little bit of an overview of the observational data. Were using two main tools. One we referred to as the assist and has been previously used. We actually go into the classrooms. Nobody goes into High School Classrooms and provides teachers feedback about the use of praise or punitive statements or who theyre providing praise and punitive statements to or much less what the students are doing. And so that data is very eye opening. We go into 25 randomly selected classrooms and provide that feedback to the schools. Theyre just amazed at how much information theyre able to get from that feedback. Then we have the safety. This is a little bit more about the built environment. It comes out of a line of research around Crime Prevention through environmental design. And so we kind of have that theme about surveillance and adult supervision. But also look at the interactions between youth and adults and where are those safe areas and not so safe areas in the school and provide this information back to the schools is another source of support. So heres just a snapshot. Like we get so much data that we have to really drill down what do we give back to the schools . This is an example of some of the feedback we give to them based on the observational data. You can see this is from the past spring. 65 of the classrooms where the students engaged during instruction. Thats pretty bad when we go into all of these thousands of classrooms across the state and only 65 of them have students that are rated as being engaged. You can see a little bit of information that were able to provide to them around the observations. This is based on the first process. Were actually finding even after just one year of implementation the program not even taking into consideration fidelity and who is implementing what and we wh we look at the main effects, were seeing significant impacts related to Safety Indicators tlachlt is really very exciting. Were hoping that those effects will continue to be sustainus i ov sustained during the course. The tier one, as i said, people are excited about picking all these other Evidence Based programs. The ruby red shiny one. Really, it ended up being the first year they focused on the tifr one. Thats where were seeing the big impacts and just over one year of implementation. I dont want to blow off the three tiered framework and say you have to jump to tier two or three. You really got to have that foundation in order to bring in other things. That was very clear to us. So the climate and connecting up with the goals of the school, many of our schools said we dont have to worry about climate, were so focused on common core, educational outcome. We really needed them to understand how climate benefited. That the use of the data and communication this is where the youth voice really came in about how were communicating data and sharing that. And certainly the coaching piece, we were actually having a conversation with all the 11 grantees a yeek ago in d. C. I brought up the kind of dirty little secret that not all coaches are created equally. And not all coaches are rock stars and not all coaches are able to affect change. And david oesher who i really admire quite a lot, he brought in the model of what is good enough coaching . We think about that good enough parent model. What is that . Youre never going to be able to hire all rock store coaches. But what is the level of change you need and what is the fit between the coach, you could have a rock star coach but then a school that has a different type of framework. So we talk about implementation support of oh, we have to have coaches and training. We really need to get into that block box wlaf is coaching . What is the good enough coaching . Is it durations, level of support, is it personalities . Is it Emotional Intelligence . There are a variety of different factors that we think are so critical for that change process. So thats just a snapshot of some of our findings from the safe and Supportive Schools initiative. And gives you a little bit of a perspective from the field and what were learning through this process. Hello, good afternoon almost. Im even closer between you and your break so i am really honored to be with you this morning and share a little bit about what i goat do in l. A. And my role in coordinating project spin. So im going to do. That but just know i wont be able to cover everything. Lets meet afterwards. I can share in more depth kind of the mot models that we use and some of the ways that were really trying to address this really important issue of working together to create safer schools. I want to say that im here representing a team of people behind me. Just in vision that visual of all the people standing behind me. I certainly want to spend some time thanking them for their work and collaboration. Project spin, spin standing for Suicide Prevention intervention now is a partnership between the l. A. Lgbt center and los angeles unified School District along with l. A. County office of ed where we partner in a very intentional way to address environments that would lead any young person to experiencing negative Mental Health outcomes because of the school experience. And we think and we believe that schools are really a change agent. And so many of you in this room obviously believe that as well. But for us, seeing parents who have maybe young people coming out to them as lesbian, gay, bisexual or trans gender or gender queer, having a school that is informed that, is intentionally taking steps to creating a safer place for their young person is really remarkable. Thats what we try to do in partnership with l unified. L. A. Unified is the second largest School District in the nation. And it has really been at the forefront in policies around lgbt inclusive experiences in schools. Both in curriculum as well as all the way far back as recognizing them in their nondiscrimination poll is czys when no other School District was doing that. They really set the ground roark for. That i want to highlight the work of judy chasen and tim cordish who works really closely with the cdc to continuing this work even today. They sent me their still doing the work today. The l. A. Lgbt center has been around for quite some time and the history of our partnership is about three years ago when there was a need on a National Level to really Pay Attention to what was happening in our schools and how they were feeling. Much like today although the conversation shifted a little bit. We decided we needed to take a role to be more intentionally supportive in a partnership with our school. So thats where the project spin, the idea of project spin came together. And we pulled together all the different organizations, national and local, lgbt youth serving, specifically, and Mental Health specifically together to look at what could we do in our partnership if we were coordinated, if we had a vision that we shared . What could we do together and how could we do that over the next couple of years . So we called together a summit and we asked three questions in our summit. What are we doing well . What are some of the challenges and the gaps and what can we do moving forward . And based on those conversations, we really set a tone for where we wanted to take this partnership. And a lot of the different practice thats weve already heard today, im not going to reit indicar rereiterate them, came from that best practices and programs that had been researched. Making sure that we were engaging in a comprehensive way our entire school community. Not just focusing on one piece of the puzzle but really addressing the whole school community. All of the themes have already been echoed this morning. It was really important to us to do so in a way that not just addressed it from a Mental Health perspective but did so in a way that shifted the narrative to celebrating the diversity of our community. And that included all of us. For so many of us when we hear about ways to support lgbt young people, it comes up in the discussion around Mental Health. Really important. But we also wanted to have the conversation around how we can celebrate the identity of our lgbt young people so that its not a narrative of higher risk for bullying. But it is also a narrative of strength much its a narrative where they get to see themselves resi resi resilie resilient and thriving in a community. In Suicide Prevention connectedness is so important. It is as well in bullying prevention. And so our work was weaving together. We were doing so much sigh lowed but in a coordinated way, there is a lot more we can accomplish. These are the Core Principles that came together in our Work Together summit. Weve been focusing on the importance of comprehensive k12 students, parents involving superintendents, involving administrators, involving teachers. Everybody thats involved in the school really making them a key part of what the work that were doing. Also looking at the whole person. Theyre wellness, all of these things. Again, coming up earlier this morning. And the third piece, collaborative. It was so important to us and really in a landmark way bringing together these players in a coordinated fashion. This is a very weve had lots of Interesting Data today. This is a snapshot of a little bit of research that was done in california. Looking at high schools and what was it that impacts an environment at a high school specifically around lgbt young people . And we chose to highlight three high schools, hostile high school, one that is typical and one that is very, very safe based on the Research Done and what we saw in this we heard it but this Research Really gave us the data around that is that there were a couple components that were really important. Visible allies and learning about lgbt identities within curriculum. That intergrated curriculum piece was key. Integrated curriculum. In the last year, we have partnered with one archive clz is one of the largest collections of lgbt libraries and artifacts and lucky for us also based out of los angeles. And were looking at how we can respond to the fair education accurate act in california. To really integrate the contributions of lgbt people in history in a seamless way so that its not in a sidebar. But that it is within the main text of what im learning. Along with Pacific Islanders and people with disabilities. Integrating that into our curriculum, we saw it can really shift how not only are lgbt young people feel but those, their classmates, right, those who are maybe struggling or those who have never had a positive context for talking about this before. Whether you talk about family diversity and our family diverse advertise and making sure that teachers felt they had the equipment and tools necessary to do so. The last thing that i want to talk about is something that im really excited and proud because it was one our newest initiatives. It is our out for safe schools. Last year in partnership with lausd we sat down and talked about the importance of out visible allies on school campuses. And there is a his tansy. How can we create an awareness around this and do so in a way that impacts young peoples lives . And thats when we created the idea of an employee badge that says im out for safe schools. It has the word ally in seven different languages. And on the back it has a list of resources that any School Member could use. We purchased 30,000 badges which was roughly onethird of all the staff in l. A. Unified. And we passed them out. And we asked people to opt in to wear them. It went along with the tool kit and several other different supports to the school. Obviously, along with us there. And we were blown away with the response. Youll see even today around l. A. Unfied not only the employee badge but the badge that says that they are out for safe schools. En that was really a visible way for people in all schools to be able to identify where they could feel the safest. Weve heard so many testimonials from staff who never felt they could come out on campus, seeing the staff, other staff, coworkers wearing the badge and making an impact on them. A and maybe one thing we havent talked about this morning is how can adults respond to incidences of bullying when themselves feel ill equipped or ill supported in that conversation . And we really found that this identifying our role as an ally, identifying our role as telling our story and being part of the community in a such a neutral yet visible way really allowed staff to feel as connected to their school and for students who scan us as adults every day to see if we believe in their potential. To be able to feel safer. Just because we were wearing a badge. This year were hoping to partner with many other School Districts across the nation who kind of like the idea and certainly looking to see how we can create more Creative Ideas, right, create more Creative Ideas of addressing this in a way that really catches the conversation right where it needs to be, engaging all of us. We certainly had challenges and we continue to learn from our experiences. Many of them had been highlighted earlier today from what our best practices. Finally, i want to leave you with these three thoughts as to what has been a Successful Partnership for us. Were so lucky to work with a School District and several districts within l. A. County that really value collaboration and partnership. But i can say personally ive learned three things. One, its a trusting relationship. Its one where we all come to the table and listen to each other. And that involves parents, students, Community Based organizations. It involves many people across the spectrum. Thats really important for a successful collaboration. The second i think is Research Really using practice thats have been studied that, have been proven. And doing so in a way thats thoughtful and not just trying something out because maybe it will work. But really being thoughtful of our practices and involving people who know what theyre doing. And the third one which i think really speaks to all the people involved in project spin both at lausd and our Community Partners is bold leadership. Leadership thats not afraid to shifting a culture around all of these different topics that weve been talking about this morning. And thinking creatively around how we can do it together. Ill leave you with that. But i can talk to you much more about all of the different interventions. Thank you for your attention. Lets give our two precenters another round of applause. I think they really shared some interesting information. We have time for two questions because we know that your stomach is growling if its anything like mine. The mikes are on either side of the room. If you have a question, please approach the mike and if there is someone that you want to address it to specifically or to both panelists, please be clear on that, too. Hi. Good morning. Thank you so much. First of all, the points that you emphasized and this is more of a comment than a question, our previous speaker as well, the importance of the core instruction, the truly the universal teaching of the skills is so important. I think when we talk about pbis oftentimes we talk about tiered model of intervention. And whether people hear that intervention word, i think our minds go to the two and three tiers and its universal instruction and core instruction that is so important. And i think you have both really talked to that today. I appreciate it. I do have a question for katherine. The schools that were involved in the pilot were they schools that had not previously implemented any of those core programs like botven or ovais . Thats correct. They not implemented botven or ovais previously. Because we had done some pbis training at the high school level, some of the schools had some prior level of exposure to positive behavior support. We assess that at baseline. And we controlled for that in the randomization. So we matched on that essentially whether we were doing the randomization. Just because when there is a big scale up in the state, some of the schools might have gotten a princip principal from another school that got trained and were concerned when you get a model in the water. It just gets picked up. Thats why we assessed any of the three tiers largely arent universal tier at baseline. We didnt have any of the schools with high scores at baseli baseline. We were able to use that as randomization. Schools are doing a lot of different things. They may have been doing a little of this and that. We get a set score that is pbis score that comes in a zero where theyre doing nothing. Most schools come in a 20 or 30 even at baseline. So we were able to control for that as part of the randomization. Any of the other advanced tirz, they didnt have access to those prior to the trial. And the survey dat yashgs the survey used, had schools also used other surveys in the past like the Youth Risk Behavior survey and were they using those concurrently . Maryland, it does participate in the yrbs. They dont get data back to a school level. There is no data provided back to the school level. It only goes up to the district level. And not all schools are part of the assessment. So its not really a tool that is helpful at a building level other than to say this is what the kids in maryland look like or this is what the kids in this district look like. But it is provided back directly for voting level Decision Making. Thank you. Is there another question . One that came through facebook. What can parents do if they feel their school is not appropriately responding to their child being bullied . And lunch. No, just kidding. I can speak a little bit about what we do in regards to working with parents. Its not just a reactive relationship but that were engaging them from day one around what are different ways for them to access support through the school, at the district and any other outside organization that is a resource. So there are multiple people to speak to whom ever they may feel the safest with. I think its really foreign have a very clear process so that parents know who theyre going to need to talk to and what that process is going to include once they do make a uniform complaint or once theyre able to kind of process, take it to the next step. I think its really important