Scientifically Based Research — U.S. Department of Education– Pg 8
Safe and Drug-Free Schools—Judy Thorne
MS. NEUMAN: I turn now to safe and drug-free schools. We’re welcoming Judy Thorne from Westat.
MS. JUDY THORNE: Well, I have to say that drug prevention and violence prevention research is somewhere between the depressed scale of the mathematicians and the enthusiastic exalted scale of reading. I don’t think we know as much about drug prevention as we do about reading. But, we know some things.
I want to talk about basically two strands of research in this field. The first is what’s been going on in the Department of Education under the Safe and Drug Free Schools, or as it started out, The Drug Free Schools and Communities Act.
There is a progressive body of knowledge and this research primarily at least I find has been in the way of helping us to understand and know what’s going on in schools in violence and drug prevention.
We started with a descriptive study that I had the pleasure of working on back in 1998 through ’91 that looked at the initial implementation of the Act.
Then there was a longitudinal study that followed from that and used some of the information form the descriptive study to select a group of school districts that we then looked at longitudinally and drew relationships between the kinds of programs that they were implementing and the outcomes for students.
Some of the important findings from that study are going to crop up again in what I have to say. So let me briefly go over those.
One is that the differences between the groups, between very extensive and well implemented programs and the less extensive and less well implemented programs were small. They were significant but they were small.
Secondly, and this helps I think to explain the small differences, is that very few of the school districts and schools were implementing models that we were then coming to understand that there was a research base growing to support specific models of prevention education, and very few of those were being implemented in the schools for a number of reasons.
We also found that districts that had a full-time drug prevention coordinator rather than someone who shared that role with five or six other roles in their district, those districts with the full-time prevention coordinator had better outcomes, and programs that combined classroom and non-classroom activities had better outcomes.
Going on from there, there have been additional studies in the department, one that focused on school violence, another that looked again at again at L.A. area school district activities.
There’s a study going on right now of the quality and impact of safe and drug-free schools funded programs and the Middle Schools Coordinator Initiative where funding has been provided to actually have full-time coordinators focussed on middle school and research to find out if that’s effective.
At the same time and sort of outside this realm of studies that focussed just on safe and drug-free schools, is a growing body of literature and findings to support specific ways of going about more often drug prevention education, but also violence prevention education, and I must say that they overlap a great deal because a lot of the risk factors in youth and in their communities are very similar.
So, based on a large number of studies, there have been a number of attempts to bring together a group of experts and sift scientifically through those studies to make recommendations about which appear to be the best models to use, mostly looking at classroom based curriculum in drug prevention.
So, we have several organizations or agencies. The Department of Education has had a panel to look at these and come up with exemplary and promising programs. The Center for Substance Abuse Prevention has done so. An independent organization called Drug Strategies has published a report on their rankings of prevention strategies. So, we have some specific curricula that can be recommended.
At the same time, others have been doing meta-analyses of these research studies and have isolated certain content that they believe is the most effective parts of these curricula and also instructional strategies that seem to be common to the most effective strategies and absent in the least effective strategies.
So, unlike the discussion that we just had in math and reading, I’m not going to go through the research and tell you what those strategies are, my main point here is that there are some established pieces of research and some knowledge of what ought to be happening in classrooms.
Are any of those programs perfect and absolutely, you know, doing away with drugs and violence among our youth? No, so we haven’t reached the pinnacle of that kind of program development yet.
Nevertheless, when the Principles of Effectiveness came out in about 1998, and now are re-emphasized and expanded on in the No Child Left Behind legislation telling school districts and schools to implement research based programs, there are some places that they can turn to find out what those are and figure out what would be best used in their own schools and school districts.
Implementation issues. The other thing we know, especially from the studies we’ve done of what’s going on, is that these research based programs are not widely implemented. We find very few districts and schools implementing research based programs.
A couple of studies, the study of L.A. activities done by the Department, and another one done by Chris Ringwald, Susan Annid and myself and others in North Carolina but looking at a national sample of schools and districts found that few are really looking at—about 25 percent, I think, were implementing any of the recommended models. And almost everybody implement a whole number of curricula, not a single one.
But when you look at the content and delivery, things that have been isolated by meta-analysis, it’s more encouraging. About 62 percent of schools reported that they were delivering the content that meta-analysis said was important, but not very many of them are using the teaching strategies that the meta-analyses say is effective.
Now, why is this happening? Well, one is that there is not a big transfer of knowledge from the research community to the schools. Another is, I think, a lack of money to do this. I don’t know how well the research supported curricula can be implemented on the amount of funding that they get from Safe and Drug Free Schools, which is about seven dollars per child, or could be reduced to around $3.50 if they decide to divert those funds for other purposes. So, they need additional funding if they are going to be doing those.
Another thing that I think is extremely important is pressure on time in class. The schools are under tremendous pressure to meet standards in academic areas. Unless they see and strongly believe in a link between the behavior and health of their students and those academic achievement areas, then it’s really tough to make the pitch for a lot of time being spent in the classroom or in the school day on prevention activities.
Efforts to improve this situation. First of all, I’ve mentioned the Principles of Effectiveness have been out in the field for a few years and are strongly reinforced by the new legislation, and I think that that as it keeps being disseminated is an important piece.
The Middle Schools Coordinator Initiative is a way also of attempting to influence and improve the standard level of research based implementation in the schools. In terms of adding an additional person in the district who has time to really focus on these issues and figure out what strategies ought to be implemented and to implement them.
Obviously, we have a long ways to go. In continuing the research, we face a number of challenges. We’ve heard about the kinds of designs and methods that ought to be used in school based research. I believe that experimental and quasi-experimental designs can be used. But they require very careful planning. They require large numbers of schools. They require enough time up front to really get your ducks in a row, get your entities selected. If it’s going to be schools that you randomize, that can’t be done sort of after the fact, after some schools have gotten funding to do something and go hunting around for maybe some comparable schools to compare them to. It takes a very concerted, planned effort of research.
I am definitely advocating that. That planned experimental or quasi-experimental designs be applied to specifically studying a targeted look at specific interventions implemented in the field.
I think this is what one of the earlier speakers was talking about in terms of field studies. Take the approaches that are research based or found in controlled studies to be effective, and look at them in a real setting in a number of school districts and schools at once.
Most of the research that we’re basing all of our actions on was done in relatively small groups and much more controlled settings.
I am not talking about applying experimental design to a national evaluation of the entire Safe and Drug Free Schools program. I could do a very long presentation on why I think that, but I think I will move on.
What, two minutes? Oh, dear, this is very hard.
MS. NEUMAN: I’m sorry.
MS. THORNE: No, I completely understand. But, it’s very hard to respond to, to try to pull all these issues in a field together and get them delivered.
One of the other challenges I wanted to mention though before I go on is the overburdening of schools. Where are all the schools to participate in all of the research that we’ve been talking about? There is not an infinite number of schools out there. Many of them are already engaged in specific research activities.
And, if they are not involved in a study of a particular intervention, they’ve been survey twelve times in the last year. It is tough to talk about this kind of research and then think about—if you’re in a school district or a school, you know how many times you’ve been asked lately to participate in studies. And you often have to turn them down because you just don’t have the time available to do it.
Going on to the possibilities. It seems to me we are fortunate to have reached the point where we have some evidence to go on and some models to try out in a field based setting. And I think we can use experimental designs for some of these studies.
If, as I’ve said, we can have large enough samples, if we can have the time in advance to plan it and if we have strong support from the administrations of those schools. One of the challenges that I sort of skipped over is sort of the whole logic model of what is the intervention, how can you tell when it’s well implemented, and how do you measure the outcomes? Measuring the outcomes in this area is tough. I mean, I hesitate to say this when we heard how depressing things were in math, but I don’t think the challenges are quite equal across all of the fields in terms of research. You know, not being a math researcher I can blithely say that that’s a lot easier.
(Laughter.)
You know I can somehow conceive of testing a kid’s knowledge in math. Driving violence prevention, we’re looking at stuff we can’t even see. We’re not supposed to see in the classroom. We want to know what those kids are doing when they’re not in the classroom. How do we find that out? Well, probably the best way we’ve come up with so far besides urine tests is surveys. And surveys, well, all the schools are over-surveyed to start with, but secondly, we’re facing the Grassley Amendment which tells us not to survey students on sensitive behaviors, which illegal behaviors like drug use and violence are, unless we have explicit criminal signed consent. That just adds a further difficulty to the research there.
When we’re looking at the possibilities, we should be looking at are the proven approaches affordable and effective in the real world, what new approaches are effective, and don’t forget the non-classroom activities. Most of the research that I’m aware of at any rate deals with curriculum. And as I’ve said before in that longitudinal study, we’ve got a pretty good sense that the non-classroom activities were important as well. And by that I mean things that happen outside the classroom in terms of conflict resolution projects, student assistance programs, other kinds of things that happen in schools or around school time that is not necessarily classroom related.
And finally, I think our research responsibility is to continue to look at those targeted studies of approaches, but also to continue to monitor the implementation of research based programs in the school setting. So, I see a really important role in continuing descriptive research, looking at and talking with schools and school districts about the specific models they are implementing to find out if in fact that transfer is happening and to somehow help that happen.
Thank you.
(Applause.)
MS. NEUMAN: I know, it’s so terrible. I’m rushing everybody, but I think you probably heard a startling statistic in that last presentation, which is 25 percent of all of the programs in Safe and Drug Free are research based. So, it doesn’t seem as much an issue of money as much as a concern about dissemination and better dissemination of research based practices into those programs.
