Last month I wrote a blog post about my lack of confidence in educational research, some of which strikes me as politicized. My basic point was that in some cases you could read only an author or think tank’s name and guess a study’s conclusions with a high degree of accuracy.
As you might imagine, the post created a stir. I had some stimulating conversations with Kevin Welner, a University of Colorado education professor and director of the National Education Policy Center, which I mentioned in my post. As I wrote in that post, I like and respect Welner. Our discussions were (to use diplomats’ language) frank and open and at their conclusion we decided this was an interesting enough topic to merit a broader conversation.
On Monday, we convened a group of nine people for a two-hour discussion about research, policy, politics and the media. We agreed that the conversation would be off the record, so I can’t say who attended. Let’s just say it was an interesting mix of academics and policy folks.
We did not solve the research and policy worlds’ problems. In fact, if anything, I left the conversation feeling more downcast than encouraged. But I came away with a better understanding of researchers’ perspectives, and why it is so difficult for advocates and policymakers to use research well.
Here are my undoubtedly over-simplified interpretations of some of the main points that emerged:
- Research by its nature is reflective and not oriented toward action. As one participant put it, good research consists of paying attention to what happened in the past, with the aim of avoiding the mistakes of the past. “Research more often describes the problem than effectively prescribes the solution.”
- From a researcher perspective, policy is “often (recklessly) ahead of what we know.”
- From a policy perspective, research often isn’t timely enough to have an impact on the policy debates of the day. “It takes decades for consensus to form around research findings, and for real knowledge to emerge.”
- While most researchers would not classify themselves as political advocates, “our values guide the questions we ask.”
- Some researchers, though, have crossed the line and have taken on more of an activist role. “Their research is pretty predictable,” one participant said.
- Most policymakers, though, “lead with their values, not with research findings.”
From the perspective of some of the academic researchers in the room, the combination of policymakers, advocacy groups and media outlets, all uncomfortable with nuance and ambiguity, form a toxic brew. While the best research lives in the gray areas, subtlety and nuance are the enemies of soundbites and ideologically-driven debates and policy fights.
Policymakers and advocates want definitive conclusions. Researchers shy away from yes and no answers. On controversial legislation like Senate Bill 10-191, the teacher effectiveness law, researchers’ innate “caution is seen as trying to undo the intent of the legislation,” one participant said.
Good academic research is by its nature reflective and deliberate. Hence the “ivory tower.” Is it possible to bring research into harmony with a political and media culture that runs on adrenaline, competition and ideology?
It’s hard to see how. One researcher at the gathering suggested that at more tranquil times on the political calendar – summer in a non-election year, perhaps –someone convene a group of researchers and policymakers to discuss in depth and detail the issues likely to emerge as hot-button legislative issues in the coming year.
Ultimately, we will never have a system that approaches the ideal. Policymakers will see some researchers as timid wafflers. The researchers will view those policymakers as impulsive and shallow in their policymaking.
One solution proposed in passing that everyone seemed to like: Have the state legislature meet only every second or third year. That would allow time for deliberation and keep laws lacking a research basis from being passed just to justify the legislators’ existence.
Now there’s an idea worth pursuing.
Popularity: 22% [?]
“Is it possible to bring research into harmony with a political and media culture that runs on adrenaline, competition and ideology?”
“Policymakers will see some researchers as timid wafflers. The researchers will view those policymakers as impulsive and shallow in their policy making.”
I think those quotes say just about all anyone needs to know about the situation.
Meanwhile, those of us who actually DO the work everyone likes to talk so much about are left to weather the shifting winds of political fortune, reconcile contradictory findings (after sorting the valid from the invalid ones), have some spark of our own thinking and imagination (since we’re professionals after all) and actually get kids educated in the process. It often feels like we and the kids we choose to serve are just props in this bizarre theater that plays in various media outlets.
And in all of it, the best we can come up with is it’s unions defending the status quo and bad teachers that prevents real reform from happening.
I think that you are wildly optimistic to believe a few years is enough to resolve policy issues based on data.
Take for example the belief that the world is flat and we are at the center of the universe. This certainly follows from everyday observations and it is good for the ego. After a few centuries there are probably still some people who believe in a flat earth. Evolution is still controversial, even with multiple fields of study supplying supporting data and a mechanism of action to support the theory. But, we have only had about a century of that idea.
I probably should not even mention human-related climate change. I think that public policy will take more than a few years to resolve that one as well.
Science labs frequently produce data that support a hypothesis, so I am not bothered by the fact that one group produces data supporting a view. There can then be experiments to show the view is true for some cases, which one lab may do. Another lab may show it is not true for all cases. Newton developed a mechanism for a universe without a flat earth and Einstein developed a model where Newton was not exactly correct. A lot of time and a lot of experiments were involved while these ideas were worked out.
An objection to the way public policy is made is the lack of willingness to admit that the policy may not be correct. I believe much better policy could be made if the policy was set up in the form of an experiment. That might produce a more thoughtful policy in the first place. A policy set up as an experiment might produce useful data at the end. That data would allow for a better policy in the future.
Large sums of money are spent and peoples’ lives disrupted when public policy does not admit fallibility. We then see a similar experiment a few years later where only the names have been changed.
Clinical trials work in medicine. While medicine is a decade or two behind where the scientific research is, I believe medicine has made much more progress than education.
Perhaps you can find optimism in a better process.
I read this piece and thought (much along the same lines as Jeff) that research and policy have far more in common with each other than either has with practitioners — for whom research is too rarely definitive to be more than directional, and policy whose prescriptions too often are both ethereal and ignore basic quality issues.
It is, to my mind, one of the sources of education’s problems that both these sides all to too often are placed directly at the center of the debate when in the hard reality of student outcomes they are far more likely to remain tangentially on the periphery.
The notion that research and policy are far removed from the actual work being done (practitioners) is, I believe, the most important point here. It’s interesting to note that even in this “gang of 9″ (no relation to any of the other gangs of nine that come up when googled) it doesn’t sound like practitioners were represented well. I don’t mean to say that anyone is at fault here, for even when teachers are invited to discussions of policy and research they tend not to show up in droves in the way the wonks and academics do.
Why is that?
Could it have something to do with finite capacity of a single teacher and the classrooms of students that occupy most of that capacity each day? Probably, but in my opinion, that’s no excuse.
Policymakers and researchers would do well to reach out to teachers early and often, but more importantly teachers have to insert themselves in this conversation and wear different hats. I mean, where else do you find a researcher with easy access to 30-100 human subjects every day for nine months out of the year or a policymaker who can speak with undeniable credibility about the current conditions of the field?
The thing is, teachers can’t wait for an invitation from their union, the education professor or state representative. Sure, it’s a dramatic shift in terms of what it means to be a teacher, but I believe it’s a shift toward the label of a “profession” for which so many practitioners yearn.
Believing that you need to work instead of attend meetings is no excuse? Really?
Academics and policy wonks show up in droves because it’s their job to do so. My job is to teach math to 172 kids a day (30 – 100 sounds like a dream). For teachers to participate on any practical and useful scale without significant disruption to classes we would need to create a system of release time or otherwise build more non-classroom time into our schedules, at least for some teachers who wish to participate in policy and/or research discussions. With the current systems of substitutes and test driven accountability, I think most teachers experience a very strong disincentive to take the time to participate.
This, of course, assumes that most such events continue to take place on weekdays at venues teachers don’t have convenient access to (lunch and breakfast meetings downtown, for example). I would be happy to go have a conversation with people over my 45 minute lunch if it were here.
Incidentally, back when we were developing ProComp and still talking about a career ladder for teachers, we wrestled briefly with the topic of different approaches to scheduling like including some ideas from university practices where professors do have time to do stuff like this. We agreed that this was both important and outside the scope of what we were doing. I haven’t really heard much about it since.
Jeff, I never said it was an either/or choice to teach or participate in those conversations. Doing both is possible. Lots of teachers do it – just like lots of teachers find the time to blog and teach.
I don’t disagree that allowing for release time would help teachers participate. In fact, I would go one-step further and instead of allowing one-off chunks of release time I would love to see more “hybrid roles” for teachers that allow great teachers to persue research or policy without losing them in the classroom.
I also don’t disagree about the timing and locations of meetings. When I was teaching, it always killed be that DK’s Hot Lunches were right in the middle of my 11th grade English class. That being said, there have also been an increasing number, I think, of events held in the evenings – more of these would be great.
When are the children part of any of this consideration?? Back in the day, when our nation just ‘taught’ based on the needs of kids, we were the best in the world. Now we have fallen so low in education and we wonder why. After reading articles like this about who is right, the researchers or the politicians no wonder its a mess!
We have to get back to it being about the kids and that means eliminate the DOE/Unions/Researchers!! Take politics out of the equation and suddenly the picture isn’t quite so gloom & doom.
I find it ironic that Judy’s comment arrived the same day I put up a blog post from Ted Hershberg that contains the following paragraph:
“Many people, with limited knowledge of our nation’s education history, have fallen into the trap of believing that once there was a golden age of public schooling and that NCLB has “ruined” our schools. They need to be reminded that before NCLB there was neither mandatory state standards nor accountability at the school level (let alone at the level of individual educators), that American students fared even worse in international assessments of skills and knowledge than they do today and that very few of our schools were providing students with the critical thinking and problem-solving skills required in a highly competitive and technologically advanced global economy.”
So thanks, Ted, for saving me from having to craft my own response.
Actually, states did have standards and extensive systems for accountability long before NCLB and NCLB has stifled instruction in critical thinking and problem solving. Teachers feel that they no longer have time to dwell in ideas, encourage children to apply concepts or engage in real thinking before they have to move on to “cover” the next topic in an endless scope and sequence of unconnected skills.
Ooh, fewer sessions and deliberation… That would be great. Anything that limits the amount of mandates they can pass down means less time spent on forms and more time spent on KIDS!
Should say “fewer sessions and MORE deliberation” above. Oops!
Convening such a conversation without practitioners in the room seems silly at best. In the end, each teacher, principal, school and district makes the decision about if, when and how to put research and policy into practice.
I don’t think anyone would argue with the point made by several people here that practitioners need to be involved in any substantive conversations about policy and research. Monday’s conversation was aimed at addressing the questions I aimed at researchers in my February blog post. It wasn’t intended to exclude any group of people, and calling it “silly at best” is just, well, silly.
Thanks for another good discussion here. I think one of the real problems with research done by Welner and other University based activists is when these “researchers” use their “objective research” to strongly advocate for a particular set of policies while they claim to be more objective than other advocates. As I have said many times here before, I have no problem with Welner advocating for some set of policies and engaging in vigorous debates on policy, my problem is with Welner’s and others like him hiding in the ivy tower and not admitting to a strong advocacy agenda. If only there were an institute to track and review Welner’s research projects and agenda in the same way he has turned his PhD on the think tanks. We could use a few more referees and better research for these debates.