An academic mentor of mine used to say that social science research almost always confirms common sense, or … it is wrong. There is some wisdom in that, but I wouldn’t go quite as far as his more cynical conclusion – “we’ve learned one thing from 50 years of research on health and education – smoking is bad for you.”
As a social science researcher, I have always found the questions we study to be fascinating, but often the answers are frustratingly elusive. Still, some recent research in education has done more than validate common sense, and has driven important changes in policy focus.
For example, by having teacher identifiers and longitudinal data in some states (Tennessee, Florida and a few others), researchers have shown: 1) the great importance for low-income student learning of having good teachers for a few consecutive years, versus having bad teachers (yes, this validates common sense, but it shows that learning gains for low-income kids are possible under good teaching); and 2) traditional inputs into teacher pay (seniority and Masters degrees) have little or no association with student learning outcomes (less clearly associated with common sense).
This evidence points to a greater focus on teacher effectiveness, which we now are seeing in Race to the Top and elsewhere, greater equity in quality teacher distribution across schools, and more focus on teacher outputs and outcomes than on the inputs. This is what evidence based research should do.
But a study on teacher induction programs released this week, brings to the forefront this issue of research findings versus common sense or common wisdom “on the ground.” I’m particularly intrigued by this study because, at a conference last year, I observed a panel discussion of the first year of findings. Now the second year findings are out.
The study is “Impacts of Comprehensive Teacher Induction: Results from the Second Year of a Randomized Controlled Study,” by Eric Isenberg, Steven Glazerman, Martha Bleeker, Amy Johnson, Julieta Lugo-Gil, Mary Grider, Sarah Dolfin, Edward Britton, and Melanie Ali – Mathematica Policy Research, August 2009.
It is funded by the Institute for Education Sciences for 5 years. The authors are expert methodologists; 1,000 teachers in 400 elementary schools in 17 districts in 13 states are involved, and the districts had to agree to a randomized design where some new teachers get one of two high quality, nationally-respected induction programs, while other teachers get “induction as usual.” This is the gold standard of research designs (and costs $17 million).
The findings: after two years, the teachers getting better induction programs do not show any increase in their student learning outcomes, compared to the control group.
So, either common sense is wrong and a great induction program for teachers doesn’t move student achievement, or there is something wrong with this study (doubtful to me), or 2 years is too short to see student learning outcome effects (possible, and a good thing the study is funded for 5 years).
Popularity: 8% [?]
Toxic school cultures will beat down good teachers. It would be interesting to see how teachers from better induction prgrams do in schools that have positive cultures: collaboration, a belief that all students can learn, and so on. If we hold for the culture of the school, would the better induction programs perform at a higher rate?
I’d like to suggest a shift from point-in-time outcomes to a longer view. It would be interesting to know how long the different groups of teachers last in the profession. Even if test outcomes are the same, the teacher that sticks around will have a greater overall benefit (or do greater overall damage) that the one who works for 2 – 5 years and decides to do something else.
My sense is that better induction probably correlates with longer teaching careers (and probably better mental health along the way). Which group would you imagine most likely to become “disheartened?” (reference to an earlier study discussed here)
Mark’s comment is a thoughtful one – presumably the study’s randomization “controls” for school culture, but he is suggesting the possibility of an “interaction effect” – where the combination of great induction and a great school culture might produce better teachers/teaching (the study doesn’t report on this, but probably could do so if they have any plausible measures for school culture).
Jeff’s comment is also valuable – but I should have noted that the study does report no difference in teacher retention rates, after 2 years, for the different induction experiences, as well as no student learning differences. Perhaps that will change over a longer number of years, and if better induction did lead to more retained, high quality teachers that would be an important outcome – but, so far, no evidence of that.
Hmmm…interesting. A few questions (and no, I haven’t read the study)…
1. What makes the “better” induction programs better?
2. To what degree does “better” associate with improved teaching and learning…and how is that established?
3. Did both groups show improvement? If so, how does any kind of induction compare with no induction at all?
4. What are the other factors that the researchers controlled for?
Mark, I think your question about school culture is a good/interesting one. Were there any conditions or factors related to school culture that the study controlled for?
It’s like the Miami study/effort to extend day… while we know that there are many schools that make good use of extended day…when Miami placed all of its bets on that one thing, it didn’t work. Why? extra time without changing the quality of the teaching doesn’t impact much…
http://mcrel.typepad.com/mcrel_blog/2009/06/the-miami-herald-reported-recently-that-former-superintendent-rudy-crews-100-million-investment-in-miami-dade-county.html
I just did my master’s thesis on this. I would like to know if their is any difference between the impact of a “good” induction program on rural vs. suburban vs. urban schools. Living in a rural community is much different and dealing with kids, parents, etc in those environments can also be a challenge when you are a new teacher (especially a new single teacher)
All good questions. Better induction means using one of two national models that experts consider top of the line – they both involve more hours of mentoring, etc. than “induction as usual.”
Beyond that, I can’t answer all of these questions, and would suggest people go to the report itself, for more details.
Generally, I have thought that common sense argues that better inductions programs are a good things, for teachers, for schools, and for students. That may still be true, as this study is not the final word, and it may yet find positive results, with more time. Like other reforms or interventions, the corrolary question would be, if there are positive results in a few more years, how much do they cost, and is the money better used in other ways.
But, it does raise this difficult question of what happens when reforms we think make sense don’t show positive outcomes? Do we believe the research, or what we think is happening in the schools? It is easier when both of those things align.
http://ies.ed.gov/ncee/pubs/20104027/pdf/20104027.pdf
Yesterday, Mathematica Policy Research, Inc. released the final report of its IES/U.S Department of Education-funded randomized controlled trial (RCT) of comprehensive teacher induction. It shows a statistically significant and sizeable impact on student achievement in mathematics (0.20 standard deviations) and reading (0.11 standard deviations) of third-year teachers who received two years of robust induction support. That’s the equivalent of moving students from the 50th to 54th percentile in reading achievement and from the 50th to 58th percentile in math achievement.
As a basis of comparison, I note that in 2004, Mathematica conducted a RCT of Teach for America (TFA). In that study, it compared the gains in reading and math achievement made by students randomly assigned to TFA teachers or other teachers in the same school. The results showed that, on average, students with TFA teachers raised their mathematics test scores by 0.15 standard deviations (versus 0.20 standard deviations in the induction study), but found no impact on reading test scores (versus 0.11 standard deviations in the induction study).