You are viewing the EdNews Blog archives.
These archives contain blog posts from before June 7, 2011
Click here to view the new First Person section of Chalkbeat Colorado

Using the right kind of research the right way

Posted by Feb 9th, 2011.

Mark Sass, a teacher since 1994, teaches at Legacy High School in the Adams Five Star School District.

Want to know why many educators close their doors to current education debates?  See Alan’s recent post “Why I don’t trust education research.”

It is dangerous for educators to act this way, but it is certainly understandable.  Whom to believe?  What am I to do?

A better approach for educators to take with research is to look at research that is at the micro-level.  Research that impacts their everyday practice versus the big policy research that gets mangled, or not,  in subjective partisan bias.  Case in point is the research that shows that constructive feedback to students has a greater impact on student learning than do socioeconomic characteristics.

Researcher John Hattie looked at meta-analyses and evaluated the relative impact of many factors, including family structure, curriculum, teaching practices and student feedback on student achievement.  His findings challenge some prevailing notions by many educators.  The following factors influence student achievement as shown by the “effect size,” or the percentage of standard deviation (the higher the number in the parentheses equals a higher impact).

  • Preterm birth weight (.54)
  • Illness (.23)
  • Diet (.12)
  • Drug use (.33)
  • Exercise (.28)
  • Socioeconomic status (.57)
  • Family structure (.17)
  • Home environment (.57)
  • Parental involvement (.51)

As you can see, many of these factors would be considered outside the influence of the teacher (this is arguable in some areas like parent influence).  These are certainly negative influences.  But are there any other influences, ones that can be under the control of the teacher that can overcome the negative factors?  Hattie found a number of practices that are more powerful than those listed above.  For example: teacher/student relationships (.72), professional development (.62), creativity programs (.65), and feedback (.73).

Doug Reeves summarizes Hattie’s findings:

“We can say, based on the preponderance of the evidence from multiple studies in many cultural settings, that feedback is not only more important than most other instructional interventions, it is also more important  than socioeconomic status, drug use, nutrition, exercise, anxiety, family structure, and a host of other factors that many people claimed were overwhelming.”

This is why some school districts, like Aurora and Adams 12, are looking at scrapping their antiquated grading systems and replacing them with grading systems that reflect this current research.  But there is great resistance to the proposed changes.  Why is this?

Our greatest challenge is to transform what we know into action: The “knowing doing gap.”  In all my years as a teacher, I have been frustrated by many things.  But the one area that has frustrated me the most is the seeming indifference to research by educators.

Yes, other fields experience this indifference, but I would argue that it is more so in education that in any other field.  I cannot imagine anyone in the technology industry being satisfied with what they know.  They want to know more and they operate from the assumption that what they do could be better.

Look at medicine and the changes that have taken place over the years when it comes to their practice.  To educators I would ask, “What keeps us from continually challenging what we know?”  Reeves says:

“Equipped with a rich literature on the theory and practice of change, educators and school leaders should be fully capable of acknowledging error, evaluating alternatives, testing alternative hypotheses, and drawing conclusions that lead to better results.  Instead, decision making processes are more likely to be guided by personal convictions that are not only antiquated but dangerous.  We can be indignant about the physicians of the 19th Century who were willing to wash their hands, but when the subject turns to educational policies, we sometimes elevate prejudice over evidence.”

I know that school and district environments can impact innovation.  But this should not keep us from moving forward. Teaching is one of the most complicated of human activities.  It is hard.  That’s why educators need to continually challenge what they know and interrogate what they do.  This push should come from within the field of education, not from left field.

Popularity: 14% [?]

9 Responses to “Using the right kind of research the right way”

  1. jj says:

    Very timely post, Mark. I suspect there are people in districts from teachers to support teams in central admin who are aware of the possibilities of valuable research. Then, there are other considerations that knowingly or not, subvert attempts to use such knowledge.

    Ever heard of Challenge Day? Many DPS schools spend thousands of dollars every year to bring this program to children in an effort to lessen violence. Does it work? Well, one would think so, if schools are willing to spend money on it and involve their students (with parental consent). The Challenge folks do offer research: http://www.challengeday.org/challenge-day-evaluations-research.php
    I wonder if this is representative of the quality of research work districts use generally, in the selection of educational programs.

  2. jeff buck says:

    I am very interested in the direction grading systems take in your district Mark, so keep us posted. I’ve used various forms of an “alternative” system for years and have found it very effective.

    I’m curious why you focused only on feedback when relationships are only .01 behind and also apparently a much larger factor than many of the others.

    My experience has been that getting to know the kids in the context of a caring, responsible adult to developing young adult (I teach high school) has been unbelievably effective. I can think of kids I don’t really like much but I care about and I make that known in many ways and I’ve seen major shifts in behavior. I’ve seen pretty obnoxious young people become willing to engage over time and I’ve seen pretty withdrawn kids show a willingness to come out of their shells. I’m not talking a panacea here but the effects are difficult to ignore.

    One girl in particular used to work hard to push my buttons every day and I’d just smile (on a good day, I’ve certainly snapped at her too) and ask if she’s feeling better after being sick, of if the movie she’s talking about with her friend was good, or if I could get some math out of her today or whatever. She eventually stated to me that she’d been very disrespectful and slowly started doing more work. Turns out she’s really smart but I might never have got to find that out if I had not persevered in caring. And I actually like her after all. (I should be clear that I understand it to be behavior and generally not individuals that I do not like)

    Frankly, it’s somewhat difficult for me to separate feedback from relationships so I guess it’s not too surprising to me that the two show a similarly high impact.

    I should also add that even with all the conversation about what makes an effective teacher, and all the attempts to capture and catalog the individual behaviors, I deeply believe that empathy (something that might not stand out in a study of video taped lessons) lies at the core of most effective teaching. We need to understand what we do in the context of humans dealing with humans, not teachers processing students. Some may bristle at the latter characterization but given the primarily industrial metaphor at the center of our education system, that’s essentially what we’re doing. And it’s not really working.

  3. Mark Sass says:

    Jeff, I focussed on feedback because we are currently engaged in changing our grading policies.

    But you are right. The relationship between student and teacher is key. I have greatly changed my own teaching “style” over the years to reflect an emphasis on a positive and professional relationship with students. I began to change after I started to use end of the year student surveys. Surveys that I’d like to see used more by educators. As you know good, constructive relationships between students and the teachers are a challenge when you have 150 students. However, there are ways to “manage” those numbers. I do not have all of the answers, as a matter of fact, I do not have many. But, I believe if we can identify teachers who have successful and constructive relationships with students we can spread the “wealth” of their ideas to others. It is difficult to identify these characteristics in teachers, but I think it is doable.

    • Mike Galvin says:

      This is a wonderfully reflective and productive conversation, thank you jj, Jeff, and Mark . This is what school improvement COULD be all about: the three incredibly powerful strategies of improving relationships with students, improving our instructional strategies, and using data to inform our work.

      I believe that our school leaders have let us down by not attending to these ideas. The job of school leadership (administrators AND teachers) should be first to provide high expectations for the “culture of learning” reflected in this thread. And second, providing teachers the encouragement and support they need to make this new culture a reality.

      To CDE and our political establishment , think more about how you could support these school cultural changes rather than spending time and resources on questionably effective policy changes like SB 191 and expensive and unproven “reform initiatives”!

  4. Thanks to you all for some wonderful insight! As a sped advocate I work very hard at bringing teachers to the “IEP table” as meaningful, informative professionals. Often times, when we first meet, the teachers are hindered by the “gatekeepers”/administrators. But once we get over that hurdle and focus on the individuals – teachers and students – often we can have an effective, nurturing environment that both the student and teacher benefit from later in the classroom. I believe it is the institutional hampering of innovation, by administrators and CDE, of educators that results in limited to no growth of the individual student. We all need to work as a team to learn what provides educational benefit for the individual student.

  5. jj says:

    I read Hattie’s book a couple of months ago. I found the presentation of the meta-analyses compelling but I’d really like to know what others think. Perhaps one reason educators eschew research is that too many of them have never really done any research on their own–and I mean real master’s or doctoral-level research. If all I had was my BA, I’d probably be indifferent as well.

    • Mark Sass says:

      I know plenty of educators with MAs and PhDs who eschew research and plenty of BA edi\ucators who embrace research.

      • Jeffrey Miller says:

        Certainly, but wouldn’t it be interesting to have some data on that? Perhaps some research into how research is used by real teachers? I’m guessing such is likely already out there, I just don’t know where.

  6. Holly Yettick says:

    Jeffrey: The Ontario Institute for Studies in Education (at the University of Toronto) is doing some interesting work on research use in schools:
    http://www.oise.utoronto.ca/rspe/

Leave a Reply

Colorado Health Foundation Walton Family Foundation Daniels fund Pitton Foundations Donnell-Kay Foundation