“The CSAPs are here! The CSAPs are here!” Or more accurately, “The CSAPs have been here, we just waited two weeks to let you see them!”
I saw my school CSAP results a few weeks ago. Why the delay in their release to the public? Nefarious politics at play? Quite frankly I don’t care. What concerns me most is the way schools and districts respond to the results.
Picture district administrators, principals, and teachers pouring over the CSAP results as if they were checking lottery numbers and asking, “How’d we do?” As if it was a crap shoot. A better metaphor for CSAPs is an autopsy. When you perform an autopsy you already know the person is dead, you want to try and figure out what killed them. It’s too late to do anything for the dead person! Well, it’s too late to do anything for those students who took the CSAP. Let’s shrug our shoulders and hope we do better next year.
In education jargon, CSAPs are summative assessments. Summative assessments are end of the unit, or end of the semester assessments that should show what a student knows or is able to do at a given point and time. We usually assign grades based on summative assessments. There is no going back to remediate based on summative assessments.
Another type of assessment and as the research points out, a more effective one is the formative assessment. Formative assessments are used to see how a student is progressing and based on the results what, if any, remediation needs to be done to get the student on track to proficiency.
Instead of an autopsy, formative assessments are checkups. The research tells us that formative assessments, and the appropriate response to their results, are the MOST effective way to improve student achievement. Why am I boring you with this explanation? Because CSAP results should not be a surprise for any school.
If a school is using formative assessments properly they should be able to predict, with some certainty, how their students will perform. If there is a discrepancy between the CSAP results and the formative results a disconnect exists between what the school expects the students to know and what the state expects. Or there may be a problem with the school’s formative assessments. These two issues can be rectified if, and that is a big if, schools are using formative assessments.
I am not arguing for the elimination of CSAPs; they are used to hold schools accountable. The public has a right to know how schools are performing. I question the focus that districts and schools place on them as ways to improve student achievement. You’d be a fool not to take them seriously. But the results should support what you already know.
Here’s a report I’d like to see from the state: How did schools perform on CSAP based on a school’s prediction? The closer the correlation and I would argue the better schools perform.
It is one thing to know what ails you when you are alive; it’s another to find out after you are dead and gone.
Popularity: 4% [?]
Thanks Mark, you are so right on…there should be a report that tracks the relationship between district and school predicted and actual scores. It drives me nuts that schools, districts and states make these predictions for improvement (always 3-10%) based on little and no analysis, the scores come out flat and everyone throws up their hands and predicts that they’ll make 5% next year.
It’s data rich and analysis poor planning. At this morning’s CDE press conference, senior officials shared all the ways they can slice and dice the data (some very cool) but offered no explanation or theories on why scores have been flat for the last five years. The building is burning…it would be helpful to know why with some supporting evidence. I don’t’ entirely blame CDE it’s a systemic problem in education.
[...] Formative vs. summative tests? One’s a checkup, the other an autopsy. (Ed News Colorado) [...]
The formative tests will only work as accurate predictors if the curriculum closely follows the state standards and teachers give periodic assessments formulated to adhere to CSAP protocols.
I know what will be on the test I create that I give to my students. I have no similar certainty when it comes to a standardized test. Fwiw, I generally follow the national standards and “creatively” work around the scripted, boring curriculum and my students’ scores have risen every year for the past several years, last year rising 9% in the proficient category. Could I have predicted that number? Maybe, if I gave a lot of multiple choice tests but I don’t.
Van said, “It’s data rich and analysis poor planning. At this morning’s CDE press conference, senior officials shared all the ways they can slice and dice the data (some very cool) but offered no explanation or theories on why scores have been flat for the last five years.”
If the experts at CDE don’t have the explanations or theories, what should those of us in schools do with this information? I have sat through many Data Team meetings and other building-level venues in which teachers are asked to study CSAP data (Data Teams can and IMHO must use other data sets too). We can certainly learn valuable things this way if we can remain within the limits of our statistical skills.
The more the original data get “sliced and diced,” the more nuanced the conclusions you could make … if you had any real idea what you were looking at. Of course, you could also tell a more subtle lie about what’s going on but I think the greater danger comes from sincere efforts leading to spurious conclusions and then to actions that do not help, and sometimes hurt the situation.
I absolutely support transparency but I will also assert that the ability to see a data set along with its analysis and meta-analysis does not qualify a person to use it well. I have a strong background in math but not in the sub-field of statistics. I know enough about it to realize that I have no real understanding of the strengths and weaknesses of the methods used. How can I say with confidence what constitutes a valid conclusion and what constitutes an invalid one? Given the absurd statements I’ve heard come out of otherwise very intelligent and educated people after digging through CSAP results and analysis, it seems reasonable to suggest that extensive use of large-scale standardized testing data probably should not be “just one more thing” we expect building level employees to do.
I agree with Mark. The data building level people need to master results from formative assessment and small-scale summative assessments like unit tests, data you can actually use to make real-time pedagogical decisions. Accountability could follow from the correlation between these data and the statistically more complicated results of large-scale summative assessment like standardized testing. Since most people seem to understand “accountability” as looming punishment for failure, I will suggest another possible use. A school could examine the gap between their predictions and their results when considering what kinds of professional development the faculty and staff might need. They could ask questions like – did we miss the mark because we lack expertise in appropriate teaching methods? in content? in developing organizational capacity? in understanding the cultural dynamic in our classrooms and extending out to the community? Grappling with such questions could give a school something useful to work with as they strive for improvement.