The PEBC Network
Click to PEBC.org
Click to EdNewsColorado.org
Click to Boettcherteachers.org
Click to Education Research and Practice

Archive for the ‘Accountability’ Category

From the publisher: Shine a light

Tuesday, August 31st, 2010

Over the past many years, we have been inundated with articles, columns, essays and rants about the widening red-blue divide in this country. People on one side of the divide can no longer even fathom the perspective of people across the way. We are a long way from reconciliation.

I’m afraid a similar chasm has opened in the world of public education. On one side are people who favor data-driven accountability, school choice, autonomy and pay-for-performance (to name a few issues). I’ll call them (somewhat inaccurately) “outsiders.” On the other are “insiders,” those who feel that market-based reforms and an over-emphasis on testing constitute an assault on public education and specifically on teachers.

Rhetoric on both sides is tilting toward invective. Name-calling is crowding out dialogue. The pitched battle earlier this year over Colorado’s Senate Bill 191 – now the educator effectiveness law – exemplifies the tenor of the debate.

An ongoing Los Angeles Times series, “Grading the Teachers,” provides the latest flashpoint in this escalating rhetorical war. The newspaper hired a researcher and crunched seven years of data from standardized tests to create “value-added” scores for 6,000 third- through fifth-grade Los Angeles Unified School District teachers.

This week, the Times published a searchable database that allows readers to find any L.A. teacher in grades three through five and examine his or her value-added score. Is this teacher, by this measure, getting below average, average or above average test score growth from his or her students?

Some teachers’ scores are based on multiple years of data, some on just a couple. Any teachers in the proscribed grades who taught 60 or more students between 2002-03 and 2008-09 were included.

The L.A. school system has had this data for some time but has never released it to teachers – who might have used it to reflect on their practice. This is one reason the newspaper decided to make the information public.

Leaders of local and national teachers’ unions responded with varying degrees of outrage. Some trotted out the canard that the paper was “anti-teacher”  because it chose to make public this potentially embarrassing and methodologically questionable data.

Fred Klonsky, a Chicago teacher and popular blogger wrote:

“For these reporters and editorial board, there is no complexity in assessing student performance that a series of tests and growth scores can’t simplify. It is simple enough that based on their results they are willing to put the names of teachers who don’t match up to the reporter’s expectations in their article.

“This is a shameful act of attempting to humiliate teachers. It is teacher bashing at its worse (sic). They treat teachers like Johns busted for hiring a prostitute. Why not publish their home addresses and phone numbers?

“Watch out. That’s next

Meanwhile, some leaders of the “outsiders” were over the moon. Charter school advocate and hedge fund manager Whitney Tilson said on his blog:

“I have no doubt that it will be among the most important and influential education-related articles of the year. This is breakthrough journalism.”

And education journalist John Merrow wrote on his blog:

“I applaud the Times for bringing this to the forefront.  I worry that it could be a step backward if it merely heightens the significance of scores on bubble tests, but that’s a risk worth taking…

“So rather than boycott the LA Times, I say we should all subscribe.  And we should turn up the heat on administrators who refuse to set  and maintain high standards for their teachers, and on unions that don’t work hard to give teachers opportunities to be excellent.”

Even as Education Secretary Arne Duncan and other prominent “outsiders” backed the Times, the paper itself published the database last weekend with a somewhat defensive explanation:

“Although value-added measures do not capture everything that goes into making a good teacher or school, The Times decided to make the ratings available because they bear on the performance of public employees who provide an important service, and in the belief that parents and the public have a right to the information.”

And there were prominent voices of moderation in this debate. Even some prominent education voices usually associated with the “outsiders” flinched at the Times’ decision to publish teachers’ names and value-added scores. Rick Hess of the American Enterprise Institute wrote:

“I have three serious problems with what the LAT did.

“First, as I’ve noted here before, I’m increasingly nervous at how casually reading and math value-added calculations are being treated as de facto determinants of “good” teaching…

“… Second, beyond these kinds of technical considerations, there are structural problems. For instance, in those cases where students receive substantial pull-out instruction or work with a designated reading instructor, LAT-style value-added calculations are going to conflate the impact of the teacher and this other instruction…

“…Third, there’s a profound failure to recognize the difference between responsible management and public transparency. Transparency for public agencies entails knowing how their money is spent, how they’re faring, and expecting organizational leaders to report on organizational performance. It typically doesn’t entail reporting on how many traffic citations individual LAPD officers issued or what kind of performance review a National Guardsman was given by his commanding officer.”

So here’s where I come down on this. The methodology may be imperfect. Some teachers can’t be evaluated based on value-added criteria. Yes, some embarrassment will result.

Still, this information serves the public interest. If we could get similar data from Denver or any other school district, I would be inclined to publish it.

I’m no longer the parent of a school-aged child, but if I were, I would want this kind of data as I chose a school and possibly even a classroom for my child. Yes, this information will make principals’ lives more difficult, as pushy parents demand spaces for their kids in the most effective teachers’ classrooms. But isn’t parental engagement what we all want?

Arguments against the release from people like Hess are reasonable and give me pause. There are a number of red flags here. But then “insiders” like Klonsky make arguments so specious that it makes me think the more we know the better, even if the information is far from perfect.

Here’s what started bothering me during the SB-191 debate, and continues to fester. Some (nowhere near all) “insiders” – teachers and teacher advocates – have made the following arguments at different times over the past few months.

  1. Anyone who wants to use imperfect, emerging data systems as part of a teacher evaluation system is by definition hostile to teachers.
  2. Standardized tests, in any event, don’t measure the stuff that really matters.
  3. Any form of evaluation that has a public component, or is released publicly represents a deliberate effort to shame and humiliate teachers.
  4. Any school that is not part of the traditional public system and shows results above and beyond those of similar schools from within the public system is teaching to the test and creating automatons lacking critical thinking skills. Their students won’t succeed in higher education, and these schools aren’t the promising models “outsiders” claim they are.
  5. Teachers get all the blame when the main challenge to student success comes from disengaged parents and unprepared kids. There’s only so much teachers can do given the raw materials with which they must work.
  6. Anyone who hasn’t been a teacher can’t have a legitimate point of view about how to reform public education. And those former teachers who have become philosophical “outsiders” are corporate toadies and sell-outs.

So the message I’m getting from these folks is that only they know what constitutes good teaching and learning. It isn’t measurable in any traditional sense, but real professionals know it when they see and feel it. If only all the buttinskis from foundations and community organizations and non-profits and the media would let teachers teach, and give them adequate resources, everything would be dandy.

History shows these arguments to be naïve and ignorant at best, disingenuous and dishonest at worst. I’m still waiting for specific, affirmative, measurable ideas and plans from the faction of people who hate what’s happening now.

So far all I’m hearing is why everything Obama, Duncan, Bloomberg, Klein, Vallas, Bennet and Boasberg  are trying is an unconscionable attempt to dismantle public education.

We’d all like to see better neighborhood schools and more money, wisely spent, for public education. So, “insiders,” how, exactly, do we get there from here?

I eagerly await your responses.

Popularity: 23% [?]

The shortcomings of value-added modeling

Tuesday, August 31st, 2010

Hello EdNews readers. I’ll be checking in occasionally with blog entries here focused on new and worthwhile research. For this first blog, I want to point you to a research brief published on Sunday by the Economic Policy Institute.

The piece, with the dry but informative title of “Problems with the Use of Student Test Scores to Evaluate Teachers,” is authored by an extremely impressive collection of accomplished researchers. If you read nothing else about education this week, please read the three-page executive summary (then continue on and read the rest!).

Before discussing this research brief, I want to re-introduce myself. I teach school policy and law at the CU Boulder School of Education, where I also direct the Education and the Public Interest Center. In my blog entries here, I will try to point readers to useful resources on the EPIC website in addition to resources – like the new Economic Policy Institute brief – from other places.

The main point of the EPI research brief is straightforward: while value-added modeling (VAM) is a technical advancement that highlights student growth, the numbers generated are nevertheless too inaccurate to be used as a primary factor in making high-stakes decisions about teachers. That is, if someone tells you that a teacher is good or bad based on a VAM calculation, you are wise to take the judgment with a sizeable grain of salt. This is the same warning that I — with far less impressive credentials — issued a couple years ago, as did the National Academy of Sciences earlier this year.

The full EPI research brief does a great job explaining how and why high-stakes VAM policies cannot be supported by VAM itself. But there’s one quote and one illustration/study that I want to pull out of the brief, to hopefully entice you to read the entire thing.

First the quote:

“There is simply no shortcut to the identification and removal of ineffective teachers” (p. 20).

As I write these blog entries throughout the year, I could probably begin each one with, “There is simply no shortcut to…” In part, this reflects the complex nature of schooling, but it also reflects the sad state of policymaking, where politicians and others are so easily enticed by the quick fix.

The replacement of ineffective teachers with effective ones is unquestionably a worthwhile policy goal, but it’s much easier said than done. A policy intended to accomplish this goal would have to reliably (a) identify the ineffective teachers (without wrongly targeting the effective ones), and (b) identify and recruit effective replacement teachers. Also, the policy should accomplish this in a more cost-effective way than alternative possibilities (but given the problems with the first part of this puzzle, we’re not yet at the point where we should worry about such comparisons).

Now for the illustration. I’ll quote from page 2 of the executive summary:

One study found that across five large urban districts, among teachers who were ranked in the top 20% of effectiveness in the first year, fewer than a third were in that top group the next year, and another third moved all the way down to the bottom 40%. Another found that teachers’ effectiveness ratings in one year could only predict from 4% to 16% of the variation in such ratings in the following year. Thus, a teacher who appears to be very ineffective in one year might have a dramatically different result the following year. The same dramatic fluctuations were found for teachers ranked at the bottom in the first year of analysis. This runs counter to most people’s notions that the true quality of a teacher is likely to change very little over time ….

This is scary stuff, but only if used unwisely – only if policy makers give too much credence to the scores. VAM approaches do tell us something; a teacher (or school) whose VAM scores are consistently at the extreme high end of the distribution are very likely of higher quality than those consistently at the extreme low end. So here’s my alternative proposal: use VAM approaches as a first-stage, cost-effective tool that will help inform a more in-depth, second-stage quality analysis. A teacher or school at the bottom (e.g., the bottom 5 percent) in a state or district should be identified for classroom observations, principal evaluation, and other hands-on information-gathering that can lead to a determination of professional development needs or removal/turnover. Similarly, a teacher or school at the top might be identified for further study that might help us learn from successes. This approach has three major advantages:

  1. The ultimate evaluations of teachers and schools will not be made based on the test scores; they will be more thorough and reliable.
  2. The use of VAM here is supportable, since it is not being used to make fine-grained distinctions among teachers, but it does serve as a good tool that allows for a cost-effective use of hands-on evaluation tools.
  3. A teacher will not feel extreme pressure to teacher to the test, since his or her career would ultimately be determined by hands-on observations and other information, not by students’ test scores.

Sadly, the approaches being considered and implemented in Colorado and elsewhere rely far too much on test scores and VAM approaches. We are rushing toward a system of teacher evaluation that is sure to wrongly identify teachers as good or bad, and it will likely be years before policymakers realize and correct the mistake.

Popularity: 13% [?]

A teachable moment?

Tuesday, August 31st, 2010

Complaints about some evaluators not fully understanding our true value …  a sense that points were taken away unfairly, despite reviewer training in the appropriate rubrics  ….  evaluators not understanding, and not crediting us, for the things we do well… a sense that someone in a higher position should reverse the injustice.   It all feels unfair.

Yes, but, most of these Colorado complaints about the round two R2T scoring could also be applied to premature teacher evaluation based upon the inappropriate use of faulty test score data.

Isn’t there some irony in the fact that some of the folks complaining about unfair R2T scoring of Colorado’s application are also among the ones who turned a deaf ear to, or brushed aside, some of the legitimate concerns about using current test scores to evaluate teachers?

My colleague Robert Reichardt made a similar point in April, after Colorado lost round 1 of R2T.  Now we feel twice the pain.

Let me be clear.  I support better teacher evaluation and we need to move in that direction, using multiple measures of better and more frequent principal and peer evaluation, and some appropriate use of student test scores.

There are certainly some individuals and groups who have looked for any reason not to advance real teacher evaluation, because they want to preserve the status quo (which is basically no useful teacher evaluation), and I don’t want to support that position.  At the same time, there are lots of others who see legitimate problems with the current technology that ties student test results to specific teacher evaluations, and want to proceed carefully, in order to do this right.  I was surprised how little attention policy makers gave to that latter group this spring.

As the implementation of SB 191 moves forward into the implementation stage, but now without federal funding to support it, we should keep these concerns in mind.

There are at least four reasons why we can’t now validly and reliably link teacher evaluations to student test scores.  When we address some of these elements, we will be able to more fairly and more effectively evaluate teachers.

First, we don’t have good value-added tests.  A annual March CSAP test is not good enough (you need a valid beginning and end of year test to the same students whose gain you want to assess), and more than half of Colorado grades/subjects don’t even have the annual CSAP available anyway.

Second, students are probably not randomly assigned to teachers, as this evaluation processes requires.  If teacher Jane is known by her principal to be good at teaching students with serious family problems, and thus gets assigned a group of difficult students, and moves their knowledge forward by 0.75 grade levels, while teacher Joan is known to not be good with difficult students, and gets all of the easier ones, and advances their knowledge by 1.0 grade level, who has done a better job?  (It isn’t clear that we can, or want to, “fix” this, but it is a reality that skews the data).

Third, one year of data is not a large enough sample to use for a teacher – you probably need 3.  Classes of 26 students, with 50% mobility levels that are not uncommon in urban areas, leave 13 students with a particular teacher all year – that is not enough data to make a reliable judgment about teacher quality.

Fourth, lots of good teaching is joint and collaborative, especially at the secondary level.   The social science teacher may be as responsible for improved student writing as is the English teacher.  We don’t want teaching to only be a solitary practice with no sharing and collaboration.

Added to these concerns, making student test scores very high-stakes will greatly increase the likelihood of outright cheating, as well as more subtle “teaching to the test” (and not the good kind, where people teach the subjects they are supposed to teach, but the overly narrowing kind where you only ask the types of questions known to be on the test).

I won’t try to make this post double-ironic, but among the beauty of Denver’s own ProComp is that it was put together by and with teachers, and advanced by a teacher vote, and it incorporates multiple measures, to recognize that we can’t really nail down a single dimension of teaching to assess and reward.  It is disappointing that we couldn’t summon that kind of process at the state level.

To see a different way of handling this issue, Chad Aldeman of the Quick and Ed blog (a strongly pro-reform  voice) recently contrasted LA’s handling of teacher data with Tennessee’s approach:

“In contrast, Tennessee has been using a value-added model since the late 1980’s, and every year since the mid-1990’s every single eligible teacher has received a report on their results. When these results were first introduced, teachers were explicitly told their results would never be published in newspapers and that the data may be used in evaluations. In reality, they had never really been used in evaluations until the state passed a law last January requiring the data to make up 35 percent of a teacher’s evaluation. This bill, and 100% teacher support for the state’s Race to the Top application that included it, was a key reason the state won a $500 million grant in the first round.”

Popularity: 13% [?]

Information, evaluation and accountability

Monday, August 30th, 2010

Get ready to see a cheer from those who oppose any type of student data used to evaluate teachers after a report by the Economic Policy Institute is released late Sunday.  The report titled, “Problems with the use of student test scores to evaluate teachers,” is already being heralded by anti-test groups like Fair Test.  The report is actually a nod to policies  like SB191 which state that student data should only play a part in teacher evaluation.  This is something that I and other supporters emphasized in our support of the bill.  The report says:

“A review of the technical evidence leads us to conclude that, although standardized test scores of students are one piece of information for school leaders to use to make judgments about teacher effectiveness, such scores should be only a part of an overall comprehensive evaluation.”

The report goes on to conclude:

” What is now necessary is a comprehensive system that gives teachers the guidance and feedback, supportive leadership, and working conditions to improve their performance, and that permits schools to remove persistently ineffective teachers without distorting the entire instructional program by imposing a flawed system of standardized quantification of teacher quality.”

I am good with that.

Larry Ferlazzo, who blogs for the Teachers Leader Network, describes such an evaluation system currently in place at his high school in CA.  In his blog (as covered by the Washington Post) Ferlazzo describes an evaluation process that is not done “to” teachers, instead it is done “with” teachers.  This includes the use of student data that does not drive instruction, but it informs instruction.  (I have  stopped using the term “data” and in its stead use the term “information,” as “data” has grown into a controversial topic, even an impediment for discussion with many teachers).

Ferlazzo goes on to describe how his school uses multiple forms and types of data that should inform one’s practice.  When Ferlazzo had a year of poor test results, the administration did not rush in with punitive measures.  They looked at possible mitigating reasons, evaluated whether additional resources were needed and kept up with his usual yearly observations.  The following year his students were back at prior year’s levels.

Ferlazzo also relies on colleagues, students, and parents for feedback.  Colleagues observe his class on a regular basis, providing feedback on the learning taking place.  Note that the feedback is on the learning and not the teaching.  Ferlazzo also relies on something that I see missing in many teachers: self-reflection.  Ferlazzo asks himself a question that Robert Marzzano asks of teachers: If I was a student would I want you as my teacher?

Ferlazzo sums up his take on evaluating teachers:

“[Evaluations] underscore the importance of providing resources so well-prepared administrators have more time to observe teachers – of making sure teachers have regular opportunities to observe each other and give constructive feedback – of giving educators common time to prepare and evaluate assessments that show higher-level thinking skills – of making it possible for all school staff to engage more with parents. These are the ways we can help teachers become the best that they can be.”

Popularity: 10% [?]

Loco control

Sunday, August 29th, 2010

Defeat often begets a scapegoat.  In the wake of the twice-short Colorado application to R2T, this has now solidified: the judges were “perplexed by local control” which led to a lack of objectivity. This is a familiar refrain — them pointy-headed Eastern elites jest don’t git the way things work out West, what wid our frontier sensibilities ‘n all.  So local control is the Western value we refuse to sacrifice to appease these high-fallutin fiscal brutes.

Except I think it would be prudent to entertain, at least briefly, one small possibility:

Um… What if they are right?

Colorado has 178 independent school districts, and the differences in size are staggering.  Using CDE data (Fall 2008), let’s look closer at these 178 districts that contain over 800,000 students:

  • The average district has 4,560 students.  But because there are a few large districts and a lot of small ones, a better metric is median district size, which is just 603 students.
  • The largest district has over 85,885 students, the smallest has just 54.
  • 106 (60 percent) of districts have fewer than 1,000 students. 79 districts (44 percent) have fewer than 500 students.
  • The largest 10 districts combined house 56 percent of total students.  The smallest 100 districts combined house 4 percent.

Now, say what you want about Eastern elitism and impenetrable Western values, but these numbers show a control system that is loco, not local. When the median school district contains just 600 students — the same size as many urban schools, it’s not local — it’s microscopic. We are, after all, the United States, not Cities, nor Towns.  But for school districts, we somehow ended up with micro control — the Districts of Individual Buildings (and not very large ones at that). Is it really so wrong to dock points in a competitive competition for this system?

The most lucid discussion on R2t and local control was from Robert Reichardt who makes several excellent points and highlights a central contradiction. Reichardt writes that Colorado “can’t draw that straight line of authority from the Colorado Department of Education to classrooms” and this bumps up against the pervasive belief that ”top-down command and control is the way for states to get things done in school system.” This, in turn, discriminates against Colorado’s local control system which is a “tight-accountability, loose-compliance model.”

But I don’t buy it: R2T was geared to move many districts away from command and control systems, and favored “tight-loose” models (for example, charter school expansion). Moreover, Colorado is clearly a national leader with the Innovation Schools Act which provides school-level autonomy within a broader system of district accountability.  So the conventional defense — that it is the reviewers judgment, not our system which is at fault — rings hollow.

There are, of course, plenty of ways to have a “tight-loose” system, but when a super-majority of 60 percent of  school districts have 1,000 students or fewer and combine for fewer than 5 percent of Colorado’s student population, I think it fails a basic logic test, and I don’t need to blame a complicated judging system. That two of five judges took off significant points for this actually makes sense to me.  Colorado’s single largest school district has more students than the combined population of the 136 smallest districts.  So forget the technical arguments for a minute, and let’s admit that our district arrangement is nuts.

Now I’m expecting (and encourage) some worthwhile discussion here, and I am certainly no fan of large school bureaucracies, but I have yet to encounter a single person who, given the choice, would set up Colorado’s system of local districts in the same way.

Yes, local control has somehow become a given in Colorado, and any change seems off the table of discussion  – not because it has merit, but simply due to the same old education demon of politics. Maybe in the wake of the R2T decision we should take a hard look at what the Western value of local control could mean, instead of what it is. Because schools districts of 600 students it ain’t.

And Colorado already has an interesting model – the Charter School Institute (CSI) which is not counted among the traditional 178 districts, but governs 19 schools and 5,728 students in various regions across the State.  CSI has a different organizing factor: It is the district for numerous charter schools, regardless of location. As a district, it groups its schools by their governance structure (charter), not by location.

Because the idea that geography is the primary defining characteristic of any organization has been in decline for almost 15 years, yet it remains the single way we define school districts.  What would happen if we instead, like CSI, organized school districts around something other than geography?  Could we not have a single governing body for the 79 school jurisdictions with 500 students or fewer (which would comprise a total of 19,000 students)?  Could we not have one for schools receiving increased autonomy under the Innovation Schools Act (which might even encourage more to do so)?

For many of the 41 middle-sized districts with between 1,000 and 5,000 students, should we consider school districts that encompass factors other than geography — whether it is instructional emphasis, grade levels, or something else?  This would not be mandated — schools could have the choice of belonging to their geographic district, or finding a district model that would provide better services and support.

For my guess is that many of those 79 jurisdictions with 500 or fewer students actually have a lot in common, and might benefit from not creating 79 versions of many similar things.  In fact, I bet most of the smaller districts have more in common than many of the schools clustered within larger districts (for example, what does the selective-admissions, 10 percent FRL, Denver School of the Arts high school have in common with open-enrollment, 95 percent FRL Cheltenham Elementary?).

Perhaps the R2T decision offers one of those moments where we can look at a legacy system with new eyes. If we were to preserve the idea of a “tight-loose” system, could we have a more sensible method of local (not micro) control districts structured around something other than geography is one thought.  Any others?


8/31: Paul Teske’s posting from almost two years ago deserves more prominent placement than his comment below. It’s a good read, and one wonders why this obvious issue was somewhat glossed over during R2T.

Popularity: 20% [?]

On branding, and failure

Monday, August 23rd, 2010

Cross-posted from the ‘Failing Schools’ blog

This statement probably won’t surprise anyone who knows me, but I think about words a lot. Given the kind of work I do, and the places where I do it, I think about the word “failure” all the time.

I hate it.

As a lifelong perfectionist and overachiever, I’ve spent more than my fair share of time trying to avoid it. But when I decided to become a teacher, and especially once I decided to work with kids in poor communities, that became impossible. There’s the constant discussion of failure by people outside of the community: “Oh, those schools are awful. They always fail the state tests.” “Someone should really do something about those failing schools.” There’s the labeling of teachers and students: “Oh, it’s no wonder those students are failing. Look at the teachers they have!” There’s the labeling of public schools in general: “There’s a crisis in America’s public schools! The schools are failing our children!”

And of course, there are the feelings of failure: “I’m never going to get [any of a laundry list of tasks] done.” “I hate the way I handled that. I need to apologize.” “I wish I could have…”

But the one place I never actually see any failure is in the children. Sure, I’ve met plenty of kids who don’t test well, or have any of a number of “issues” some adult or another would love to wring out of them. But of the 200+ students I’ve tutored, taught, or mentored, I’ve never met a single one I’d consider a failure. I’ve never met a child I thought was stupid or deficient in any way. When you see children every day, and get to know them as whole people, there is always evidence that they’re capable of more than we assume at first glance.

The child who flunks all of the math tests may be a phenomenal cashier during a fundraiser, easily making change, multiplying and dividing with decimals in his head. The child who can’t write might be able to express her ideas very clearly if she’s allowed to draw or speak out loud. The child who might not have the quickest oral reading fluency scores may be making rich and meaningful connections to written text, if I’m willing to respect his slightly slower pace. Seeing that, I started to question whether it’s really possible that students as young as mine could fail. At least, I wondered if it was possible that they could fail all on their own, given how many other people have an influence on what they do.

If I give a math test, and 70% of my students fail, did they fail, or did I? I’m thinking it’s the latter. I may not have modeled or explained the concepts well enough, or given enough independent practice time. Maybe the test I gave didn’t test what I thought it did. Maybe some students needed to show what they could do in a different format. Maybe others needed an extra day of review.

Then I started thinking, “Well, what about me? If I work for 12 hours, sometimes more, five or six days a week, and never actually get done, am I a failure? Or is something else going on here?” I started to question how I’d been thinking of myself, and some of my colleagues. Is that teacher bad or lazy because she leaves earlier than I do? Sure, she’s not doing all the glitzy things I am, but she’s got her own children and a husband to whom she’s responsible. Is there something wrong with us, lowly teachers in a failing school, because we struggle to keep up with everything that’s required of us?

Take last year as an example. On any given day, I could be called upon to attend a faculty meeting, and a grade level meeting, and a data team meeting, and a parent conference, as well as provide food for a hungry child, help resolve a conflict, provide a safe space for children who are frightened or angry, break up a fight, report suspicions of child abuse or neglect, investigate petty crimes (stealing is huge), avail my classroom to people investigating less-than-petty crimes (Drugs? Weapons? In 5th grade? Yup!), respond to–ahem– inappropriate behavior at recess, tend to injuries (a full-time school nurse is apparently a luxury these days), dry tears, boost sagging self-esteem, maintain bodies of evidence for myself and my students, prepare lessons, prepare materials for the lessons, buy materials for those lessons, grade assignments, administer assessments, attend professional development sessions, give a presentation for others’ professional development…and, you know, teach. Fully differentiated instruction, in several different subjects, for 32 students, whose Data tell me that their performance levels span from kindergarten to 8th grade.

And I’m happy to do it all (provided I don’t have to fight crummy leadership at the same time…). Like Maria, I made a conscious choice to align my work with my values, which means serving these children. I don’t do this because there’s nothing else I can do (not to brag, but I’m a talented lady!), but because I feel it’s one of the best things I can do.

But it takes a village to raise a child. If I’m simultaneously doing my own job, and picking up the slack of five or six other villagers (nurse, social worker, role model, valet, therapist, nanny…), is it really fair to label me a failure if I can’t do all of those jobs exceptionally well?

In light of that, I started to think about this “failing schools” jive differently. Failure is still an uncomfortable concept, but if we’re going to brandish the term so frequently, then I think we ought to really dig into it. Get up close, and examine it. Think about all of its implications, not just the convenient news-hour sound-bytes.

Thus, our clunky, branding-disaster of a title. Yeah, we know. But we take it on anyway, and hope you will too. “What is a ‘failing’ school?” “What does it mean to fail?” “Who’s failing?” “Are schools failing, or are we failing schools?” It’s uncomfortable at times (though there are so many things going right in our ‘failing schools’, and we will share those, too!). But we want to provoke you to think, and question, and empathize, and get angry, and celebrate, and work for something better.

‘Cause that’s what good teachers do.

Popularity: 15% [?]

More on the L.A. Times database kerfuffle

Thursday, August 19th, 2010

American Federation of Teachers President Randi Weingarten has weighed in with a more nuanced argument against the Los Angeles Times‘ plans to publish a database of teachers and their “value-added” scores. As I wrote in the previous post, L.A. union President A.J. Duffy slammed the Times in what struck me as a hysterical and shallow diatribe. Here’s how the Times described Weingarten’s take:

(Weingarten) said Wednesday that she believed parents have a right to know how well their children’s teachers are rated on employee evaluations, but strongly disagreed with The Times’ decision to publish data showing how individual teachers may have influenced the standardized test scores of students.

Such data should be considered only as part of a well-rounded evaluation of a teacher’s performance, Randi Weingarten said, and then should be available only to the teacher, his or her principal, and individual parents. It is wrong, she said, to make such information widely available to the public.

…Teachers “look at this as a hammer, a sledgehammer, and they’re scared about it,” she said. “They’re schoolteachers; they’re private individuals…. They’re not public figures. And they just woke up one day and 6,000 names were going to be in the newspaper.”

Even my friend and fellow-blogger Van Schoales, executive director of Education Reform Now, thinks the Times will have crossed a line if it publishes the database. In an electronic newsletter sent yesterday, Schoales wrote:

…publicizing teacher names with accompanying test scores in the LA Times not only crosses an ethical line but could cause reform backlash.  I’m guessing that Diane Ravitch and a few in the NEA couldn’t be more pleased for the overreach.  While I applaud much of what has appeared in the articles, it seems patently unfair for the LA Times to publically shame a select group of elementary teachers because the retrograde teacher union and spineless administrators have been unwilling to do their jobs.  Less than 1% of LA Unified teachers were rated “below standard” by administrators according to a recent New Teacher Project study and only 13% of fourth graders are proficient readers.   Do you think there’s a problem?

Popularity: 19% [?]

The L.A. ostrich

Tuesday, August 17th, 2010

United Teachers Los Angeles President A. J. Duffy has made a complete fool of himself by calling on teachers and members of other labor unions to boycott the Los Angeles Times for…telling the truth.

How did the newspaper incur the wrath of Duffy? The Times on Sunday published the first installment in what will be an ongoing series on teacher quality. The paper contracted with a Rand Corporation researcher (though not with Rand itself) to conduct a statistical analysis of seven years of math and reading scores from L.A. students.

Not surprisingly, the paper found that some teachers are far more effective than others. The database allows readers to search by teacher name to see how much value an individual teacher has added — or in some cases failed to add — to student learning.

The paper is careful to qualify its findings, and to describe the value-added methodolgy’s shortcomings.

No one suggests using value-added analysis as the sole measure of a teacher. Many experts recommend that it count for half or less of a teacher’s overall evaluation.

And in Los Angeles, the method can be used for only a portion of the district’s roughly 14,000 elementary school instructors: California students don’t take the test until second grade and teachers must have had enough students for the results to be reliable.

Nevertheless, value-added analysis offers the closest thing available to an objective assessment of teachers. And it might help in resolving the greater mystery of what makes for effective teaching, and whether such skills can be taught.

The series is sure to spark serious debate, and merits a serious and thoughtful response from teachers and their leaders. Instead, we get this from Duffy.

After learning of the analysis and the database last week, union leaders began making automated calls to teachers objecting to publication. In the Friday evening call, Duffy said the database was “an irresponsible, offensive intrusion into your professional life that will do nothing to improve student learning.

“Our attorneys are looking into the legalities of this database,” he said in the recorded message. “This is part of the continuing attack on our profession, and we must continue to fight back on all fronts.”

One can only hope he has taken his blood pressure medication.

Tellingly, American Federation of Teachers (UTLA’s parent organization) President Randi Weingarten, who is sophisticated and smart, has remained silent on the matter.

Popularity: 18% [?]

Check-ups, not autopsies, are diagnostic tools

Tuesday, August 10th, 2010

“The CSAPs are here! The CSAPs are here!” Or more accurately, “The CSAPs have been here, we just waited two weeks to let you see them!”

I saw my school CSAP results a few weeks ago. Why the delay in their release to the public? Nefarious politics at play? Quite frankly I don’t care. What concerns me most is the way schools and districts respond to the results.

Picture district administrators, principals, and teachers pouring over the CSAP results as if they were checking lottery numbers and asking, “How’d we do?” As if it was a crap shoot. A better metaphor for CSAPs is an autopsy. When you perform an autopsy you already know the person is dead, you want to try and figure out what killed them. It’s too late to do anything for the dead person! Well, it’s too late to do anything for those students who took the CSAP. Let’s shrug our shoulders and hope we do better next year.

In education jargon, CSAPs are summative assessments. Summative assessments are end of the unit, or end of the semester assessments that should show what a student knows or is able to do at a given point and time. We usually assign grades based on summative assessments. There is no going back to remediate based on summative assessments.

Another type of assessment and as the research points out, a more effective one is the formative assessment. Formative assessments are used to see how a student is progressing and based on the results what, if any, remediation needs to be done to get the student on track to proficiency.

Instead of an autopsy, formative assessments are checkups. The research tells us that formative assessments, and the appropriate response to their results, are the MOST effective way to improve student achievement. Why am I boring you with this explanation? Because CSAP results should not be a surprise for any school.

If a school is using formative assessments properly they should be able to predict, with some certainty, how their students will perform. If there is a discrepancy between the CSAP results and the formative results a disconnect exists between what the school expects the students to know and what the state expects. Or there may be a problem with the school’s formative assessments. These two issues can be rectified if, and that is a big if, schools are using formative assessments.

I am not arguing for the elimination of CSAPs; they are used to hold schools accountable. The public has a right to know how schools are performing. I question the focus that districts and schools place on them as ways to improve student achievement. You’d be a fool not to take them seriously. But the results should support what you already know.

Here’s a report I’d like to see from the state: How did schools perform on CSAP based on a school’s prediction? The closer the correlation and I would argue the better schools perform.

It is one thing to know what ails you when you are alive; it’s another to find out after you are dead and gone.

Popularity: 34% [?]

From the publisher: Hard truths or sour grapes?

Tuesday, August 10th, 2010

Why is the DPS administration withholding CSAP results from the Board of Education, especially since the principals already have them?

-       Denver school board member Andrea Merida “Tweet,” July 28

This year we received the(CSAP) information on Friday July 30 and the embargo is until August 10.  I am particularly concerned since the principals received their scores early last week, and the embargo is now two weeks long.  Please explain why that is occurring.  This Friday is reasonable; a week from tomorrow does not seem so.

-       School board member Jeanne Kaplan email to DPS brass, August 2.

This was the rumor being floated last week: Denver Public Schools had done badly on the Colorado Student Assessment Program (CSAP) tests. To protect U.S. Sen. Michael Bennet (a former DPS superintendent) in his primary battle against Andrew Romanoff, Bennet supporters, including Education Commissioner Dwight Jones, had hatched a plot to delay the CSAPs’ release until primary day – today.

Well so much for nefarious conspiracies. DPS, we now know, has continued to make steady progress on the CSAP. Most impressive is the district’s trajectory over the past five years.

Overall scores in reading and math have climbed by 15 percentage points since 2005. In writing, the gain has been a more modest 5 percentage points. Still, this kind of slow, steady progress is what you want to see in an organization where real change may be taking root. One-year spikes are cause for skepticism.

Everything having to do with DPS has become politicized in this political silly season. Even before their big New York Times pension story coup, Romanoff supporters were peddling the CSAP delay tale. As circumstantial evidence for this conspiracy, whisperers told reporters to check out Commissioner Jones’ central Denver home. There they would find not one but two Michael Bennet yard signs. The horror!

Well that much was true. Jones, as is his right, displayed two Bennet signs in his front yard. (Yes, I took the bait and did a drive-by).

I don’t expect anyone involved in this nonsense to say “my bad” and retract anything. That would require some humility. As a community, though, we should acknowledge and celebrate success.

And it’s not as though I’m a DPS cheerleader. In 2005, when I was at the Piton Foundation, I co-authored a study with Van Schoales, then with the Colorado Children’s Campaign. Among other things, the study examined DPS CSAP scores from 1996-97, the first year CSAP tests were administered, through 2003-04. At that time, DPS was being lauded by then-Gov. Bill Owens as the state’s shining star.

The data told a different story. CSAP scores had basically been flat over the long-term. Telling this hard truth incurred the wrath of the superintendent and board members. But the numbers were there for all to see.

So by all means, let’s take a hard look at the numbers and dispense with happy talk, which has long been run rampant inside 900 Grant Street. Are Denver’s CSAP scores good? No, not yet. Do they approach those of the state, which sets a pretty low bar? Again, not yet. Are achievement gaps still a major problem? You bet. Is DPS still rife with problems? Yes.

Still, let’s give credit where it’s due. Gains over the past five years have been real and impressive. If they continue, Denver will catch up to, and eventually surpass the state. Sustaining this kind of progress is perhaps the most difficult challenge for urban public school systems. So I wouldn’t bet my house on the trend continuing over the long haul.

But I tip my hat to Bennet, Tom Boasberg and others for the legitimate progress the district has made under their leadership.

And what does Kaplan, an elected steward of the district, have to say about these new results? Once her conspiracy theory crumbled to dust, she resorted to badmouthing the CSAP scores. She may think she’s telling hard truths. But I catch the scent of sour grapes, which in this context strikes me as perverse.

In an email to Chief Academic Officer Susana Cordova and her fellow board members last week, Kaplan wrote (the following is unedited):

“Please explain to me why are results are really positive.  I understand the trend over five years, but honestly, when the Board set its goals in Policy A of a minimum of 3.5% gain per year per subject, and the only category where we came close in all 5 years was this year’s reading at 3.3% (which is cause to celebrate, I agree), what am I missing?  I don’t really care how we are doing vis-a-vis the state.  We are consistently 15 – 20% behind the state, and why is the state the benchmark?  The state scores are nothing to write home about.  68% in reading, 55% in math, 53% in writing.  State scores in Escruita (sic)  and Lectura aren’t any better 53% to our 47% and 59% to 51%.  Again, if our goal is to educate all kids, it seems to me we aren’t anywhere near where we ought to be, nor are we moving fast enough. Nor is the state.

“So, if I am wrong in analyzing these results, please tell me why. We have been instituting “reform” for almost five years now.  I had a conversation with Tom over a year ago before last year’s scores came out where we both agreed if we didn’t see significant improvement, perhaps it would be time to change course.  I don’t see significant improvement.  All I see is very, very slow growth with non-proven experiments added to already overburdened school environments.  Tell me what the next plan is and why I should believe it will have any better results.  One of my frustrations: this isn’t a pr campaign; this is about the kids and why we aren’t serving them as well as we should be.”

Reading Kaplan’s email, I  imagined a kid coming home from school one day, flush with pride. After years of failure, he has finally made the high school baseball team. He bursts through the door and tells his mother the good news.

She looks at him, unmoved. “I’m sure you’ll never get in a game,” she says. “You still suck at sports.”

Popularity: 55% [?]

Daniels fundColorado League of Charter SchoolsColorado Childrens CampaignCollege InvestPitton FoundationsDonnell-Kay Foundation