You are viewing the EdNews Blog archives.
These archives contain blog posts from before June 7, 2011
Click here to view the new First Person section of Chalkbeat Colorado

Author Archive

Might Colorado’s school funding picture change?

Wednesday, June 1st, 2011

Paul Teske is Dean and University of Colorado Distinguished Professor at the School of Public Affairs at the University of Colorado Denver.

(These views represent the personal opinions of the author and may not reflect the position of the University of Colorado Denver or the University of Colorado system).

As we start summer – the real, post-Memorial Day, school is out, summer – it is worth reflecting on the near-term future of education funding in Colorado.

The legislature recently finished its session, which focused mainly upon budget cuts.  Both higher ed and K-12 took cuts, but in the end, these cuts were somewhat less than some feared (higher ed), or less than the original level of cuts (for K-12).  Remarkably, as the session ended, the fact that that cuts could have been worse seems to have been spun as mainly good news.

EdNews recently linked to new U.S. Census data that ranks Colorado’s per pupil K-12 spending (all revenues divided by number of students) as 40th among the 51 states (including DC).  That 2008-9 data is now two academic years behind  – two years, by the way, full of deeper cuts in Colorado (and some cuts in some other states, too, to be sure).  Consistent with other data on this subject, the Census Bureau shows Colorado spending about $2,000 per pupil below the national average.

I will leave it to others to figure out more precisely what $2,000 per pupil could buy.  It would seem, in a single class of 25 students, even if only two-thirds of funds were spent in the classroom, it would buy $33,000 worth of extra instruction for the students in that single classroom – a para-professional, lots of useful technological aides, or whatever students need most.

As we look ahead to fall 2011, districts and schools will face the legislature’s budget cuts, and it will be interesting to see the reactions.  In the meantime, a few noteworthy events will occur.

First, the Lobato lawsuit, about whether Colorado’s K-12 funding is “adequate” under the “thorough and uniform” clause, should be heard in August.  While court decisions are somewhat unpredictable, it is possible that the Colorado Supreme Court will rule K-12 revenues to be inadequate.  Should they make that decision, the next step would be somewhat unclear – as courts have limited enforcement powers (not zero powers – the New Jersey Supreme Court, after several rulings on school finance, and legislative inertia and stalemate, eventually forced budgetary actions).

Second, State Sen. Rollie Heath’s (D-Boulder) proposal will likely be put on the ballot. Heath proposes to to roll back tax rates to the rates of the late 1990s, a period of a major economic boom in Colorado (yes, that rolling back involves a tax increase from current levels).

While passage may not be likely, no tax increase proposal in Colorado is ever likely.  There is a self-fulfilling prophecy element here as well – many potential supporters are not yet on board, because they fear defeat or believe that voters have “no appetite for new taxes,” but all that could look different in November, if the Lobato plaintiffs win, the economy continues to improve, and parents see the results of the latest budget cuts as their children return to school with larger class sizes, fewer programs, fees for buses, fees for extracurriculars, etc.

Third, the lawsuit against TABOR (the Taxpayer Bill of Rights), started by Herb Fenster, will move forward, now with lots of signatories from both parties.  This argues that TABOR has gutted the representative form of government for the state of Colorado.   While also perhaps a long-shot, there is logic to the argument.

I have previously proposed consideration of the hypothetical CIBOR –the Citizens Bill of Rights.  Under CIBOR, the legislature could not cut any programs that affect Colorado citizens (where cuts are defined in real dollar terms, per affected citizen), without a vote of the citizens.

Most people would say that CIBOR is ridiculous – how could the legislature be bound by such constraints, as it needs to balance the budget?  And yet TABOR pretty much does the same on the spending side.  So, as silly as CIBOR might seem, it is just the symmetrical, mirror-image of TABOR. In either case, the power of the legislature is substantially restrained.

Fourth, a committee made up of key foundations, non-profits, and other stakeholder groups is examining the school finance act, with a potential “reform and resources” agenda. While past legislative commissions on school finance, in 2005 and 2009, were unable to come to actionable consensus, perhaps this outside group can move such an agenda forward.

So, the legislature is done for 2011, but funding issues will continue to evolve.

Popularity: 17% [?]

Challenging the myths of international comparisons

Thursday, March 3rd, 2011

Paul Teske is Dean and University of Colorado Distinguished Professor at the School of Public Affairs at the University of Colorado Denver.

(These views represent the personal opinions of the author and may not reflect the position of the University of Colorado Denver or the University of Colorado system).

The Brown Center report on American Education 2010 from Brookings was released in February, with some very interesting new data and analysis by Tom Loveless, probably America’s top scholar of mathematics scores.

The idea of American performance decline over time is not as true as people, especially reformers, seem to believe – indeed, it may not be true at all.

First, Loveless examines comparative international test results, and shows that the relative performance of American students has actually improved over the years.  Comparing recent tests with those first given in 1964, seven years after Sputnik, at a time when many baby boomers were educated, Loveless shows that American students never scored very well (debunking the myth of some past American dominance here) and that our students have actually gotten somewhat better over time, compared to other countries.

He also shows that the much admired Finland actually only scores well on one particular type of test, and notes that Chinese and Indian students have yet to take most of these tests, so their supposed rising “dominance” is largely myth, at least in terms of what we know from these tests.

It is always fascinating to me to hear how many people implicitly believe in a past “Golden Age” of American education, when we were world-wide #1 and our systems were so much better than now.  In addition to the data in this report, that idea is debunked very impressively, in a nice short read, in Richard Rothstein’s 1998 book “The Way We Were?”

One should retain the usual concerns of course – our system today has lots of problems, we need to do much better, especially with low income students – but this suggests that the idea of American performance decline over time is not as true as people, especially reformers, seem to believe – indeed, it may not be true at all.

Brookings also examines aggregate state-by-state NAEP performance, with two econometric models – one that looks at 2009 NAEP scores compared to how the state did on NAEP in the 1980s or 1990s, when NAEP was spreading across the American states, and another that looks at relative state performance in the recent period of 2003-2009.

While the focus is whether the right states won “Race to the Top” (some did, some didn’t), EdNews readers will be interested to see that Colorado ranks 18th of the 50 state on long-term performance improvement, but only 30th for the period 2003-9, with an actual decline in scores in the past six years, when adjusted for the demographics of students.

Finally, Brookings compared NAEP test items with the proposed common core standards, and finds that the NAEP questions are not well aligned and are actually too easy, compared to the standards.  Loveless suggests that this lack of alignment will further confuse Americans about performance, going forward.

The report itself summarizes the overall findings:

“An overarching theme of this year’s report is that events in the field of education are not always as they appear to be—and especially so with test scores. Whether commentators perpetrating myths of international testing, states winning races while evidencing only mediocre progress, or an eighth grade test dominated by content below the eighth grade, the story is rarely as simple as it appears on first blush. This report tried to dig beneath the surface and uncover some of the complexities of these important issues.”

Popularity: 14% [?]

A chronic shortage of revenue

Wednesday, February 16th, 2011

Paul Teske is dean of the School of Public Affairs at the University of Colorado Denver.

(These views represent the personal opinions of the author and may not reflect the position of the University of Colorado Denver or the University of Colorado system).

OK.  So, while the Joint Budget Committee and the legislature still need to work on the budget a lot – they don’t have loads of options – and based upon Gov. John Hickenlooper’s proposed budget yesterday, we are now looking at the likely magnitude of cuts for Colorado in fiscal 2011-12.  And it ain’t a pretty picture.

I am usually the one commenting on resource issues on this blog, and I guess I’ll be happy to maintain that role for awhile.  But I suspect that we will be hearing a lot from districts, schools, teachers, and parents about what these cuts mean, and what $497 less per K-12 pupil means in the classroom, and what $877 less per higher ed student means.  At a minimum, lots of cuts in jobs, programs like breakfasts and counselors eliminated, larger class sizes, etc., all at a time when we were already near the bottom of the nation in spending and programs.

Some on this blog think we have enough money for K-12, and just need to spend it smarter (like a game of limbo and “how low can you go,” I’m curious if those folks believe that there ever comes a threshold point where, yes, “money matters” (?)).

But, I’ve never seen a post or a comment that suggests we have enough money for higher ed.  And, somehow, so far, I don’t see the nascent development of more for-profit private institutions of higher education (you know, the ones that have been in the news, of late) as an encouraging sign of the private sector’s ability to deliver higher quality at lower cost in higher ed.

If we are going to have a discussion about cuts and education budgets in Colorado, let’s get a sense of where we are right now.

I think the answer is pretty simple – we have a shortage of revenue.  Our general fund budget in 2012 is almost the same as it was a decade ago, despite 700,000 new Coloradans getting services. Cuts don’t have to be the entire solution to budget-balance, over time.

The Associated Press reported last week that federal taxes are the lowest, as a percentage of personal incomes, since 1950, during the Truman administration.

In 1950, of course, we were fighting a pretty consequential and costly Cold War.

So the current, widespread notion that we are historically “overtaxed” by Washington, D.C. is simply not supported by facts. (I know there are other elements to federal spending, such as Social Security, but those involve direct and specific benefits in return for payroll taxes withheld).  And income/wealth inequality has not been at the levels we see in the U.S. today since before we had a federal income tax, and with the extension of the tax cuts for higher income Americans late last year, their deal got even better.

In Colorado, taxpayers are fortunate to have very low taxes at the state and local level – 44th lowest in the US, in fact (and 49th as a percent of personal incomes).

So in historical and comparative terms, a Colorado resident in 2011 has some of the lowest combined federal, state and local taxes in the country and in the past half century.  What we don’t have are very impressive K-12, higher ed, transportation infrastructure, health or other public services.  Such lack of public infrastructure investment may partly explain the fact that our per capita personal income rankings have actually dropped over the past decade, from seventh in the US to 15th among the 50 states.

If we want an infrastructure that supports growth and economic development, in a state economy of $250 billion per year, we probably need to talk about raising taxes to pay for it.  And, despite so much of the rhetoric on this issue and the slowly-recovering economy, now is a reasonable time, when our tax rates are historically and comparatively low. (And lots of other states are raising taxes to balance their budgets – in Oregon and Arizona by votes of the people, and in Illinois by the legislature, with lots more yet to come).

For a discussion of all this, and more, Dr. Mark Fermanich of the Buechner Institute for Governance has recently authored a report that assesses Colorado’s fiscal situation, and presents some options to consider.

Popularity: 18% [?]

No breakfast for you, kid

Friday, January 21st, 2011

Paul Teske is dean of the School of Public Affairs at the University of Colorado Denver.

Today’s Denver Post story about the Joint Budget Committee not approving more funding for school breakfasts for low income kids is emblematic of where we are with the state’s fiscal position.  Cuts will be made, to education, higher education, and other areas, even though we know that many of the things being cut are important to achieving broader educational goals.

Parents have been told, repeatedly, by all forms of research and media, that “breakfast is the most important meal of the day for kids” – so that they can learn without being hungry.   But, I guess we don’t want to pay the 30 cents for kids whose families can’t afford it –so it must not be that important.  In our $250 billion gross state economy, we can’t find $124,000 to pay for 56,000 eligible children for the period from March–June 2011.

The Post article quotes JBC member Rep. Cheri Gerou, R-Evergreen: “I honestly felt like asking parents to spend 30 cents per meal was an appropriate vote,” Gerou said. “If we did not have a revenue shortfall, my vote would be different than it is.  Nobody wants to charge children for those breakfasts, but we are where we are. We just don’t have any money.”

No, we don’t.   We are where we are.

Mainly this is because we tax ourselves at the 48th lowest overall rate in America.

And, this is hardly the only impact of under-funding.  At the higher ed level, research is strong and solid that student retention and eventual graduation are greatly enhanced if freshmen are taught by full-time faculty who have offices, with whom they can make a real connection – but, because of budget cuts, the vast majority of our Colorado higher ed first-year classes are now taught by “one-off” adjuncts or lecturers (many of whom are terrific teachers), who can’t fill that mentorship role.

More broadly, this illustrates the extreme disjuncture of Colorado’s services and revenues.  While everyone agrees that they are out of balance, as Rep. Gerou notes, the only thing the legislature can do is to cut services.  “The people” don’t get to vote this week on a new tax (which would be between 2 and 3 cents for each Coloradan) to pay for those kids’ breakfasts for the rest of the year.

This is true, even though there is polling out there that suggests that citizens would favor raising revenues before making more cuts to education services.  And, in other states like Arizona and Illinois, voters and/or legislators are increasing taxes, sometimes a lot, to pay for critical services.   But, in Colorado the legislature can’t raise revenues, and the soonest it could possibly be done by the people, with a ballot initiative, is next fall, well after fiscal 2012 cuts are already made by the legislature.

To demonstrate the disjuncture and asymmetry of our situation, imagine a hypothetical alternative: Say that homeless advocate Bruce Douglas gets petitions for a ballot initiative to require that the citizens vote on all cuts to state and local programs – where cuts are defined as decreases in spending on any specific program, when accounting for inflation and the number of service recipients.  This might be called the “Citizens Bill of Rights” or CIBOR.

I would imagine that newspaper editorials and other forums would immediately condemn the irresponsibility and ridiculousness of CIBOR – “it would take representative government away from the elected legislators,” and “it would provide no flexibility in balancing an annual state or local budget.”

But, it is not clear to me how CIBOR would be any different from, or asymmetrical to, our current situation with TABOR – the legislature can and will only cut services (totaling $3-4 billion over the past 3 years), while the citizens, who might want to support some of those services, don’t get a say, unless someone launches a big ballot initiative, 9 months later.

Popularity: 25% [?]

Quality Counts subtext: Time to implement reforms

Thursday, January 13th, 2011

Paul Teske is dean of the School of Public Affairs at the University of Colorado at Denver.

Every year Education Week gathers data on all 50 states and compares them along various dimensions in the publication’s Quality Counts report.

As always, this year’s data are interesting and worth more than a glance (due to data lags, most of the data actually come from 2008).

While I usually use this data to show how badly funded Colorado is compared to other states (we appear to be 39th in cost-of-living adjusted per pupil spending), this year I want to discuss it, in aggregate, in a somewhat different way.

Quality Counts provides four databases for states, with scores incorporating multiple data comparisons.  Colorado’s rank among the 50 states (and letter grades) are as follows: school finance 44th (D+), K12 achievement  21st (D+), chance for success 11th (B), and transitions and alignment 29th (C).   Overall, Ed Week ranks Colorado 39th, with a grade of C.

While one can quibble with the indicators used and how they are aggregated, it is important to note that, unlike state level interest groups here in Colorado, Ed Week has absolutely no stake whatsoever in where Colorado is ranked.

When we lost the Race to the Top last year, there was a widespread local feeling that we were robbed, because of our active reform agenda.  And maybe we were.  But R2T included lots of emphasis on proven results, not just on future promises, and no matter how you slice it, aggregate Colorado performance to-date is simply not impressive.  Indeed, R2T winning states were ranked #1, #2, #3, #5, #8, #11, #19, #20, #22, #23, and #31 by Ed Week – all the winning R2T states (except for DC, a special political case I think), were ranked substantially ahead of Colorado (at #39).

As one engaged in the processes of ed reform in Colorado, I do feel the state has done a great deal of legislating, and now needs to focus on implementation, which if done well can have a good payoff.

Policy scholars like to paraphrase some version of Yogi Berra on pitching when discussing the topic of implementation, noting that it is 90 percent of half of the game.  That is, the legislative fight over SB 191 was exciting, controversial, and well-engaged on all sides, but now that it is passed, the group that is, even today, meeting to implement the details of the legislation, in a context where there is considerable statutory contradiction and ambiguity, may be more important in terms of getting it done.

In my view, we have greatly underdone successful implementation in Colorado, partly because it takes resources we don’t have.   We typically have to beg one of the four or five friendly local foundations for the couple of hundred thousands of dollars to do state level studies of new reforms.  Then we expect the districts to implement the reforms with no new resources, indeed at a time when their resources are being reduced significantly.  I think that if Republicans and conservatives didn’t support most of these reforms they would tend to call them “unfunded mandates.”

So, “reform with resources,” or “resources and reform” seem like good rallying cries for the near future in Colorado.

I’ll end with a couple of specific Ed Week data points, which I think help make my point:

Colorado is ranked:

  • 46th in eligible children enrolled in kindergarten programs (2009)
  • 49th in the change in the scale score reading-gap for 4th grade NAEP (change from 2003-2009)
  • 50th in the actual reading gap for 4th grade NAEP scale score (2009)

Popularity: 13% [?]

Some thoughts on this blog

Wednesday, December 22nd, 2010

Paul Teske is dean of the School of Public Affairs at the University of Colorado at Denver

Alan didn’t ask me to write anything on this, but I want to weigh in on these recent discussions about Ed News, as 2010 comes to a close.

First, an advertisement for this site – I have found it to be a very compelling venue – now with fabulous news coverage of CDE, DPS and other districts, state board and legislature, and just a valuable one-stop for education news.  The writers are excellent and tireless.   As an experiment in “the new journalism” I think it is working very well.

Second, some related, but still shameless, self-promotion.  I like Ed News so much that my school – the School of Public Affairs at UC Denver – has started a new health policy website – called Solutions – that has many similarities.  It is edited by Diane Carman, former Denver Post reporter and columnist, who really came up with the idea, and Katie Kerwin McCrimmon is the main writer.  (It is funded by the Colorado Health Foundation and the Piton Foundation).   While not exactly the same as Ed News, we have learned much from Alan and PEBC’s efforts here, and I hope that many of you will go there, too, for Colorado health policy news.

Third, some thoughts about the blogs.  As an academic, I’ve enjoyed blogging, and find it particularly useful to try to synthesize and express ideas and research concisely for a (hopefully) wider audience than I can usually engage in the world of academia.

I find the other blog posts helpful, and the discussion exchanges mostly valuable.  The blogs on DPS board politics have been the most problematic for me – I’m not directly involved in those fights, and I do find their tone to be somewhat discouraging.   But, most of the other blog posts seem to me legitimate efforts to insert data and ideas about reforms – charter schools, teacher evaluation, teacher pay, choice, funding, turnarounds, etc – into the state discussions.

The discussions have forced me to think about what I really believe about ed reform, an area that I think often times has to go well beyond what the research evidence, limited as it is on some of these topics, can tell us.

As an academic, I see my role as a skeptic (hopefully not a cynic).  That is, I don’t think I should be a cheerleader for a particular set of reforms – there are such advocates, as there should be, and they play an important role.  I generally favor the set of reforms much discussed at this site, mainly because I think much of the non-reform, status quo in education is unacceptable.

But I am skeptical about whether these reforms will really work, especially at a larger scale.  They need to prove themselves, and while academic research is helpful in that domain, there isn’t yet enough of it, most of it isn’t good enough (largely due to data gaps, not researcher skills), and it lags the need for real –time decision-making.  Any one study rarely decides an issue definitively.

I don’t have a problem with most of the tone of the blogs and comments.  I think it is reasonable for advocates to ask others to defend what other changes they do support, if they oppose the reforms that form the core of this blog.  Or to defend the status quo, if that is what we are left with.   However I do think we should refrain from comments like – “my side is all about the kids, your side is about the adults” – “your reading of the evidence is wrong, this study proves it conclusively” – “because a district screwed something up in the past, they will screw up this new thing up.”

With blogs and comments, It can be far too easy to demonize those on the opposite side of the debate, just as it is to send quick angry emails, when you don’t see or know someone face-to-face.  This is why I appreciated the couple of beer-blogger events Alan held, or the Colorado State of Mind/Ed News event where several of us got to talk face-to-face for the first time.

At the end of the day, I think the blog is very useful. I hope the commentators who have “left” will come back to the conversation, and we will keep discussing critical issues in this format.

Popularity: 4% [?]

When Gates MET value-added

Monday, December 13th, 2010

With lots of debate about how much we can rely upon student achievement value-added scores to evaluate teachers, the Bill and Melinda Gates Foundation just weighed-in, in a timely manner, with some new, real data.  This is a very good thing for that discussion.

While newspaper headlines overstate the findings, Gates now has preliminary data from five large participating districts (including Denver) that have been part of their MET – Measure of Effective Teaching – studies.

What MET is trying to do is correlate student achievement value-added scores, scores on other tests, student evaluations of teachers and peer evaluations of videoed teaching practice.  This will give us a much better sense of who good teachers are, across multiple measures, the stability and validity of these measures,  what good teaching looks like, and, perhaps most importantly, ideas about how teaching can be improved.

The data suggest that teachers’ valued-added scores on high-stakes tests correspond with similar scores on tests that go deeper into content, suggesting that overly “teaching to the test” is not a major bias problem.  They also find that student evaluations of teachers, especially around their ability to maintain focus in class, are correlated with student learning.

This is very helpful information to have, especially as we consider implementation of SB 191 in Colorado.  It suggests that student achievement value-added data is more related to other plausible measures of quality teaching than past data might suggest.

Still, I will note that this should hardly be considered the final word on the debate.  While the many scholars involved in this study have great credentials, Gates has a large stake in proving the value of this research (and they have funded, directly or indirectly, many of the people doing this type of research).

This is an initial technical report, nothing like a peer-reviewed academic journal publication.   And, they have not yet addressed the concern raised by Jesse Rothstein – non-random assignment of students to teachers – though this is slated to be assessed next year.   They have also not yet assessed enough of the videotaped teaching practice to understand how it relates to student achievement outcomes.

Popularity: 9% [?]

Proceed with caution on value-added

Monday, November 29th, 2010

For fairness in blogging, I feel compelled to write about the relatively new Brookings report “Evaluating Teachers: The Important Role of Value-Added” that takes a different perspective on using (value-added) student achievement data than the EPI report (“Problems with the use of student test scores to evaluate teachers”) released a few months ago.

Both studies were co-authored by a large group of distinguished researchers, so there is lots of food for thought in this debate. But let’s be clear that this is not just another case of “top researchers disagree on facts, so I can just ignore all of that confusing research and support what my gut says.”

The EPI researchers point out many flaws in the current technologies of using student value-added achievement data, and therefore recommend against its use in high-stakes decisions (like teacher rewards or firing).  The Brookings researchers agree that there are many flaws in value-added data, but ask the reasonable question “compared to what,” noting that other current methods of evaluating teachers are not very good either.

So it is more an issue of interpretation than what the facts are.

Perhaps the most interesting thing the Brookings researchers do is look at one problem with value added data, the low correlation for a given teacher across years (the same teacher compared in year 1, 2 and 3), with parallel correlations in other domains, like baseball batting averages, insurance salesperson rankings, mutual fund performance rankings, etc.

The relatively low correlation of year-to-year teacher’s rankings (0.3-0.4 range) has been cited by critics as a reason not to use value-added data, since it appears to have too much measurement “noise” to be as accurate as we would like (that is, in a disturbing large number of cases, the same teacher is ranked as “good” one year, and then “bad” the next year.)

The Brookings researchers suggest that the “noise” is not greater than what we see in these other domains and that this is an argument to use value-added for high-stakes decisions, even with its flaws. In some sense, this is argument by analogy, the accuracy of which we should examine.

First we have to think about whether we believe a teacher varies greatly in her performance over time.  Does a teacher have a “true type” (good, bad, average) and our challenge is to sample and measure that (pretty consistent) “type” correctly, or does the teacher actually vary greatly in her ability over time?  (see my earlier blog on this topic.)

If we think the teacher type is relatively “fixed” than the low correlation is a big problem, and it represents “noise” and our inability to sample well enough to find the “true type.”  Personally, that perspective makes more sense than a belief that teacher quality varies greatly from year to year.

Second, let’s examine the analogies more carefully.  Hitting a baseball thrown 60 feet at 90 miles per hour is a notoriously difficult and fickle skill – concentration and mental state do seem to matter a great deal – perhaps also so does the quality of pitching, non-random choices about which hitter to put up against which pitcher (lefty against righty, fastball versus curveball), etc.

It makes sense to me that top hitters one year might be less effective the next year, when they might be facing a divorce, or a contract year, or whatever.  Also, baseball players are well-known to be highly compensated, perhaps partly for the risky elements of their performance over time.

This seems to me a weak analogy to a teacher who has six hours and 180 days a year in a more self-controlled environment to perform “good, bad or average.”

Second, let’s look at mutual fund performance.  Here, from living 15 years in NYC, where the financial markets are eaten for breakfast, I am quite confident that the top investors in boom times (buy more tech stocks in 1997!) are also likely to be among the worst investors in downturns (buy more tech stocks in 2000!) or slow growth periods (when the top performers are cautious, diverse portfolio investors).

Academic studies of financial markets strongly suggest “random walks” and very little likelihood that we should expect high correlations across years in top performers, making this, again a poor analogy.

Third, insurance sales seem similar to financial investment to me – perhaps during good economic times a particular type of salesperson is more easily able to get families to spend money on insurance – it might require a different set of skills to be a high sales performer  in a recession.   Thus, the low correlation is in fact caused by factors outside the salespersons’ control (as with the financial markets, and probably partly in baseball too).

Their best analogy might seem to be the use of ACT/SAT scores for college admission despite a relatively low correlation with student GPAs (and the fact that no other measurable admissions factor has a higher correlation).  While I first found this argument  more compelling, it is severely flawed by the fact that they are now correlating two different things – actual student course achievement and a specific test – not the same thing over multiple years – a statistician would expect more “noise” in correlating two different things.

So it seems that the Brookings researchers can’t have this both ways.  Either you believe teacher type is fairly fixed, and then the low correlation is really a problem with our measurement technologies and a true problem with using the data.

Or you believe that teachers’ quality varies as much as professionals in these other fields, where it seems quite clear that much of this variation is a function of the external environment.   Many teachers have argued that their lack of control over their environment is a major barrier to how much they can move students achievement – the non-random assignment of teachers to groups of students, the variation in students from year to year, variations in other supports in the school, changing curricula, etc.

So where does this leave us in a discussion that is not just theoretical? After all, Colorado’s State Council for Educator Effectiveness is working, even as we blog, on implementing the SB 191 requirement that 50 percent of a teacher’s high-stakes evaluation be based upon student achievement.   Researchers agree that value-added has problems – they disagree about how severe those problems are, and whether focusing too much on these flaws, and not on problems with other forms of evaluation, makes the “perfect the enemy of the good.”

My belief is that there are many current problems with using value-added – some are fixable, with more and better tests, with better administration of tests (to avoid outright cheating and overly teaching to the tests), and with a clear understanding of the challenges created.

But other problems are not easily fixable, and perhaps we don’t want to fix them because they have value within schools – the non-random assignment of students to teachers within a school, the group nature of teaching, especially in higher grades, the difficulty assessing performance improvement in the arts, physical education, and other domains.

These issues suggest using value-added data very carefully, and as only one component in a meaningful, high-stakes teacher evaluation system.

Popularity: 10% [?]

Are vouchers really back?

Monday, November 8th, 2010

I was surprised to see the Douglas County proposal to use public money for private school vouchers, which has already received some commentary.  Douglas County supporters seem to be selling the idea more on the possible financial savings than other possible benefits of expanded choice.

In Colorado, since the 2004 court ruling, vouchers have largely been dead, and they haven’t made much headway elsewhere, when put up for votes.  At the same time, advocates have made more progress in this domain by promoting state-level tax credits for private schooling – not the same as vouchers, but with some similarities, as in Arizona, where is a case is currently up for legal review.

Interestingly, if you had to draw a line in the sand between Democrats for Education Reform’s education reform agenda and that of the Republican party, the biggest differences would be vouchers, which are not supported across-the-board by DFER types.  Charters are the preferred form of choice for DFER, since they keep public monies in public schools.

I suppose that the Tea Party and the recent Republican electoral gains have and will embolden advocates to renew their push for vouchers.

As they do that, they might want to think about the distinctions between where vouchers are applied.  Terry Moe, in his excellent 2001 book “Schools, Vouchers and the American Public,” which examines public opinion on school choice, found a lot of public support for using vouchers to help low-income families in urban areas with mostly terrible school options.  He did not find nearly the same political support for more widespread use of vouchers.

Indeed, wider public opinion about vouchers in a place like Douglas County has tended to view the idea as meant to promote segregation by class and public subsidization of religious schooling, which are not widely popular ideas.

Personally, I support greater experimentation with vouchers for low-income students in urban areas – the DC program showed some promise.  The broader research on these vouchers is actually pretty good, mainly from Paul Peterson’s Center at Harvard, and by his former graduate students – Patrick Wolf and Jay Greene.

While the findings are mixed and certainly don’t show panacea-like results from vouchers, there do seem to be some modest student achievement gains from using the vouchers (and the best studies have utilized lotteries to reduce the likely self-selection effects, though the methodology doesn’t allow us to completely rule that problem out).

But, the idea of Douglas County saving money via vouchers may be problematic.  I haven’t seen any details, but the idea seems to be a voucher at 75 percent of per pupil operating revenue (a number that I assume includes some state money, a point to consider again in a minute for legal purposes).  While, in a static sense, the district would obviously save 25 percent of the amount when families exercise the voucher, it is hard to see what kind of policy will prohibit current Douglas County parents who already send their child to private school from utilizing the voucher, and thus costing the district more (if memory serves, the state voucher plan from 2003 would have required families to be in public schools for one year, before exercising the voucher – something like this is possible, but only a short run solution to the “switching” problem).

If this idea does move forward, it would also re-raise some interesting legal questions.  While I don‘t play a lawyer on TV, I do recall that the state decision was made on local control grounds. Not tested thoroughly was the “no public money for private religious schools” element of the state Constitution (the “Blaine amendment”), which would seem to rule vouchers out, on their face, if any state money is involved (as opposed to just local funds).

Popularity: 3% [?]

Learning the right lessons

Monday, November 8th, 2010

I have been critical of efforts to jump too quickly into evaluating teachers largely based upon student test score data because our current available tests are poorly aligned for this task, infrequently given, and the statistical technology of tying (relatively small numbers of) specific students to specific teachers is more limited than most advocates seem to realize.

At the same time, I do think we need to evaluate teachers (and other education professionals) much better than we do now, and there is lots of room for improvement since our current systems are so poor (and there is some truth to guarding against making “the perfect the enemy of the good”).  And we need to be clearer about why better evaluation is important, both on the downside (perhaps firing or redeveloping poor performers) and on the upside (rewarding and learning from top performers).

Often, in these discussions, we make assertions about how things are done in the revered private sector and more specifically within business firms.   Often these assertions have no empirical basis.

So it is nice to see a thoughtful new report, from Julie Kowal and Emily Ayscue Hassel of Public Impact, that actually examines the literature on corporate employee evaluation systems, and what public education can learn from that domain.

While most of their conclusions align with common sense, and tend to promote mixed evaluation approaches, they do have some more interesting suggestions.  These include measuring more often, being ready to adjust measures when they don’t associate with organizational success, taking team outcomes seriously, and protecting evaluation from leniency and bias problems.

Popularity: 4% [?]

Colorado Health Foundation Walton Family Foundation Daniels fund Pitton Foundations Donnell-Kay Foundation