You are viewing the EdNews Blog archives.
These archives contain blog posts from before June 7, 2011
Click here to view the new First Person section of Chalkbeat Colorado

The performance of Denver’s charter schools

Posted by Dec 14th, 2010.

The movie Waiting for Superman, and the recent signing of a district and charter compact, has energized an intense debate about the quality of charter schools compared to their traditional school peers. Local opponents of charters have focused much of their criticism by emphasizing a national statistic quoted in Superman which is based on a patchwork, multi-state CREDO study that concluded just one in five charter schools outperform traditional schools.

The question of charter school performance is vital. However this line of critique is largely irrelevant. The overwhelming majority of education policy and practice is not national, but local — charter results in Dayton and Detroit have little to do with school decisions in Denver. And in Denver, the very same CREDO study explicitly stated — and further analysis of more recent performance data confirms — that charter schools are doing far better than their traditional school peers.

Indeed, school districts across Colorado would be well advised to look at Denver’s model with an eye to replicating its success.

It’s helpful to quickly revisit the essentials. A central premise of charter schools is simple: encourage innovation and a variety of school models. Measure outcomes. Expand the good schools, and change or close the bad ones. This basic combination of innovation, evaluation, and adjustment should lead — particularly over time — to more high-quality schools and better outcomes for students.

Denver is a vibrant example of this theory.  Over the past several years a consistent (if fragile) coalition on the Board of Education has established a solid process for encouraging and approving innovative charter proposals. Denver Public Schools (DPS) created a comprehensive annual evaluation system to measure school quality. And both forged the collective political will to close charters that do poorly.

Denver’s charters now display numerous models, including Expeditionary Learning, dual-language immersion, and entrepreneurship. These innovations have indeed produced a wide variation in quality: on the 2010 School Performance Framework, three of the top five schools were charters – and so were two of the bottom five.

However the best charter schools are expanding to serve more students, while the worst are being reconfigured and face closure. And in the aggregate, charter schools in Denver are now doing far better than their traditional peers on both quantitative academic criteria and qualitative metrics.

In Denver, we have a rare and somewhat unique ability to make comparisons based on two local frameworks for measuring school quality: the Colorado Growth Model (which measures the academic growth in individual students from year to year), and DPS’s School Performance Framework (which derives academic data from the growth model but also includes non-academic measures such as student engagement and parent satisfaction). Using these frameworks should provide considerable insight into the performance of Denver’s charter schools. And what they show us is a significant different in school quality.

On the growth model, adjusted based on the number of tested students in each school, Denver’s charter schools outperformed their traditional peers in academic growth by 15 percent, with an aggregate median growth percentile of 61.3 versus 53.4 (median growth across Colorado is 50). Charter schools scored higher at every school level, and in all subjects, with a single exception. The gains were stronger in the secondary grades (particularly in math and writing) with differences of up to 30 percent.

Denver’s School Performance Framework (SPF), similarly adjusted for school size, also showed a similar 15 percent gap in academic growth among charter and traditional schools – and both schools served equal percentages of students in poverty. However, charter schools excelled even further on the SPF’s non-academic metrics, with astounding differences in measures of student engagement (43 percent higher), parent satisfaction (27 percent), and re-enrollment (16 percent) (see data section below). These gains extended across all grade levels.

The final statistic — a school’s re-enrollment percentage — is particularly interesting.  Among the many unproven claims against charter schools is that they filter out low-performing students.  It turns out that charters have a far better track record of retaining kids.

Here is a full summary of the charter and district data from both frameworks.

Even their strongest proponents agree that charter schools are not a panacea for all of public education, and there are many external and societal factors impacting schools that also badly need our attention. More research should continue to look at the similarities and differences — but the direction here is clear. Across a growing body of evidence and several years of data, the performance of Denver’s charter schools surpasses their traditional peers. Superman is not coming to Denver, but the charter schools here are doing very, very well.

Extra Credit: The Data

If one wants to dig into the details, here is a larger discussion of the data:

CREDO Study: The 16-state CREDO study, which looked at five years of data ending in 2007-08, and to which charter detractors regularly refer, has received its share of criticism over its methodology.  I don’t know it if it useful to revisit that debate, but what is inexcusable is that the same people who cite this study for evidence against Denver’s expansion of charters completely ignore its local conclusions.  The CREDO study, which used only charter schools in Denver for its statewide comparison, explicitly found and concluded that these schools performed “significantly better” than their peers (see their own press release). To argue the whole of the study while not acknowledging the most relevant part is patently absurd, even for partisan political hacks.

Colorado Growth Model: Using this 2010 data, I did a weighted average based on the number of students in each school who took the CSAP. The growth model splits grade levels neatly into K-5, 6-8, and 9-12 – so if a school is a 6-12 program, the growth model counts it as two different schools (a 6-8 middle and a 9-12 high school).  A similar division happens with K-8 schools. This allows for a more precise comparison by grade.  Under this formula, Denver has 135 district schools and 26 charters (which comprise 12% of students of all students taking the CSAP).

Remarkably, charter schools did better on academic growth in every subject and school level, with the single exception of elementary school math. Aggregated across all 161 schools, charters received higher median growth percentile (MGP) scores in reading (+4.8), in writing (+9.4) and in math (+9.3) for an average difference of +7.8 points, or almost 15% higher:

Breaking it down by grade levels, in the elementary school grades (charter students composed 9.3% of tested students), charters did slightly better on average (+1.4); better in reading (+1.1) and writing (5.7) and worse on math (-2.6).

In middle school grades, (14.8% charter students) the differences in median growth percentiles were stark: reading (+7.9), writing (+12.1), and math (+15.5), or a double-digit average of 11.8 points better.  This is a percentage improvement of between 15% and 30%. The district’s lowest scores were in the middle school grades, suggesting that the renaissance in Denver’s middle school years is primarily driven by charter schools.

High school scores (8.7% charter students) were also positive: reading (+1.0), writing (+7.2) and math (+10.1), or an average of +6.1 points or 11% improvement.

School Performance Framework: Note first that this is based on 2010 SPF data (which covers the 2009-2010 school year), which is different than the recent 2010 count day data listed at EdNews.  The 2010 SPF lists 18 charter schools and 114 district schools. Using this data, I again adjusted scores based on enrollment (so that each school provides a weighted average in its category). I did not include alternative schools in either group.

Across the entire city, district schools enrolled 67,203 students in 2009-2010, while charters had 6,105 (or 8% of the total). The percentage of students in poverty is very close: 73% to 72% FRL. However, in the aggregate on the SPF, charter schools did considerably better on growth (+8 points); status (+13), reenrollment (+13), student engagement (+17), and parent satisfaction (+12).

But to look even closer, Denver has 68 traditional (K-5) elementary schools and just one charter elementary school, which makes any comparison for K-5 meaningless. What happens if we subtract all 69 of these elementary schools and look again at the aggregate SPF metrics? You get this:

Without elementary schools, the relative performance of charters improves even further.  The percentage of students enrolled in charter schools rises to 14%, and the percentage of FRL students is the same (71%). However charter schools receive higher marks across the board — on quantitative academic criteria (+13 growth, +16 status), on re-enrollment (+15), as well as on the qualitative aspects of student engagement and parent satisfaction (+21 each).  These are remarkable and meaningful differences, and as close to a viable district-wide comparison as I think we can get.

What happens if we continue to drill down into specific school grades, comparing K-8 schools, 6-8 middle schools, 6-12 schools, and 9-12 high schools? Well, of the eight academic criteria, charter schools outperform district schools in seven. Charters also do better in every measured level in both student engagement and parent satisfaction. I won’t cover all levels (you can see the full results in the link above); however let’s look at one particular segment: middle schools.

There are 12 district middle schools, and 5 charters (I included KIPP SP, which is listed as K-8 but only offers grades 5-8). The 14% of middle school students in charters provide a reasonable volume for comparison.  What are the results?

The academic differences are remarkable: +23 points on growth, and +18 on status — and charters have 17% more students in poverty.  Charter middle schools also do far better on student engagement. Two of the five charter schools did not have re-enrollment data, and two also did not have parent satisfaction, so I did not do a comparison for either.  But the schools who did report re-enrollment and parent satisfaction were higher than the district school mean. The academic data here is so strong, it makes me question how much of DPS’s recent middle school success is due to the impact of charters.  I suspect it is considerable.

Charter 6-12 schools also had double-digit scores: growth (+25%), status (+19), student engagement (+19), and parent satisfaction (+39). Reenrollment was lower (-13), but this was based primarily on just one school with a particularly low score (and which is being reconfigured).  FRL was comparable with charters at 58% and district schools at 60%.

K-8 schools also showed higher scores across the board for charters: growth (+8), status (+13), reenrollment (+24), student engagement (+13), and parent satisfaction (+17), however they did so with 11% less FRL students.

Where did charter schools fail to outperform district schools? Only in 6-12 high schools — which had the fewest number of charter students at any level and composed just 4% of the total — and in one category.  Charter schools lagged in growth (-10), but were higher in status (+5), and student engagement (+33), and had an FRL population 15 percentage points higher.

That’s the data, which included results from 2002 to 2010.  Of course, the arguments about charter school performance in Denver is, at many levels, based on political calculations and interest groups who have priorities other than the educational outcomes for students.  Those people and groups will continue their protests regardless (in fact, it would not surprise me if they called for a repeal of the metrics themselves). Which, or course, does not change the data: Denver’s charter’s are doing very, very well.

Popularity: 25% [?]

29 Responses to “The performance of Denver’s charter schools”

  1. Leigh Campbell-Hale says:

    Once again I’m responding to another number-crunching article with the following question: What’s the percentage of union teachers to non-union teachers between regular and charter schools? Of course, all this data also can’t take into account that students, but mostly their parents, choose to send their kids to charter schools. They’re already more engaged.

    • Alexander Ooms says:


      Charter schools employ non-union teachers (with one or two exemptions such as Green Dot in California where it is a different union altogether); traditional schools employ union teachers.

      The self-selection argument deserves more research, but the best data point is the Hoxby study in NYC, which looked at the differences between applicants who were and were not accepted into charter schools via lottery (thus controlling for self-selection) and found a significant difference. I’d welcome a similar study in Denver.

      The other, more simple question on the self-selection argument, given that there are charter schools who do really well and some that do really poorly, is this: how it is that only the good charter schools self-select?

      Lastly, fully 45% of DPS parents now choice out of their school of assignment. If all it took was an affirmative choice, we would see far better school performance across the board. School quality matters, a lot.

      • Kevin Welner says:

        Yes, school quality matters.
        So does research quality. Why, then, would the Hoxby study be cited as the “best data point”? Here’s the summary from professor Reardon’s review:
        “How New York City’s Charter Schools Affect Achievement” estimates the effects on student achievement of attending a New York City charter school rather than a traditional public school and investigates the characteristics of charter schools associated with the most positive effects on achievement. Because the report relies on an inappropriate set of statistical models to analyze the data, however, the results presented appear to overstate the cumulative effect of attending a charter school. In addition, the report does not provide enough technical discussion and detailed description to enable a reader to assess the validity of some aspects of the report’s methodology and results. Policymakers, educators, and parents, therefore, should not rely on these estimates until the authors provide more technical detail and the analysis has undergone rigorous peer review.

        To this, I would add the obvious (which prof Reardon also mentions), these lottery studies DO generally have strong “internal” validity — meaning that they do a good job answering the (test score) question for the schools included. But they often (as is the case with the Hoxby study) have weak external validity — meaning that they can’t tell us much about other schools. This is because only over-enrolled charters (those with lotteries) can be studied. If the basic principle of school choice is correct — that parents choose the best schools and eschew the worst — then measuring only over-enrolled schools will give us a very biased estimate of overall charter school effects.

        • Alexander Ooms says:


          It’s rare indeed to have such a considerable response hang on the use of the word “best.” I believe the Hoxby study is the “best data point” on the issue of self-selection because it is the only one I know that uses for a control group students who applied to and did not receive enrollment in a blind lottery, thus controlling for self-selection.

          If there is a study that you believe is a better data point and uses a superior methodology to control for self-selection, please share it (but my guess is that if you knew of one, you would have).

        • van schoales says:


          As soon as I saw the reference to Hoxby, I knew you’d respond. How about responding to the data presented here in Denver? are the charters in Denver doing better than the district managed schools? What do you think?

  2. Nancy Mitchell says:

    Ms. Campbell-Hale – You posted this question on a recent news story as well and it’s taken some time to get the data and ensure its accuracy. I’ll post it both places:

    Henry Roman, who is the president of the Denver Classroom Teachers Association, said he is not aware of any charter school teachers who are members of the union. The percentage of teachers in all other schools and programs who are members of the union in 2010-11 is 58 percent, according to district records. That’s down from 66 percent in 2003-04.

    Clarification – These percentages refer to all DCTA eligible employees who are active teachers receiving benefits. So part-time teachers are included.

    • Leigh Campbell-Hale says:

      Thanks for getting these union statistics. So, Denver Public Schools is succeeding in busting the teachers’ unions. That’s one of the underlying goals of “reform.” (See today’s article about Michelle Rhee. Her “reform” agenda is no longer even hidden.) Once teachers’ unions are broken, teachers will revert back to their old, underpaid, fired-at-will status. Now there’s a reform.

      • van schoales says:

        I doubt any of the parents looking for good schools care whether the school is unionized or not. Every parent I’ve ever talked to is looking for a good school, including teacher union members who tend to be even more choosy.

      • Alexander Ooms says:

        I’m personally interested in better educational outcomes for kids, so I don’t particularly care one way or another if a teacher is a member of a union or not. But if one believes that the primary purpose of K-12 public eduction is a jobs program for union members, you are probably correct in your disappointment.

        It’s also worth pointing out that teachers is charter schools are not kidnapped and brought there against their will. They choose to teach in charter schools. The suggestion that teachers should be denied that option — much like the people on these pages who insist on telling parents where they should enroll their children — is one with which I disagree.

        As a consistent advocate for higher compensation for teachers, I also disagree with the assumption both that charter teachers are paid less (as with other things, there is greater variety including some charters who pay more, and some who pay less) and that the option of charter schools is somehow bad for the profession (it’s not like the union teachers have a great deal).

        However there are a number of policy inequities between union and non-union teachers in Denver: most prominently that the former receive ProComp dollars funded by taxpayers; the latter do not. If one is truly interested in raising teacher salaries — and not just those of union teachers — this might be a place to start.

  3. jeff says:


    Do you have data on teacher turnover at charter schools? That people choose to go there is certainly a fact. I’m interested to know if they choose to stay and for how long, on average, and how this compares to regular district schools.

    High turnover of professional employees is enormously expensive and it has the secondary effect of making it much more difficult to develop and maintain any kind of positive momentum. Some charters seem to have a high turnover employment model that they have made to work but I stress “seem to have” since this impression is based on anecdotal evidence.

    I don’t know if this is a larger or smaller problem in charters vs. regular district schools in Denver but it’s a major problem in public education in general. I happen to regard teaching as a career rather than something to experience before moving on to something else. As such, I’m concerned with identifying and addressing the underlying causes rather than capitulating to the less than 5 years and out trend.

    As far as “union teachers” go, 42% of whom do not pay dues in DPS, not all of them get ProComp dollars either. The choices we make affect our outcomes. And no teacher of any kind outside of DPS gets ProComp either.

    ProComp was designed/negotiated between DCTA and DPS. Like the single salary schedule, it is a part of our collectively bargained agreement (contract), not a policy level issue. And, as you have pointed out, teachers may choose to work in places where the union holds no sway.

    Charters also have the option to raise money to pay people any way they want which regular schools do not. Is that an inequity?

    In any case, the financial model was based on the number of district employed teachers and the mill levy question that passed provided enough money for that group with an explicit exclusion of all other purposes. To include anyone else would require another mill levy override, additional state money or a new source of funding. None of these seems very likely to materialize any time soon.

    So, yes, we have access to tax payer money that others do not. But it’s the tax payers of Denver County and no place else. We told them how we would use the money and promised not to use it for anything else and they agreed to give it to us. I have a hard time classifying that as an inequity.

    • Alexander Ooms says:

      Hi Jeff,

      As always, a lot of interesting topics in your comment. I don’t have broad data for teacher turnover in Charters, but I would guess if you controlled for age it would not be very different, particularly since in the traditional model something like 50% of teachers last less than five years. It’s hard for me to reconcile the belief that teaching must be a career with the reality on the ground. Surely there is room for improvement here.

      I also disagree that teaching has to be a five-year or more decision. For many people it may be a rewarding career spanning several decades; for others, including some who are excellent teachers, it may not be (Mike Johnson comes to mind). There is considerable difficulty deciding on one’s full professional career at age 25, and I think teachers who are not ready to make that commitment still have a lot to contribute. And in no other profession that I know of would one demand such a requirement or expectation (even the military lets you out after 2-3 years).

      The additional expense depends a lot on what you measure. If our goal is student outcomes, and one hires two teachers over five years who produce a 25% improvement in academic achievement, it may be well worth the investment. If we continue to judge education purely as a cost center, any marginal spending will not be worth it (PD and others included). And again, with a 50% churn rate, it is not as if we have a successful system currently.

      ProComp is an interesting issue because it is separately funded and approved by taxpayers (not just the contracted participants). Taxpayers did not vote on the rest of the collective bargaining agreement (though it would be pretty interesting if that is what you advocate). DPS and DCTA are free to contract and self-fund whatever programs they want, but that is not the case here. If you asked taxpayers if they intended to fund a policy under which two teachers, getting the same results with the same student population in the same neighborhood would be treated differently based on the type of school where they taught, I would bet they would not see the distinction. The applies equally to other funded programs through mechanisms like mill levies and bonds.

      And ProComp works very differently than DPSPRS and PERA, which charter schools are required to support. If you don’t want to share taxpayer-funded ProComp, why insist that charter schools pay into the DPS/PERA pension system? The history here is to require pension dollars in with one hand, yet refuse ProComp dollars out with the other.

      I don’t agree with your financial assessment, particularly since initially ProComp could not spend a sufficient amount of its fund. I’d be interested in an update, but I don’t buy for one second the argument that charter teachers should not participate because the ProComp fund is impoverished. Nor do I believe that question should be left up to the interested party of DCTA.

      If you can show me where, in the copious amounts of material devoted to passing ProComp, taxpayers explicitly were told that it would only cover union teachers, and not charter school teachers, I’d be grateful. I would bet that most people who voted had no idea. In any case, I would certainly be open to going back to the public, and being much more explicit about the differences, and asking them to affirm the initial decision. Would you support that with me?

      Again, it is hard to me to reconcile the views of people who claim to be “pro-teacher” with a taxpayer-funded compensation policy that excludes non-union teachers. Why, in all honesty, would any DCTA teacher believe that their peer down the street is less deserving of a taxpayer-funded initiative. If we are serious about wanting teachers to be better compensated, why would one not extend that to all Denver public school teachers?

      • jeff says:

        First, the most misleading of your questions – I have not here or anywhere written that I believe my “peer down the street is less deserving of a taxpayer-funded initiative” or anything even remotely similar to that. I’m saying that the system we’ve got cannot afford to accommodate charter school teachers. You don’t want people twisting your words or misusing the facts so don’t do it yourself.

        The reality on the ground is that we’re not doing the job we need to be doing. Apparently, you do not see the value in continuity that I do so, as usual, I will agree to disagree with you. Much of what we have decided to do at Denver Green School is to encourage longer term (if not career-long) commitments to teaching. We’ll see how that works out.

        I’m the last person to say one should not change careers since I’ve done it myself. However, I had what I understood as a career and decided it was a poor fit. I did not become an engineer to have something to do while I figured out what I really wanted to do. In my opinion, preparing our young for the future they will face is not the place for adults who don’t really know what they want to do. Can such people add value? Of course. Are they contributing to the solution of the larger scale problem? I do not believe so. In fact, I think they’re making it worse. Again, I suspect we will disagree.

        Taxpayers don’t vote on any part of the contract, only the means to fund some of them. Their elected officials (the school board) do, however ratify every provision of the contract. Since governance in the U.S. is based on representative rather than direct democracy, I find that situation satisfactory. Replace the school board with the mayor and I’m still OK with it.

        Two teachers getting the same results with the same population in the same neighborhood can and are treated differently because not everyone is required to participate in ProComp and that was made abundantly clear at the time of the election.

        What was also made clear, which the A+ commission either didn’t hear, understand or chose to ignore, it the fact that we expected surpluses in ProComp during the early years because the system was balanced toward salary rather than bonuses. I don’t have printed material in front of me but I made many of the presentations myself.

        Based on the financial modeling we had, we believed, and clearly told the public, that those surpluses were necessary to allow the system to reach equilibrium as more people opted-in or were hired in and the general level of salaries went up. We expected overruns followed by a tapering off to current mill levy income as the bubble of highly paid veteran teachers began to retire. In the long run, the model showed there would be no surpluses.

        In the context of questions about how DPS had spent previous mill levy and bond money, we worked hard to give the public something accountable that we believed would last. And since the system was flipped to favor bonuses rather than salary, we will never know if the model was actually predictive or not, though many have already speculated one way or another. We acted in good faith on the best information we had at the time.

        ProComp must be seen as a test case to show how pay for performance can work. If you want a larger scale problem solved, why on earth would you look to a union local to do it? Taking your line of reasoning, why should a DPS teacher be paid differently than one doing the same job in eastern JeffCo, Aurora, Cherry Creek north of the reservoir or some of the Adams County districts? It is not DCTA’s business to bargain for charter teachers any more than it is to bargain for teachers in other districts. Charter teachers have chosen not to have representation, and so they do not get it. I’m pretty sure we’re all OK with that idea.

        We got what we could and it can do what it does. If you want it to pay more people, you need to get more money. I frankly find it disingenuous to complain that the system does not do something it does not have the resources to do. If you decide to mount a new mill levy campaign to get ProComp for charter teachers, go right ahead. I will happily endorse it.

        btw- No where did I say or even suggest the ProComp Trust is impoverished, though if we applied it to all of the charter teachers in schools approved by DPS, it quickly would be.

        Not I nor DCTA nor DPS requires anyone to pay into PERA, the law does. I couldn’t care less if charter teachers pay into it or not. Let them pay social security like everyone else. I have no idea what that has to do with anything.

        And, if you insist on an explicit statement that ProComp applies to DPS employed teachers (which you continue to call “union teachers” even though nearly half choose not to join a union) and not to charter employed teachers, you will win. However, in press releases and news items, ProComp was described as collectively bargained and as ratified by DCTA members. And everyone knows that charter school teachers do not belong to unions. Most people then saw that as exactly the point of charter schools. Since I don’t believe the tax paying public is stupid, I assume most people could connect the dots. The fact is, this was not an issue 6 years ago.

        • Alexander Ooms says:


          I do agree that in many place we can simply agree to disagree, but I’ll cover a few areas:

          1. When you say that two public school teachers, both in the same district, should have different policies on a taxpayer funded program, of course you are saying one is less deserving than the other. On what other basis would you argue? That they are equally deserving?

          To make this absurdly simple, lets take one of the co-located buildings and imagine the same teacher, teaching the same subject and the same grade, who switches from a traditional school to a charter school. Why exactly should she be denied the opportunity to continue to participate in ProComp just because she teaches in a different part of the building?

          2. In my mind, quality trumps continuity. I would rather have three high-quality teachers over 10 years — even at a higher cost — than one average teacher.

          3. There is a substantial difference between a teacher who does not participate in ProComp by choice, and one who is not given the opportunity to participate.

          4. It is not that a DPS teacher should be paid similarly to one in JeffCo, it is that two DPS teachers should both get to participate in a program funded by DPS taxpayers. Similarly, two JeffCo teachers should both get to participate in a program funded by JeffCo taxpayers. The entire point is that it is within the same district.

          5. ProComp was based on an assumption on the number of participating teachers, which I assume was based on a demographic projection of a number of enrolled kids. Assuming this projection was pretty accurate, it should make no difference if these are DCTA teachers or non-DCTA teachers. If anything, every kid who either does not drop out, or who attends a charter instead of a private/secular school is adding enrollment and revenue to the district. The claim that the system would be injured if extended to charters is just not credible.

          6. No one expects DCTA to bargain for charter teachers, but I don’t believe it is unreasonable to expect them not to exclude charter teachers from taxpayer funds. What would your reaction be to a similar public initiative that rewarded charter teachers and not DCTA teachers? Would that be considered fair?

          7. Yes, I do refer to district teachers as union teachers since even if they choose not to pay dues they are still subject to all of the provisions of the collective bargaining agreement.

          Schools where teachers can participate in ProComp include a) traditional schools; b) magnet schools; c) innovation schools (which waiver out of many CBA provisions), and to your other point you can participate in ProComp without being a dues-paying member. There is only one thing you cannot do, which is to teach at a charter school. ProComp was intended to help public education, not only certain segments of public education. Would you deny students in charter schools any taxpayer funded initiatives? if not, why not?

          8. I think we’ll agree to differ on what sort of information was given to the public around ProComp and what people understood. I do agree with you that it was less of an issue (there were far less charter schools). But times change, and so should basic equitable policies.

          The initial post here centered around the high performance of charter schools, which is primarily the result of a lot of hard work and a deep commitment by many outstanding teachers in those schools. It’s time those efforts were recognized and compensated similarly to all other outstanding teachers in Denver’s public schools, instead of being treated like second-class teaching citizens.

          • jeff says:


            Some charter teachers also do not to get to participate in the minimum salary established by our contract. Is that OK with you? Using your reasoning, according to the very schools that employ them, these teachers are less deserving of tax payer funded income. Why don’t you have a problem with that?

            You cannot have it both ways. Argue for equitable pay or do not. If I recall, ProComp funds amount to 10% or less of total teacher payroll so it seems like making sure these folks get their fair share of the other 90% would be of some interest to a person who is really concerned about equity.

            Student enrollment does not add revenue to ProComp because it is funded out of a mill levy override on property taxes, not through the school funding formula from the state. I’m pretty sure you understand how property taxes work so I’m not at all sure how to interpret your claim that “every kid who either does not drop out, or who attends a charter instead of a private/secular school is adding enrollment and revenue to the district.” Per pupil funding is never spent on ProComp payments ever. That would violate the contract.

            The ProComp mill levy override got the Trust $25 million the year after it passed and it gets inflation adjusted each year. CPI has been low and last year it was negative. The number of kids enrolled is totally irrelevant to how much money the ProComp Trust takes in and in a bad economy, property tax collection rates go down so the funding stream can be further constrained.

            Paying more people out of a pot of money that is fixed in terms of real dollars will crash the system. Who would that serve?

            So, as I said above, find another source of on-going revenue or mount a campaign for a new mill levy override to fund a wider application of ProComp and I will happily endorse it.

            Until then, if you’re really concerned about equity, you should work on getting all charter teachers paid at least as much as DPS teachers get.

  4. Kevin Welner says:

    Alex, do you recall when the CREDO report came out, showing that only 17% of the national sample of charters outperformed matched conventional public schools in math, while 37% posted math gains that were significantly below what students likely would have seen if they had enrolled in a conventional public? It’s the report you cite in your entry above (except that 17% translates to approx 1 in 6, not 1 in 5 – and you neglected to mention the 37% part).
    Anyway, when that report came out, we asked Prof. Gary Miron to review it, and he found that while it was a solid study, it had clear limitations as well ( The study’s lead author, Macke Raymond, has also stressed that her study shouldn’t be put on a pedestal – that it needs to be understood as a single data point, as part of the larger research base.
    That’s the nature of these things, and that’s the answer to your question.
    Now, the CREDO study is much better executed than is Hoxby’s, and it’s much more useful in terms of external validity. But it too should be considered only as part of the overall picture.
    Another note: As professor Reardon wrote in his review of the Hoxby report, even if all the flaws were corrected, it appears likely that the NYC charters are doing well. The same may very well be true of the Denver area charters or the Colorado charters. I’ve always believed that at the level of individual schools, excellence can be found in any sector – independent private, archdiocese schools, charters, neighborhood publics, homeschools, etc.
    Regarding other studies, the Dept of Education/Mathematica study from this past April (I believe) which also uses lotteries and which covers schools across the nation, should certainly be considered an important data point:
    Even that one, however, should not be read in isolation. It’s the overall body of research that we should look at. And that body of research is very, very clear and very consistent with the outcome of the Mathematica study. There is large variation within the charter sector (as within other sectors), but, “On average, charter middle schools [in the case of this study] that hold lotteries are neither more nor less successful than traditional public schools in improving student achievement, behavior, and school progress.” Even moreso than the more than 2:1 advantage shown in the CREDO study (advantage for noncharters), I think this is representative of the overall research: if we’re merely comparing sectors, don’t expect to see any significant difference.
    Anyway, I’m so glad I didn’t disappoint Van!

    • van schoales says:


      You did not disappoint, in fact your responses are remarkably predictable, I wish it weren’t so. I agree with much of what you say about the general research on charter schools but that does not change the fact that charters in Denver, NY and Chicago are mostly doing better than other schools, not true in states like AZ, OH and TX (again I don’t care whether they are charter or not, I want better schools).

      My problem is that you and your advocacy organization regularly advocate against charters, value-added assessment, choice, testing, accountability, school finance reforms, etc using your status as a university-based research professors. As I’ve said here before, you and your organization’s writings should always be taken with a grain of salt because of your personal perspective and support from the National Education Association.

      It’s obvious to anyone that follows ed reform that your choice of projects, researchers and subjects is designed to advocate against the current wave of reform. Paul Peterson is the counter example advocating for vouchers from Harvard. Your publications should not be considered as comparable as reports from NIH, NRC or many other legitimate research organizations. You are an advocate against the school reforms currently being practiced and are using “research” to slow or kill it.

      • Kevin Welner says:

        Sadly, Van, I really do feel like I’m saying the same thing here, over and over again. And I do wonder whether it’s worthwhile.

        Perhaps it would be more interesting if I chimed in every once on a while with praise for a low-quality report, just because it advocates for one of these faddish reforms. It’d be fun to say, “Sure, the research is crappy. But the findings are really political popular, so I’ll get behind it as well. Maybe then Van’ll say something nice about me.”

        NEPC “advocates” for bringing the mainstream research base — the highest quality research — to a broader audience. We also value very much the role public schools play in a democratic society.

        If you look at research from groups like NAS, NAEd, NIH etc, (groups you seem to think highly of — and should) you’ll find that they are saying much the same things we are. Take a look, for instance, at and and

        Anyway, I can say that I personally am a strong supporter of school finance reform as well as accountability reforms (although I’m not sure I understand those the same way you might).

      • Alexander Ooms says:

        Hi Kevin,

        I assume your lead question is rhetorical as I link directly to the CREDO report above, but I appreciate your ability to live up to the final sentence in that paragraph.

        I will confess that my powers of recall with your comments are helped by a simple heuristic: you do not believe any study on an education reform policy is definitive, and thus you do not support any specific change in the way districts or schools are run, which means things stay pretty much the same. Nothing in the above contradicts that formula.

        I do think the relevant question is if charter schools are dong better in districts (note that is explicitly not at the individual school level) where there is a consistent and substantive authorizing and review process. I think it is too early to say, but clearly results in Denver, NYC and other cities mean that there is enough evidence to make that a question worth pursuing.

        • Kevin Welner says:

          I think your heuristic needs a bit of work. I do strongly believe that our schools — particularly those serving low-income communities of color — need improvement. Since you seem to be so concerned and interested in my perspective and work, I am mystified as to how you can’t see that. Many of my publications are available on my CU website.

          Let me try one more time: Yes, I “believe” that no single study is definitive. (I put “believe” in quotes because it’s like saying that I “believe” in gravity. No legitimate scholar in the social sciences would argue otherwise.) But it does NOT follow that I don’t “believe” that a body of research can support change. I very much do. What you and Van seem to have a problem with is that the body of research simply does not support all these reforms you’re so enamored with.

          Anyway, ending on a positive note, I would definitely support research that looks more closely at places where charters as a group seem to be doing better. I have heard the plausible argument that states that are more careful about authorizing and reauthorizing have better results. That seems worth exploring.

          • Alexander Ooms says:

            Unfortunately, my heuristic is based on typed conversations going back now eight months, (such as the long comment string here: and even in the above it holds true: With the single exception of detracking, you do not support any specific change in the way districts or schools are run, which means things stay pretty much the same. The overwhelming majority (if not all) of your publications are in opposition to policies, not in support. I am in turn mystified how this pattern can repeat itself at the same time you deny its existence. If you support specific changes, list them. If you will not, stop claiming that you support specific changes.

            I don’t question your belief that schools need improvement. It’s easy to say what one believes, it’s a lot harder to say what one should then do. Part of this is our respective comfort level with attempting new things: I believe the problems in K-12 public education are severe enough to warrant trying a number of different approaches – even if some fail – in the belief that one can build on those that succeed. You believe that superintendents who try new things are “promoting unproven gimmicks as solutions” and should all be fired (

            What’s most remarkable to me about a post that now features over 20 comments, is that no one seems to question the basic data that Denver’s charter schools are outperforming its traditional schools by a substantial amount. These schools were not started by educators and teachers waiting for a body of research to tell them they were right. They were started by people who believed that doing something different would help improve educational outcomes for kids. Thankfully, they are waiting neither for superman nor for definitive academic proof.

  5. van schoales says:

    Kevin, While I do think there is lots of worthwhile stuff that comes from the reports you provide. The NAS is hardly the NIH, it is run by education school leaders with one of the directors being a partner of yours, David Berliner who is focused as you on discrediting any reform that involves more student choice, accountability or for that matter that is not supported by the teacher unions. This is one of the problems in the field of education research.

    I’m curious to know whether you think things have gotten better, remained the same or maybe worse for kids inn Denver and New York since 2000.

    For the record, I believe that they have mainly gotten better as measured by retention, achievement and graduation data. Not to mention the fact that many areas now have good schools where none existed (bronx, west denver, etc).

    • van schoales says:

      Sorry I meant to add that we have a long way to go in NYC and Denver when so few kids are reading or prepared to go to college. While better than ever, the current trends are not acceptable (it will take 20-30 years to get to a place where the vast majority of kids are getting a quality education). Because of this even more radical changes are required, i.e. teacher evaluation, compensation, new school replication, parent education, accountability for student learning, etc

  6. John Middleton says:

    Statistics aren’t everything. If you’ve seen The Lottery, you know how important these charter schools are. In NYC, you can see that these charter schools are a great improvement over public education. The families in this documentary don’t just want their children to go to charter schools, they need them to get in. The improvement in their lives is extreme, showing how sad it is when the students don’t get in.

  7. Kevin Welner says:

    I am admitting defeat and signing off this blog. I can’t keep up, nor do I really want to. Ironically, it was just a month ago that I sent Sabrina a note, praising her for her stamina — but now she’s exhausted by all this as well.

    I guess I should explain a bit. Not keeping up with a normal discussion wouldn’t be a problem — it could go on in a healthy way nonetheless. But what seems to happen here is more accuse/defend, so when I do engage it obliges me to repeatedly dive back in to ‘clear my name’. It’s not interesting, and it’s not fun.

    The most recent (above) includes Alex asserting, “With the single exception of detracking, you do not support any specific change in the way districts or schools are run, which means things stay pretty much the same.” Yes, my research has certainly included a great deal about detracking — maybe half of my work has focused on this (including my dissertation and first two books). But I “support” many other reforms, meaning that the reforms (policies and practices) have been shown by high-quality research to have positive outcomes. E.g., equitable funding (see, small schools (, various supports and interventions to reduce dropouts (, school integration ( and, and various legal issues, such as educator first amendment rights ( Of course, my job often involves understanding and explaining more than taking positives in favor or against a given policy. A large body of my research, for instance, simply tries to understand/explain the equity-focused reform process. But even within those constraints, I’ve written about quite a few reform/policy ideas in a positive way. So while i have indeed focused a great deal of energy and time debunking poor research, I have gone well beyond that. And, of course, I support policies that I haven’t personally researched that are nonetheless backed by solid research (e.g., high-quality early childhood education).

    So, that’s Alex. But Van will surely be pleased to know that he is similarly exhausting. Above, I had responded to an earlier comment/question by pointing out that “If you look at research from groups like NAS, NAEd, NIH etc, (groups you [Van] seem to think highly of — and should) you’ll find that they are saying much the same things we are. Take a look, for instance, at and and

    In response, Van writes, “The NAS is hardly the NIH, it is run by education school leaders with one of the directors being a partner of yours, David Berliner who is focused as you on discrediting any reform that involves more student choice, accountability or for that matter that is not supported by the teacher unions. This is one of the problems in the field of education research.”

    Where to begin?
    1. David is a member and director of the prestigious National Academy of Education (NAEd), which does include scholars in education schools, as well as elsewhere. If I understand the above note correctly, the accusation is that the NAEd is therefore not credible. Sigh.
    2. The NAEd is different from the National Academies of Sciences (NAS). The two are different. Not the same. Different.
    3. The first two links I provided are to NAS publications, the third is to a NAEd publication.
    4. More of the “teachers unions” Tourette syndrome? It reminds me of the Biden quip about Giuliani: “a noun, a verb, and 9/11.” I’m not going to bother defending David — he doesn’t need it — but it is pathetic to minimize the work of such an amazing scholar to a charge that he for some reason follows the Svengali directions of the all-powerful unions.

    I should note that I have known Van for many years, and we’ve had many interesting and worthwhile conversations during that time. I also met Alex once (in person) and found him pleasant and earnest. Maybe it’s something about the blog venue that makes us else substantive and less civil?

    Anyway, I wish everyone here the best. I will continue writing and speaking about important policy issues. If interested, folks can find my work, among other places, at:

    The one thing we all do seem to agree on is that schools and policies need to improve for our most vulnerable communities. It’s sad that we can’t do that more collaboratively and with more agreement about how to find that path.

    • Alexander Ooms says:

      I regret Kevin’s decision to withdraw, but as I read back over the thread of our exchanges I do wonder if it is not related at least as much to the difficulty in defending his position as much as a claim of incivility. Others can judge for themselves.

      For, at the risk of repeating myself, and in the bullseye of our differences, I’ll point out that supporting “reforms” absolutely no one opposes does not in any way qualify as policy, nor does it support any specific change in the way districts or schools are run.

      So, if there is anyone out there who favors any of: inequitable funding, large schools, increasing the dropout rate, more school segregation, limiting educator speech, or abolishing high-quality early childhood education, Kevin will be a worthy adversary. If one truly believes this sort of abstract, unopposed path is the way to better schools — well, ok, but to me it looks awfully hollow.

      The absence of additional voices is a shame for this blog and the greater education debate, and I hope Kevin changes his mind. I’ve rarely seen anything improve by people walking away, but with his bat and his ball, he gets to decide when to go home.

  8. Ed Augden says:

    Regardless of all the statistics that charter schools may, or may not, perform better than traditional schools, the obvious fact remains that charter schools serve only 10% of students in Denver Public Schools, that they can choose students and that traditional schools do not have that choice, that most ethnic minorities and the poor remain in traditional schools, that most special needs students remain in traditional schools essentially excluded from or discouraged from attending charter schools and that they are union busters since they exclusively employ nonunion teachers, with few exceptions. The “reform” that charter schools bring is high stakes testing administered by authoritarian, corporate educators. Testing is, indeed, a useful measurement for about 30% of a student’s evaluation. Beyond that, testing is less valuable than students learning research skills. Charter schools perpetuate, as do tradtional schools, a 20th century paradigm of authoritarian public school governance and do little, or nothing, to eradicate inequity. Charter schools serve the privileged and the lucky and ignore the many.

    • Alexander Ooms says:

      It’s rare to have someone admit so freely upfront that they could care less about both data and student performance, but Mr. Augden does not disappoint, for neither are as important to him as his ideology.

      Aside from being directionally correct on the percentage of charter students in DPS, the entire rest of his piece is a series of utter falsehoods, as was made abundantly clear recently (

      This has been pointed out to Mr. Augden ad nauseam, who is apparently so indifferent to his pants being on fire that he is content to continue to fan their flames.

  9. Alexander Ooms says:


    I think we are in danger of becoming two arguments passing in the night, but improving teacher compensation is an important issue to me (and I believe to you as well), and while I doubt we’ll see eye-to-eye, it’s worth it to me to unpeel where it is we actually disagree. So let me take a step back and try again.

    1. Can we agree in principal that charter and district schools should be funded equitably by taxpayer monies? This is obviously a difficult question when it comes to accounting for specific issues like transportation, facilities, center-based programs etc., but all other things being equal, there should be public funding parity, agreed?

    2. Charter schools receive waivers which give them greater control over their budgets (which consist largely of these public monies). So if a charter school wants to allocate either a greater or smaller percentage of their budgets to teacher compensation, and the BOE approves these waivers and budgets, they have the autonomy to do that, agreed?

    3. Accepting both of these, my argument is this simple: if there are public monies set aside for teacher compensation based on specific criteria, that money should flow equitably to all public schools that meet the criteria. Charter schools should retain the autonomy to allocate their budgets as they prefer, but should then follow the same (or highly similar) criteria as district schools in the assignment of ProComp dollars (in much the same way they follow Tabor, PERA or similar restrictions).

    I think you disagree with this on one or both of the following grounds: #1 is not true and there should not be parity regarding public money; #2 is not true and if a charter school takes ProComp funds they have to relinquish their autonomy over teacher compensation and pay under the exact same formula as the district schools.

    Any disagreement over the respective solvency and specific calculation of ProComp funds, the number of participating teachers, or similar logistical issues is secondary to the above. I guess you could argue that you favor including charter teachers, but ProComp as designed won’t allow it financially, but I don’t think that is your point (although maybe it is since you argue that if another mill levy could be added you would support extending funds to charter schools). But for me, you either have to disagree with 1 or 2 above. If you think I am laying it out incorrectly, I’m interested in a similar calculation from your point of view.

    Incidentally (as I think you know), I’m a strong proponent of both better and different pay systems for all public teachers, I absolutely do work on getting better pay for charter school teachers, and in my small sphere of influence at a charter network, the teachers absolutely receive higher compensation than they would on the base DCTA salary schedule.

    Truth here is that I don’t think our argument matters much except to us; my sense is that at some point this issue will go back to the taxpayers with much more definition around it, and they will decide.

  10. jeff says:

    I plan to take leave of this argument and, in fact, of this blog for some time because I really cannot stand the tone that has developed. I logged on to say that I made an incorrect statement when I wrote that per pupil funds are never spent on ProComp payments. In fact, per pupil funding does pay a teacher’s salary up to what s/he would have made under the single salary schedule which includes pay resulting from ProComp elements in place of steps and lanes. The ProComp Trust reimburses the district for the difference between that and the actual salary.

    I cannot imagine why you would think I would disagree with any of your three points. You go on to write, “I guess you could argue that you favor including charter teachers, but ProComp as designed won’t allow it financially, but I don’t think that is your point.”

    That is exactly my point and since I said it almost exactly that clearly twice, I’m not sure how you missed it. I don’t honestly care what the explanation is because I’m done with this crap. You’re welcome to think I’m a unionista who wants to deny something to charter teachers (like my wife) or a cry baby who is taking my ball and going home. I don’t care about that either.

    Happy Holidays.

Leave a Reply

Colorado Health Foundation Walton Family Foundation Daniels fund Pitton Foundations Donnell-Kay Foundation