You are viewing the EdNews Blog archives.
These archives contain blog posts from before June 7, 2011
Click here to view the new Voices section of EdNews

Archive for the ‘International comparisons’ Category

Challenging the myths of international comparisons

Thursday, March 3rd, 2011

Paul Teske is Dean and University of Colorado Distinguished Professor at the School of Public Affairs at the University of Colorado Denver.

(These views represent the personal opinions of the author and may not reflect the position of the University of Colorado Denver or the University of Colorado system).

The Brown Center report on American Education 2010 from Brookings was released in February, with some very interesting new data and analysis by Tom Loveless, probably America’s top scholar of mathematics scores.

The idea of American performance decline over time is not as true as people, especially reformers, seem to believe – indeed, it may not be true at all.

First, Loveless examines comparative international test results, and shows that the relative performance of American students has actually improved over the years.  Comparing recent tests with those first given in 1964, seven years after Sputnik, at a time when many baby boomers were educated, Loveless shows that American students never scored very well (debunking the myth of some past American dominance here) and that our students have actually gotten somewhat better over time, compared to other countries.

He also shows that the much admired Finland actually only scores well on one particular type of test, and notes that Chinese and Indian students have yet to take most of these tests, so their supposed rising “dominance” is largely myth, at least in terms of what we know from these tests.

It is always fascinating to me to hear how many people implicitly believe in a past “Golden Age” of American education, when we were world-wide #1 and our systems were so much better than now.  In addition to the data in this report, that idea is debunked very impressively, in a nice short read, in Richard Rothstein’s 1998 book “The Way We Were?”

One should retain the usual concerns of course – our system today has lots of problems, we need to do much better, especially with low income students – but this suggests that the idea of American performance decline over time is not as true as people, especially reformers, seem to believe – indeed, it may not be true at all.

Brookings also examines aggregate state-by-state NAEP performance, with two econometric models – one that looks at 2009 NAEP scores compared to how the state did on NAEP in the 1980s or 1990s, when NAEP was spreading across the American states, and another that looks at relative state performance in the recent period of 2003-2009.

While the focus is whether the right states won “Race to the Top” (some did, some didn’t), EdNews readers will be interested to see that Colorado ranks 18th of the 50 state on long-term performance improvement, but only 30th for the period 2003-9, with an actual decline in scores in the past six years, when adjusted for the demographics of students.

Finally, Brookings compared NAEP test items with the proposed common core standards, and finds that the NAEP questions are not well aligned and are actually too easy, compared to the standards.  Loveless suggests that this lack of alignment will further confuse Americans about performance, going forward.

The report itself summarizes the overall findings:

“An overarching theme of this year’s report is that events in the field of education are not always as they appear to be—and especially so with test scores. Whether commentators perpetrating myths of international testing, states winning races while evidencing only mediocre progress, or an eighth grade test dominated by content below the eighth grade, the story is rarely as simple as it appears on first blush. This report tried to dig beneath the surface and uncover some of the complexities of these important issues.”

Popularity: 20% [?]

From the publisher: Practice, practice, practice

Wednesday, January 19th, 2011

Which of these two approaches do you believe generates better results, whatever the endeavor?

  1. Constantly shifting gears; placing a high value on trying new programs and strategies without necessarily waiting to evaluate the results of what you have initiated before moving on to the next new thing.
  2. Deciding on a course of action and then sticking to it faithfully for an extended period of time, even when evidence surfaces suggesting that there may be better strategies and newer, more innovative techniques that could yield better results.

In the real world, of course, one needn’t choose one or the other. But I would argue that public education systems in this country have most often and by default chosen something resembling the first approach over the second. The frenetic pace of new mandates and initiatives have made it almost impossible for educators to settle in and get good at implementing any particular series of changes.

As state Sen. Mike Johnston wrote on this blog last week, Colorado finds itself in a place today where slowing down and implementing the state’s abundance of new initiatives would be the wisest course of action:

With Colorado in the middle of rolling out new standards, developing new state assessments that will replace CSAP, and overhauling our principal and teacher evaluation system, a number of the most critical components of our statewide system are in flux.

This summer I had the opportunity to talk with more than 1,000 teachers and more than 70 superintendents and their consistent message was that they are committed to getting standards, assessments and evaluations done right, but they need the time to do that before embarking on another big initiative.

In the following paragraphs, or course, Johnston said yes, but we must also address “student accountability” and the method by which we count enrollment in districts. So don’t expect stagnation at the State House.

I keep thinking back to a trip I took to Mexico in March 2004, when I worked at the Piton Foundation. Educator Rob Stein, community advocate and Mexican national Jaime Di Paulo and I spent a week in the state of Zacatecas visiting schools. (We also spent an enjoyable evening following around  a burro that dispensed shots of mescal, but that’s a story for another venue).

We wanted to see what we could learn about Mexican education at the classroom level, and whether any of those lessons might be useful to educators in Colorado, who were working with growing numbers of Mexican immigrant students.

You can read a full account of the trip here, the highlight of which is Stein’s insightful analysis of Mexican classrooms.

Mexico is not known for its stellar public education system. In the latest (2009) Programme for International Student Assessment (PISA) results, Mexico ranked in the bottom 20 percent of countries in reading, mathematics and science, behind the underperforming U.S. in all areas.

And yet the Mexican teachers with whom we spoke almost to a person observed that students who came to them after being schooled for a time in the U.S. consistently lagged behind their counterparts who had been educated solely in Mexico. This was especially true in math, the teachers said.

And in the hamlet of El Tepetate, where cars with Colorado plates are a common site, we talked to a couple of students from Denver (sent home to live with grandparents after misbehaving in the U.S.) who said they had to struggle to catch up academically to their Mexican-educated peers.

As Stein observes in his piece, Mexican education is old school. We visited about a dozen schools and everywhere the methodology was identical. It probably hasn’t changed since 2004. The teacher stands in front of a class of uniformed students in an unadorned, cinderblock and cement classroom and delivers lessons.

Stein wrote:

In urban and rural schools alike, the curriculum follows an almost lock-step adherence to the prescribed series of subjects, materials and textbooks issued from Mexico City. Each state teaches its own history, which is the only curricular area in which one might find differences in material taught.

In classrooms around the state of Zacatecas, on any given day, students of the same age and grade can be seen studying the same chapter, from the same textbook, sometimes at the same time of day. The full course of studies includes mathematics; science; history and social studies; reading and writing, which give way to literature in the older grades; physical education; and English.

It doesn’t sound exciting or stimulating, does it? It’s not exactly Expeditionary Learning. Yet I  came away with the impression – and I think Stein and Di Paulo shared it — that the Mexican education system in many ways does a better job fulfilling its mission – educating students through ninth grade – than some U.S. schools do fulfilling theirs – preparing all students for post-secondary training or education.

As Stein wrote, it’s as if “the floor is raised but the ceiling is lowered at the same time.”

So despite the glaring deficiencies, I left Mexico respectful of its education system. Let me be clear: I know Mexican schools are not what we want to emulate in this country in most ways. Many Mexican kids arrive in the U.S. with weak language skills – in Spanish as well as English, making it hard for them to catch up. But it’s not clear to what extent we can blame Mexican schools for this deficiency. What I’ve been told is that may of those lagging students come from rural areas and may have attended school only sporadically if at all.

Stein wrote:

Students seem highly engaged in Mexican classrooms, probably more so than in the United States. Learning seems more purposeful on average, with fewer interruptions and clearer agreements about the process of schooling. The level of learning, especially in basic literacy and numeracy, rivals that in the United States.

There is general agreement among teachers, administrators and parents that schools, in spite of their lack of books and technology, are working pretty well. Parents do not question the role of the teacher and school in giving their children a fundamental education. When asked, educators will respond that Mexican schools rival those in the United States.

Mexico is a clear case of a country that employs the second of the two strategies I described at the outset. Do we want to emulate Mexico? Probably not. Does the doggedness of Mexico’s approach to education offer us some valuable lessons? Probably so.

It’s essential that we have charters and other autonomous schools that are innovating and trying new strategies and methods. In the end, those schools will probably help us get to the next level. Meanwhile, though, it might make sense for the bulk of our public schools to slow down and get good at what they’re doing rather than endlessly questing after the next great idea.

Popularity: 18% [?]

Leaning tower of PISA

Tuesday, December 7th, 2010

As I write this, 2009 results of the Programme for International Student Assessment, an internationally standardized triennial test of 15-year-old students’ performance in reading, math and science have just been released.

Here is some language from a news release issued by several national education organizations:

“15-year-old students in the United States continue to rank at average or below average in international comparisons of reading, math, and science.

“The bottom line of the 2009 results is: The U.S. ranked 14 out of 65 countries in reading, virtually the same ranking as the 2003 test; 17th in science, which is an improvement from 21st in 2006; and 25th in mathematics, the same ranking as 2006. The good news is that U.S. students, especially those with the lowest performance, have significantly improved in science since 2006.”

Like so many education matters these days, your reaction to these numbers depends in large part on your educational ideology.

On one side, we will hear people proclaiming that the sky continues to fall. In a new analysis of 2006 PISA scores that appeared in the current issue of The Atlantic , Stanford economist Eric Hanushek said when it comes to international comparisons, the news for the U.S. is all bad:

Even…relatively privileged students do not compete favorably with average students in other well-off countries. On a percentage basis, New York state has fewer high performers among white kids than Poland has among kids overall. In Illinois, the percentage of kids with a college-educated parent who are highly skilled at math is lower than the percentage of such kids among all students in Iceland, France, Estonia, and Sweden.

In other words, Hanushek says, our education system doesn’t just fail low-income kids of color in urban districts. It fails everyone. Expect more handwringing of this sort later today.

On the other side, we will hear people who decry standardized testing as a soul-killing and one-dimensional measurement tool that tells us almost nothing useful. And international comparisons are especially pernicious, they say. In her Washington Post blog, Valerie Strauss quotes the late, lamented Gerald Bracey:

But really, does the fate of the nation rest on how (kids) bubble in answer sheets? I don’t think so. Neither does British economist, S. J. Prais. We look at the test scores and worry about the nation’s economic performance. Prais looks at the economic performance and worries about the validity of the test scores: “That the United States, the world’s top economic performing country, was found to have school attainments that are only middling casts fundamental doubts about the value and approach of these [international assessments].”

How politicized has this debate become in a politicized and polarized era? Here’s a brief sampling of comments that followed Hanushek’s article on The Atlantic website:

Reporters need to stop taking Hanushek’s (quiet, gentle) word for it and actually question his logic. They will find it to be quite ideological, and not bound by his data. There is a lot of sophisticated hand waving, but it masks an ideological agenda not based on the data. A simple understanding of what an effect size is, what portion of variance explained means, and basic economic (and psychological) research methods would help journalists be more skeptical of listening to “that guy you go to for What’s the other side of the story?” This sort of false equivalency is what makes many scientists hate the majority of science reporting, and social science reporting (which is what this is) is no exception.

And:

It is also worth noting that STANDARDIZED TESTS have been showing these discrepancies for THREE DECADES, since we were proclaimed “A Nation at Risk” as a result, in the report of that title. And yet the generation raised in those schools has ACED “the real test,” further extending America’s lead in virtually every area of human endeavor. To me, this discrepancy between test results and REAL-WORLD results suggests something completely different–that we have not yet figured out how to measure WHAT REALLY MATTERS in an information-based society. Until we do, the results of standardized tests whose results do not correspond to real world performance must be taken with a LARGE grain of salt…

Then there is this:

Math education at the grammar school/middle school level in the US is mostly about time wasting. Teachers ignorant of mathematics teach kids that basic arithmetic is a concept rather than a process.

The reality is that any average 8 year old can master fractions and decimals and any average 10 year old can master algebra. US schools spend a decade teaching kids two years worth of material in math. Don’t even get me started on foreign languages. Nowadays many kids have had three years of foreign language by the time they enter high school, and they have absolutely no skills- they can’t speak, or write or read. One summer immersion program would be more effective and cheaper if the goal were for the student to become proficient. Clearly that is not the goal.

In general the US system is about punching the clock, wasting time, filling up the day. The European and Asian systems emphasize actual proficiency in key concepts and demand more work and more proficiency in their students in shorter periods of time. Americans- if you want your kid to learn math, sign them up for Kumon or other similar programs that emphasize continual practice and allow students to proceed at their own pace. Your kid will know that 2+2 is not a “concept”, it is “4″. They will perform well on standardised tests and be ready when the time comes for real concepts (i.e. mathematics rather than arithmetic).

Even the UK system (not one of the strongest in Europe), puts the US to shame. The average kid with good A levels (essentially a pre-University qualification) is better educated than the average US graduate of a four year state university.

It is not that US students are less capable. They are simply less educated. Standards are lower and true proficiency is not the object of the US educational system. The object is to get the kids through and keep teachers employed. Ironically, at the PhD and higher graduate levels in technical fields, US graduates start to shine. By then these students have made up for their poor primary/secondary schooling and combine knowledge with understanding and insight (as US higher institutions emphasize creativity and problem solving over rote processes and obedience, unlike many countries in Asia).

I could go on excerpting, but you get the point.

Popularity: 10% [?]

Colorado Health Foundation Walton Family Foundation Daniels fund Pitton Foundations Donnell-Kay Foundation