Editor’s note: Brendan Craine is a junior at the Denver School of the Arts, where his focus is creative writing.
So, I recently got my PSAT scores back. As is to be expected, this has gotten me thinking a lot about standardized tests.
In my history, standardized tests have always been a frustrating experience, despite the fact that I generally do quite well on them. I am not entirely sure why I am irked, but If I had to guess, I’d say that something about the idea of ranking each and every student in DPS gets under my skin – it just seems so Orwellian.
Also, I’ve always been against the idea of turning large groups of people into numbers. Do our tests say anything about us, other than our basic level of academic competency? More importantly, is that what DPS and the National Merit Scholars consider important?
I understand the purpose that standardized tests serve, and I have been unable to come up with another way to accomplish what they do, but I can’t shake the feeling that something about them needs to be changed. I can remember that ever since I first took them in third grade, (third grade?) I have hated them, even if it was just on principle, before I was able to really break down their problems.
The analysis and evaluation of the effectiveness of standardized testing is extensive, and so I will not be touching on very much of it here, save for one major point: Standardized testing is standardized. People, however, are not. This is sort of a no-brainer at first. After all, the point of standardized testing is that people will get different scores on the same test. That’s the point of any test, in fact. The only difference is that while individual teachers can craft their tests to fit their curriculum, students, timeframe, and so on, standardized tests are entirely standardized. Everyone takes them at the same time, and they all take the same test.
The idea of making the test uniform for every student is so that you can put all test-takers on an even footing. If all the students take test A on Wednesday morning for a period of 3 hours, and they all receive the same questions, then we will be able to clearly see which students are being taught with the best method, right? Well, this idea ignores several factors that contribute to the test that cannot be standardized. Economic standing, previous education, and other social factors. If my dog, Scruffypaws, dies the night before the test, you can bet that I will not perform as well as the kid next to me, Tommy Wilkshaw, who spent that time studying.
Additionally, once tests become standardized, they have to be dumbed down in the process. The majority of CSAP questions are multiple choice, and the entirety of the PSATs are multiple choice, as well. It takes far less thinking to fill in one of four bubbles than it does to write an essay. There is a chance, if small, that someone filling in bubbles completely at random can receive a good or perfect score.
And the most disgusting part of this is that the PSATs score based on writing ability! How can you gauge someone’s writing ability without actually having them write anything? True, it takes far more time and money to grade essays, but enough of both has already been spent on formulating, creating, distributing and collecting the tests. Surely the grading is worth a little of that, as well?
My opinions on this are not mine alone. This article by writer and critic of the educational system Alfie Kohn outlines a number of major problems created by standardized testing, and this article in the New York Times adds to some of his arguments.
Still, I said that the purpose of this article was not to criticize, and that is true, so I will stop myself before I get carried away. My main concern is: is there another way to fulfill the purpose of the standardize test, but also remedy some of these issues? I openly admit that my understanding of the construction and use of standardized testing is lacking. Because of this, any alternative that I suggest has the possibility of missing some key point, and being completely worthless.
However, breaking down what I perceive to be the purposes of the standardized test, they are as follows:
-Gives a convenient basis for district budget distribution
-Helps the tracking of different curriculums, methods, etc, and their “success”
-Makes it easier for colleges to scout for more desirable students
-Makes it possible to rank Colorado against the rest of the country
So, what if we used a system of student-driven tests?
For example, a “writing” question from a current standardized test might look like “Read the sentence below and circle which alternate composition, if any, would be more correct” followed by a poorly constructed sentence, and three alternatives, one of which is correct under the rules of grammar.
Compare that to this question: “Write out four sentences that use the same subject, verb, and themes, but that are progressively more correct in composition.” Rather than having the student pick from one of four answers for the “right” one, you are having them undergo the exact same thought process of “what is wrong with each of these four sentences, if anything?” But they must be able to identify each individual grammatical mistake on their own, without any assistance.
Following this idea, math questions would be largely proofs and student-constructed story problems, rather than multiple-choice problems with one specific numerical answer. In mathematics, an understanding of ideas is far more valuable than an ability to follow steps. One is based in knowledge of mathematics, while the other is based in imitation of the teacher’s given method.
A student who can explain why it is acceptable to cross-multiply two equal fractions with differing denominators has learned far more than one who simply knows that it is acceptable. This is especially true when dealing with higher math. If all I know how to do is duplicate the results and methods of mathematicians before me, then how much of an understanding of math can I really have?
Finally, reading would be nothing more than a passage of text, followed by space for a response. The current system involves asking the student several questions about theme, metaphor, and major literary devices, and while it is good to be able to recognize these things, it is a skill that is far more valuable in writing than in reading. (For instance, a possible writing question: “Write a short piece that makes use of metaphor.”)
Reading questions shouldn’t assume that there is a “right” answer to any of the questions asked. If a question asks if Old Farmer Brown’s broken left leg was an allegory for the economic crippling of the U.S. during the Great Depression, then suddenly I am forced to treat my own opinions of the text as if they could be wrong, rather than just personal interpretation. If it is widely accepted that Farmer Brown’s leg does represent the great depression, but I didn’t think it did, then I am wrong.
However, if I were given a space in which to respond, then I have the option of bringing up that I thought that Farmer Brown’s leg was a metaphor for his outdated view of the world, which was holding him back. This answer shows thoughtful analysis, but lets me identify and express my own interpretations of the text, rather than having someone else’s forced upon me. Isn’t that a far more ideal goal?
Using this test would take more time and money to grade, but would be much easier to write, since the answers would be generated by the students, rather than the test designers. It would be extremely important for the test to be graded thoughtfully, rather than from the mindset that there is a “correct” answer for every question.
The results of these tests would reveal much, much more about each individual student, since the tests show that I write with blunt, nonflowery sentence composition, a preference for mathematical proofs that are simple and effective, but leave out some deeper elements, and have a tendency towards applying political and historical themes to writing, rather than emotional ones.
All of this dwarfs the information that a current standardized test might give. Namely, that I answered “B” on reading question 6, “11.61” on math question 13, and “D” for writing question 20, that I rounded only to the hundredth place, and that the correct answer to writing question 20 was “C”, and so I was wrong.
Colleges would have so much more information to use for scouting out students. The effectiveness of a teaching curriculum would be clear, since unless each student has a full understanding of the source material, they will not be able to fully explain it.
Likewise, the ranking of Colorado against the U.S., and the distribution of the budget would be essentially the same as now, but based on a much wider scope of information.
I realize that this post is nearing the 1,500-word mark, and so I won’t keep ranting, but just consider this: Students are smart. They are smart enough to understand things and make connections and express themselves without getting suggestions on how to do it.
If the goal of the educational system is to produce a generation of free, intelligent thinkers, then let’s stop testing them on whether or not they can fill in one of four bubbles.
Popularity: 11% [?]
well reasoned, well written, hopefully, well listened to.
Why thank you.
Keep that up. The World needs more people able to thoughtfully critique something and go on to describe interesting alternatives.
Thanks! Hopefully, I still have a few ideas left in me for the next post.
Bless you from a state that fully surrendered to the standardarized demons long ago.
I’m not giving up the battle yet!
I’ve been on Brendan’s side of this issue. Then, I actually did some research. Turns out, most tests such as the PSAT or DPS’ benchmarks or even the dreaded CSAP’s do a reasonably good job at measuring what is defined in the test as student achievement. The kicker is, how are the tests used? What they really measure is not what many think. It’s not the test that kills, it’s who pulls the media trigger.
Standardized tests are not meant to measure creativity. Duh. They are standard-ized. And, the example of the broken leg is a red herring. It doesn’t really matter in a standardized test whether the test taker has another opinion–that is not what is sought. OK? IF the test-maker had wanted Brendan’s opinion on the matter, the test-maker would have asked. The point of the question was clearly about common perceptions and this is not a matter of squelching minority viewpoints but rather about if the student is aware of cultural norms. The more a student is aware of cultural givens, stereotypes the better and the more able said student might use his creativity to subvert such norms for future generations.
You are a very reliable Devil’s Advocate, JJ…
I would respond, but Mr. Buck has beaten me to it. Still, I do appreciate it, it makes me think about my arguments and improves my next posts.
Are you saying that once a widely held interpretation of a piece of literature (or, I guess anything else that appears on a standardized test) has been established that no further opinions are allowed? What kind of an education is that? It is certainly not one that prepares anybody for college, especially those who might actually study literature or writing. And what about the many cases where common perceptions are simplistic, biased or simply wrong.
So test makers really want kids to regurgitate “common perceptions” and it is clear that to a large extent the test drives what happens in classrooms. Do we then conclude that the recent press about 21st century skills like collaborative problem solving, lateral thinking, etc. are simply not aligned with the purpose of public schooling?
Of course the test was not designed to measure those things. However, you get what you decide to measure. If standardized tests are not meant to measure creativity (duh is right), then what do we plan to do about the many problems bearing down on us as a society? Common perceptions have not only not done anything to solve the problems we face, they have contributed to them. And public schools must find an appropriate balance between transmitting the past and taking us into the future.
I assume the question is for me. No, I am not saying that and I’m not sure how it can be construed that I did. And what about the rhetorical questions that never get answered? Did you know that it is a common perception that “Hamlet” was written by Shakespeare? Of course, there are a few who think Marlowe did but who am I to squelch minority viewpoints? Not every single scientist agrees with the climate change data so, best not to test about that. Like I said, you get what you decide to measure–thanks for repeating my essential point.
Look, the days of culturally or ethnically-biased testing is essentially over. The Man is no longer the force he once was and is already in post-modern decline. We don’t need to keep fighting those battles. A perfectly bias-free test will likely never, ever be written about anything, even math and science.
Public schools, so far as I can tell, exist to help people discover themselves and the world around them. What direction we go is up to each one of us, not the schools.
JJ,
I have reread your post several times now and see that I misinterpreted much of what you said. I’m sorry for that. I will still push back on a couple of your points.
I’m not really sure what battling The Man has to do anything and frankly, I lost interest in the Post-modern critique a long time ago. I focus on the effect the “test-maker” has on the educational program I deliver to my students. You state that cultural bias has mostly been removed from standardized testing which seems like a direct contradiction of your assertion (with which I agree) that testing measures “common perceptions.”
By definition these arise from the dominant culture. Whether the test-maker intends to squelch minority opinion or not (I tend not to subscribe to conspiracy theories) is largely irrelevant. The dominant culture exits and kids identify with it in varying degrees. The extent to which they do not identify places them at a relative disadvantage in a standardized testing situation. The clear message some kids get from their test results is, “you’re not up to the task” despite the fact they may be perfectly capable of making their way and even thriving in the world past high school.
I’m not actually sure of your position on this since in the same paragraph you say the days of cultural and ethnic bias are essentially over and go on to say a perfectly bias-free test will likely never be written. I get that the world comes in color, not in black-and-white but I don’t really have a sense of where you fall in the spectrum.
I also want to lift out the unexamined assumption that we can understand standardized testing as measuring something we can properly call “achievement.” Maybe you have examined this assumption but I don’t think we have done so on a scale appropriate to the undertaking.
Based on your two last sentences, it seems we largely agree on the purpose of schooling. I would add that we have a responsibility to equip students with some tools to help them successfully navigate their discovered selves through the discovered world. I am not aware of a body of evidence that correlates high standardized test scores with these outcomes.
Do high CSAP scores, for example, suggest higher matriculation rates? That would be good. Do they suggest lower post-secondary remediation rates, higher college completion rates or better job readiness? Can we connect them with elevated civic engagement or any other aspect of a productive adult life? Do they even point to an ability to read, write and do math in a way useful outside of a testing situation?
I do not know the answer. I’ve put these questions out there before and so far, no one has brought any study or body of evidence to my attention. If you know of a source, I would be grateful if you were to pass it along.
I am familiar with findings that correlate higher test scores with a more superficial understanding of content in some situations. Given the scant evidence available to me I have to ask, Do standardized tests really measure “achievement?”
Thanks Jeff and I suspect we largely agree. Overt or really egregious bias are over imho, but yes, you are correct that bias does still exist. Achievement is indeed a slippery term and idea about which it may be impossible to measure. Agreed. Standardized tests, I’ve found, do a reasonably good job at measuring the limited outcomes for which they were designed. Whether or not they can predict certain outcomes in an individual student years down the road is of course, beyond their design specs.
Trouble is, too many people in positions of power have placed their chips on such tests because for the powerful, cashing-in those chips never actually happens. Who is going to remember 15 years from now how well those 2009 CSAP students actually did in life and how those politicos back in 2001 who supported the whole NCLB canard are now in 2024 elder statesmen angling for reelection and renomination were responsible for how they said CSAP kind of tests were all that?
Brendan (and anyone else who’s interested in learning more about testing):
I would highly recommend the book “Measuring Up” by Dan Koretz, a Harvard education professor who explains many aspects of the construction, use and misuse of testing in a clear and down-to-earth manner.
I’d also like to add if I may the notion that I think you’ll find the vast majority of teachers in DPS agree with — the importance tacked onto the CSAP’s formulaic standardized tests forces teachers to build their curriculum and the format of their classes around the test — I am not suggesting that anything and everything put into the CSAP is garbage that no one needs to know (some of it is), but unfortunately it disables the ability of teachers to use much of their classroom time teaching the details and the in-depths of topics that keep students actually interested in their classes (and trust me, as a student in the DPS system for several years, interest is a big factor in what keeps kids motivated to actually go to class); it also forces many teachers to skim their topics and teach only what is tested in the CSAP, an unfortunate handicap for many classes whose environments would thrive were they allowed a little more freedom.