David  D.Thornburg
[DirectConnect]
-----------------------------------------------
Pencils Down! How Decontextualized Standardized Testing Can Destroy Education
by guest columnist David D. Thornburg
Director, Global Operations, The Thornburg Center • Lake Barrington, Illinois
-----------------------------------------------

[Editor's Note: David Thornburg, author of this issue's guest "DirectConnect," is known worldwide as a futurist whose contributions to educational technology go back to the early days at Xerox PARC in the 1970s. He is a member of the Congressional Commission on the Future and speaks before 170,000 people each year. His dedication to education, particularly achieving the promise of technology to improve learning, is unsurpassed.]
Roger Shank, chairman and CTO of Cognitive Arts and former director of the Institute for the Learning Sciences, tells a compelling story: When you learn to drive a car, you are given two kinds of examinations: One is multiple choice, and the other has you actually drive the car. Think about which of these is the most important. As a fellow driver, or even as a pedestrian, I would imagine that you feel more comforted by the fact that actual demonstrations of car driving ability are required before getting a license.

While the motor vehicle department requires both forms of testing (only one of which is truly relevant), our educational system has chosen to emphasize only one kind of testing for academic subjects—and it is the one that is least relevant to actual practice. The decontextualized multiple-choice standardized exam has become more than a measuring instrument, it has become the shield behind which people are hiding to avoid transforming our educational institutions. "Yes," many say, "I'd love to do more authentic assessment, include new curricular areas, do more project-based activities, use technology in more creative ways, and so on, but my students need to do well on the alphabet soup of standardized tests required by the powers that be, so I'll just teach to the test."

This response (and variants of it) are common among many educators, some of whom are truly afraid of taking a risk that students should actually know something of significance, in case this approach might mess with their capacity to fill out a bubble sheet properly. Never mind that students who actually master material will probably do well, no matter what kind of test they are given, our educational system is operating on the principle that what gets measured gets done. Furthermore, the medium and the message of the test have merged in a McLuhanesque manner so that the content of the exam is taught in ways that support the form of the exam. Students often have worksheet assignments that mimic the multiple choice questions they will be asked, and software vendors are only too happy to fill our hard drives with more and more test practice, complete with little "attaboys" for right answers and gentle prompting when the answer is wrong.
 

B. F. Skinner Died for Our Sins
Some months back, I devoted a PBS commentary [http://www.pbs.org/thornburg] to this topic and recounted an experience I had speaking to a roomful of superintendents in a state with high-stakes testing for tenth graders. On a whim, I picked up a pile of papers and announced that I was going to have each of them take just one page of the tenth grade test, which we would then grade and explore. The room became so still you could hear your heart beat. When I asked these superintendents why they were afraid of their own test, they suggested that they would not do well because the test didn't measure the kinds of things people needed to know once they left school.

Well, that's almost true. If you get to play Who Wants to Be a Millionaire?, you'll have the chance to show off your capacity to answer decontextualized questions on random factoids using the multiple choice format. Furthermore, unlike our classrooms, participants are allowed to phone a friend, poll the audience, or eliminate two wrong answers. But, other than this show and others like it, the capacity to know the difference between the median and the mode of a table of numbers is probably not required for many people to live fully productive lives. Those who do need this knowledge have ample chance to use it in context and therefore do not need to "remember" it the way we crammed for tests.

What's wrong with asking kids to spend an hour or so answering multiple choice tests? What's wrong with the SAT, IQ tests, and the myriad progeny that have students quaking and teachers angry? I'm not going to recount the sorry history of decontextualized multiple choice tests. Others have written about the use of such tests to justify keeping African Americans and Italian Americans out of the armed services many years ago.

What's wrong is simply this: Standardized tests freeze the perception of learning as the acquisition of isolated facts that can be unambiguously answered through a multiple choice question.

For example: Art is a higher form of knowledge than experience. True or False?

Um, ah, well.... This is a compelling question. One that should form the foundation of an entire course. Aristotle might have seen this issue one way (volume 1 of his Metaphysics would make good reading at this point), but others would have a completely different take on the topic. In fact, the whole point of making a statement like this is to stimulate (gasp!) thought.

Wow. Thinking! Now there's something worthy of our attention. How much thinking goes on when you are taking a timed multiple-choice test? Furthermore, how much of your thinking is devoted to formulating a guessing strategy as opposed to thinking about the topic?

Of course, some might argue that this means we should use essays or performances to evaluate writing, reading, social studies, and the like, but that multiple choice tests are just dandy for mathematics. Well, depending on what you are asking, certain math skills can be gleaned from multiple choice exams. If you have four choices for an answer and you don't know which answer is correct, logically you might think that the test designer has tossed in an answer that is way off the mark to trap guessers. You could then look for this answer and eliminate it. Statistically, you may have noticed that the answer C seems to be coming up quite often, but then you have to be careful because the test designer may have tried to randomize the order of the correct answers, so you decide to eliminate this as a sure thing. Still, by eliminating another wrong answer, you'd be able to guess with a 50 percent chance of getting the right answer. Because a wrong answer has a 25 percent penalty to your overall score, you'd be ahead (statistically) by guessing at this point, so you toss a coin and fill in your bubble.

I know a person who passed a college entry test in physics and chemistry (subjects she never studied) by simply applying these (and other related) principles. Standardized tests measure our capacity to take standardized tests at least as much as they measure our knowledge. And, for every student who has a good test-taking strategy, there are many who don't. Don't believe me? Then think about this:

Bush

Gore

Buchanan

Nader

Pick one.

Many thousands of people (not all of whom live in Florida) made what they thought was their choice, only to find that the test grading equipment (the voting machines) could not deal with their response, no matter how clearly it was expressed. We tell students to be sure to fill in the dot or the area between the lines, but don't go outside the box. So, even if you know the desired answer, the test taker has the added responsibility of encoding this answer in a form that is readable by the grading equipment.

One simple reason we give multiple choice examinations is because they are easy (read cheap) to grade. Having students demonstrate their work and thinking process requires human intervention, which would just make it too darned hard to administer the tests to millions upon millions of learners every year. Of course, if we tested less, but used assessments that measured students' capacity to think, then we'd probably be able to keep within our existing budget. Of course, this misses another point: Testing is a big-money business, and, like the textbook publishers, testing companies will logically resist any attempt to transform education, especially if these reforms come at the expense of their bottom line.
 

So, What Do We Do?
Well, in my PBS commentary, I suggested a boycott/teach-in. All the teachers in a chosen district would simply refuse to administer the test to students on test day. Instead they would offer the test to parents, administrators, government officials, real-estate agents, and other business and community members. The tests wouldn't even have to be graded (although having the politician's grades published in the local paper is a delicious thought). Instead, the people who took the test could engage in a discussion around these questions:

1. Does this test measure anything of lasting (e.g., post-school) value?

2. Are there important skills and habits of mind that these tests don't even bother to measure?

3. Are there more effective ways to find out if our students actually are learning?

As MMS editor Ferdi Serim correctly observes, there is a world of difference between high academic standards and standardized tests. Meaningful assessments are essential in education and in life after school. The current crop of decontextualized multiple choice exams are, in my view, a lousy way to measure student skills.

I'd love to write more, but it's time for my driving test.
 

David D. Thornburg is director of global operations for The Thornburg Center [http://www.tcpd.org]. Communications to the author may be addressed via e-mail to dthornburg@aol.com; or via phone at 847/277-7691.

Communications to the Editor may be addressed to: Ferdi Serim, MultiMedia Schools, 11 Palacio Road, Santa Fe, NM 87505; 505/466-1901; fax: 505/466-1901; ferdi@infotoday.com.
 

[Information Today Inc.]
Information Today Home Page
[MultiMedia Schools]
Home Page
[Current Issue]
Current Issue
[Current Issue]
Top

Copyright © 2001, Information Today Inc. All rights reserved.
Feedback