Statistics…are laid down for our guidance.—Lady Bracknell
Teaching is an act of perpetual discretion.—Professor Barzun
When Lady Bracknell first said that, Oscar Wilde expected his audience to laugh at her. One mark of how far our culture has moved from 1895 is the sense we get now that Of course they are laid down for our guidance! Why else would we have them? It is almost not a joke that the apocryphal woman with four children, hearing that one in every five births is Chinese, decided not to have another baby because she didn’t want it to be Chinese.
Nearly as mixed up as the Sinophobic mother is often the educationist or politician whose calls for action proceed from misused statistics and end in trouble. Educationists’ or politicans’ desire to give young people an education should be driven by the wish to see particular individual graduates who are knowledgeable, capable, and discerning. The stats should be an afterthought for the record only, especially if they are drawn from results on standardized tests.
A story that appeared recently in The New York Times will illustrate an aspect of the problem. The principal of an elementary school populated mainly by poor immigrants and other seemingly backward students, well liked by those students, their parents, and school-district officials, had to be fired in order for her school to qualify for funds under a certain Federal program. The reason was that the students, many of them fresh off the boat, scored poorly on standardized tests. The law required either the principal to be fired or the school to be closed down.
Now, that may sound all right to the firm-action enthusiasts, but it is not all right in a world that values discretion: not so much circumspection, though of course teaching often requires it, as separating, distinguishing, and using latitude of choice and decision properly and effectively (Merriam-Webster). Intellectually, the mandate is on a par with the Sinophobe’s decision to have no more kids.
The test for which the principal was fired required students to read a passage about the first moon landing aboard the Eagle spacecraft. One of the questions was whether the passage was fiction or nonfiction. One of the students reported answering that it was fiction because he reasoned that no one can actually ride to the moon on an eagle. Given his background, he had no idea that a space ship could actually fly to the moon, or that its name, in italics, is distinguished from a label, in Roman type. His not knowing the answer was not due to stupidity, nor was it due to a failure of school in his new homeland to teach him reading.
But there is a more troubling element in this test. Telling fiction from nonfiction is a rather sophisticated operation. Many adults cannot do it reliably, as some media recognize with pleasure and profit. If we think it important to ask ten-year-olds to make a distinction that those media happily depend on adults’ not making, we must do so with discretion and not a one-size-fits-all question.
I have not seen the test, but I guess that the students were given a multiple-choice question like This passage is a work of a) news b) history c) fiction d) persuasion. The question was probably worth one point like all the others. Though the answer can be graded easily by a machine, and though the answers to many such questions may easily produce something that could be called a statistic about educational attainment, there are problems. Such questions leave out of account all parts of a judgment except the result, and they prevent us from using our discretion to size up a student’s attainments. This particular question also omits to deal with the problem of classification that it presents without allowing an entirely satisfactory answer. The structure and requirements of a certain kind of standardized test actually rule out questions that would elicit thoughtful answers, forcing a sometimes unsatisfactory choice among a number of givens.
School districts and other government entities want numbers on the cheap and are impatient with demands for subtlety and discretion. They then use these results to decide on the awarding or withholding of money that districts or schools need for their programs. Also, teachers with very large numbers of students naturally look for ways to lessen their burden of work: there is a strong and understandable temptation (that should be resisted if possible) to use machine-scorable tests or their hand-graded simulacra. So it’s not surprising that these tests are widespread, but it is sad, and it is destructive of good education. At some point people will stop—have stopped—thinking of an education as something incidentally sized up by a variety of means including examination and instead think of an examination as the goal towards which an education proceeds. If what mattered in education were what can be examined by multiple-choice questions, much that is good in teaching and learning would be swept away. Even more would go if the resulting statistics were laid down for the guidance of those who must continue to teach and learn.