Sometimes it’s a pity that a valuable dictum cannot be presented on a stone tablet by a prophet. Lightning and thunder might help it make an impression too. My dicta of the day are sometimes called Campbell’s Law or Campbell’s Laws, after Donald T. Campbell, a social scientist who died about fifteen years ago. They appear not on stone tablets but on page 49 of a paper he wrote he wrote in 1975:
“The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures, and the more apt it will be to distort or corrupt the social processes it is intended to monitor.”
The next few pages of the paper give examples of corrupted processes from law enforcement and government administration, but they end with a clincher drawn from the world of education. A program of compensatory education in Texas was corrupted by the private contractors hired to administer it. It turned out that success was to be determined by the performance of the targeted students on a test, and the contractors “coached” their students in order to produce good results.
Campbell reports that the contractors “defended themselves with a logical-positivist, operational-definitionalist argument that their agreed-upon goal was defined as improving scores on that one test,” but that their methods were generally regarded as scandalous. How far we have come since this scandal was reported in a 1971 paper can be gauged by the fact that not just the city of Texarkana but much of the United States since No Child Left Behind is doing the same thing. Campbell saw it coming and warned that “when test scores become the goal of the teaching process, they both lose their value as indicators of educational status and distort the educational process in undesirable ways.”
Education, he points out, is not uniquely corruptible: the problem lies in the very idea of gathering simplistic data, deciding that they represent or encapsulate a complex state or process, letting them become normative, using them to determine whether someone has done something right, and attaching these evaluations to a system of rewards and punishments. He was pessimistic about the possibility of circumventing his laws, and the growing “testing-&-accountability” fiasco is bearing him out.