Categories
Uncategorized

Can Teacher Come Out and Play?

When I started reading that a public school in New York was called Quest to Learn, I thought at first that I’d bumped into either a send-up or another article on branding—like, say, a story about a new convenience store named Quest to Graze or about a family farm calling itself Quest to Bring In the Sorghum Harvest. The impression of parody or irony deepened as I read that the school offered a course in video games called Sports for the Mind,  “a primary space of practice attuned to new media literacies, which are multimodal and multicultural, operating as they do within specific contexts for specific purposes.” Things became yet more complicated when the reporter advised us readers that this class was “really…a class in technology and game design [emphasis added].” Is the reporter winking and nodding at the helping of baloney offered as a course description while letting us in on its secret?

We are no closer to disambiguation after a long description of a lesson in “enemy movement” in this class, during which the teacher navigates his “sprite” through a maze pullulating with hostile “spiky-headed robots” and into a goal zone. The lesson reaches its climax when the teacher attains the goal with a mere two seconds to spare, causing the students to cheer, pump their fists, mock-swoon, etc. All that’s needed to make it a perfect Hollywood teacher movie moment is the swelling choir of brass and string instruments, but it doesn’t make sense. If this is a lesson in “enemy movement” in a course on the design of technology and games, the reader wonders why no designing has been done; no enemies’ moves have been analyzed; no principles of design have been elucidated, explained, or practiced; and no learning of any kind has been seen or verified.

The key to the mystery appears to be a question-and-answer about sixty lines into the story: “Had he taught anything? Had they learned anything? It depended, really, on how you wanted to think about teaching and learning.”  When I see a line like this, I begin to fear that the writer will start talking about “paradigm shifts,” and that is exactly what happens. Another idea for liberating our thoughts about teaching and learning from actual teaching and actual learning requires “new thinking.” We see it here rising like a turkey that believes it’s a phoenix from yet another educational ash-heap. The ashes came from a bonfire in which “teachers gave up the vestiges of their educational past, threw away the worksheets, burned the canon[,]… reconfigured the foundation on which a century of learning has been built,….blurred the lines between academic subjects, and reimagined the typical American classroom” in order to see what would happen if they did so. Surely only one thing could happen, which explains why so many of education’s pell-mell reforms fail. (William Spady, who has spent much of his professional career promoting “outcome-based education,” used to describe its adoption as requiring a “paradigm shift.” During a two-day workshop for administrators given in 1990, which I attended as a teacher on sufferance, he and his faithful sidekick appeared on the second day wearing t-shirts proclaiming that SHIFT HAPPENS. So it does.)

The argument for all the turmoil this article advocates comes down to three propositions: that children are becoming game-oriented, and so education should too; that kids find games fun, so they should find their classrooms fun in much the same ways; and that classrooms will be left behind if they do not innovate, usually in ways requiring lots of expensive purchases and consultations.

To take the last argument first, some years ago I went to an Indian cultural center operated by the Indian High Commission to watch a film about education in India. The setting was a school whose classroom was the shade under a large tree. Students sat on the ground, except when the teacher addressed them. Then they would spring up with alacrity so they could be standing when they answered him. If this is a feeder school for Bangalore or the Indian Institutes of Technology, where is our argument? I mean that question in two senses. Where is the argument from necessity if (at least some) Indians can become the engineers and designers of tomorrow by starting under the shade-trees of today? And where is the general hope in methods that require funding of a school on a scale achievable only by supplements from an immense private foundation and a school district with a budget in the billions? Many schools, even in the U.S., do not have much bigger facilities budgets than that Indian school’s watering bill (though it might repay study to examine how Finnish schools, generally acknowledged as the world’s best, make do with rather less funding than American schools get).

To answer the second proposition we may start with William Blake’s apt and characteristic observation that “I love fun, but too much fun is of all things most loathsome.” This from a man who penned a poem that is one of the great short critiques of grimly bad education. He still managed to see, as we should, that we must not overdo a good thing. We then move past Blake to view the photograph that appears at the top of the article I am writing about. It shows three children, gadgets in hand, looking excitedly and intently at something, presumably a game screen, behind the photographer. If this is what education-by-games means, surely it must be worth doing if they are so involved in it? When I saw this picture, I thought of another, taken in the early 1960’s by Alfred Eisenstadt. The kids in this picture are just as absorbed as, and perhaps more worried than, our three students. What are they so absorbed by? A puppet show. The proposal attached to the New Paradigm is to enter an arms race of thrills requiring gadgets whose costs will be immense but whose benefits to students may be achievable much more cheaply.

To the first argument I reply with Flannery O’Connor, always helpfully astringent and final: “Ours is the first age in history that has asked the child what he would tolerate learning,” or, I would add, how he would tolerate learning it. Since the kids are not in the position to know the answer to the question What will I learn?, we must put our discretion and the courage of our convictions at the service of the education we lay out–not for their amusement, but for their instruction.

Categories
Uncategorized

JB and the Benelux Boys: Flying over the Radar of Value-added Learning

Years ago we had a new American 9th-grade student whom I’ll call JB. His father was in business development for a large international firm, and whether JB remained in our school or not would depend on whether his father’s efforts were successful. From the start I thought that the boy had something special either to offer or to take away from an English class.

I was right, but sometimes what he had to offer was more than I felt like taking. He had an astonishing facility at classroom leadership that quietly but firmly challenged his teachers. One day he and almost every one of his classmates arrived in class tardy. As usual, his presence was unobtrusive as he sidled in, not at all exultant in successful mass action. My usual punishment for tardies was to have the offender sit in the school yard outside the classroom to meditate for half an hour or 45 minutes on the need for punctuality.

A tardy student had the choice  of sun or shade. For the time stipulated, he or she would sit in a chair without books or electronic entertainment for the course of the meditation. After ten or fifteen minutes of silence and nothing to do, meditation would usually lead the latecomer to a bit of regret that he was alone, silent, and idle. Views that had tantalized the tardy student when they lay on the other side of a beckoning window palled when they surrounded him. Passersby would tease him or her for having to meditate. Thirty to forty-five minutes thus spent usually led meditators to a firm resolution to be on time in the future: it was not the kind of school where punished students meditated and then enacted violent revenge on The Enemy.

But a whole class! Students must learn to expect a certain consistency and firmness from their teachers, yet I wondered whether having them all “meditate” would be an instance of that foolish consistency which is the hobgoblin of little minds. No, I decided, it wouldn’t be, so I quietly told them to take their chairs and follow me outdoors. I placed them without fuss at some distance from each other one by one. When JB’s turn came to sit in mediation, he looked at me incredulously and said in his usual quiet way, “You are going to have an entire class meditate?” “Well,” I answered, “The entire class was tardy.” It was the last time he engaged in one of his exercises in alternative leadership.

That was good because it meant that he could give his entire effort to improving his writing. One knows that some students have a latent talent for writing that has been warped by uncorrected bad habits. JB was one of those. He could find ideas, good ones, but he always had to track in a lot of rubbish with them: clichés, paragraphs that wrote themselves only slowly and tentatively into a topic, the too-frequent use of it is and there as expletives, too much passive voice, and rhetorical inflation.

When he brought a draft in for the doctor’s visit, we spent some time at it. He was, as I guessed he would be, a quick study; and he rebelled much less energetically against rules and strictures of usage and editorial advice than against the demands of punctuality and regular attendance at class. I don’t usually expect 9th-graders to be sensitive to all the stuff that he got—and was evidently getting for the first time. In the course of the year his writing improved remarkably, going way beyond where 9th-graders usually go.

His father didn’t turn up the hoped-for business, so JB left our school after only a year—something that I heard he had done before. Maybe a feral style in one so talented was the result of transferring from school to school before his long-term instruction, as opposed to his annualized value-added learning, could add up. Maybe the schools he attended concentrated on averaged additions of value without regard to individual sums, as it were, or to dividends to be gained by individuals in the future. I was sorry to see him go.

A number of years later I had an email from—who else?—JB, who now writes for a living. He credited me with turning him towards the long-term concern with his writing that ultimately determined his choice of a career. I report his story not to toot my own horn but to suggest how some important conditions of teaching and learning can escape the various nets of value-addition, the obsession with skills instead of skill, and a counterproductive present-mindedness of concern that ignores what my former colleague called “seed-planting.” I am sorry to think that under value-added learning the main satisfaction that a year with JB would bring a teacher is relief at an end to adolescent rebelliousness and a minuscule rise in “average scores.” I am sorrier to think that much of what JB accomplished during that year would escape the standardized tests set to capture evidence of 9th-grade learning.

Lest my readers think that these conditions matter only for kids headed in writerly directions, I want to mention the Benelux Boys. One was from Holland; the other was a Dutch-speaking Belgian. I had them both for the International Baccalaureate Program’s course in Theory of Knowledge. Both were capable and diligent, one of them more so than the other. They didn’t start to study seriously in English till they were in high school.

I taught ToK, as it is known, using among other things a somewhat difficult reading-list drawn from all kinds of writers of the present and past: Aristotle, St. Anselm, Bertrand Russell, Thomas Kuhn, Susan Sontag, Karl Popper, Malcolm Gladwell, Isak Dinesen, William James, Jacob Burckhardt, Hannah Arendt. The class was fundamentally a colloquium with lots of discussion and Socratic questioning, though we did many things other than read, parse, question, and discuss. They included writing, which I examined and commented on closely.

The Benelux Boys acquitted themselves well in that course and were graduated, the Dutch boy with high honors, and were both accepted for the international business program offered in English at the University of Maastricht. After their first year they showed up in my classroom for a visit. The purpose was to thank me for the ToK course.

In the chapter “The Road to Life Adjustment” from his classic Anti-intellectualism in American Life Richard Hofstadter deplores the persistent deprecation in American education of the “transfer effect,” by which intellectual attainments taught in one course transfer their effectiveness to efforts made in others. He notes this persistence in the face of solid evidence and the authority of such respected educators as Jerome Bruner. That book, forty-five years old and fresh as May, should give pause to today’s narrowly focused educational efforts to establish a curriculum for the “delivery” of instruction aimed at “adding value” over the course of a year instead of a lifetime, and at educational demands made for the sake of vocational training taken narrowly.

The Benelux Boys told me that ToK gave them a better preparation for their business course than anything else they had studied because it taught them how to work productively within a language, how to read and parse, how to ask questions of a reading and of each other, and how to evaluate what was presented to their consideration. They also said it conferred an advantage on them over their European classmates whose education had been more overtly a kind of  “vocational preparation.”

With all the benefit to be gained in high school by the marvelous reading available, it seems a deep, deep shame that some schools and districts are spending their book budgets on books of test preparation instead of novels, histories, or anthologies like the Introduction to the Great Books, and that they can claim it is better for their students to have read such stuff than to have worked their way through the writers I named above. I mean not just in preparation for careers in the clouds but careers as writers, businessmen, and human beings who lead good lives.

Categories
Uncategorized

Testing 2

We can tell a good exam by what it does and by what it does not do:

Does

  • distinguish between remembering and understanding
  • require students not just to know stuff but to use it
  • distinguish between sound evaluation and unjustified holding-forth
  • require at least some qualitative distinction among the grades awarded, to be judged by the teacher
  • require good writing whenever it requires writing at all
  • allow the marriage of cow and bull (see below)
  • provide opportunity for learning as well as for demonstration

Does not

  • reward guesswork
  • reward bull or baloney
  • comprise disjecta membra
  • invite the “montillation effect” (see below)

Since there is so much positive to say about a good exam, we can afford to turn first and briefly to what a good exam does not do, thereby getting it out of the way. In my last blog but one, I discussed the rewarding of guesswork and hope that I made a good case against the kind of test that does so.

What I didn’t speak about is another kind of guesswork, called “bull.” In using bull the student essayist slings concepts, generalizations, and abstractions untamed and undisciplined by connections to the world of persons, places, and things, usually in the hope of slipping by with a passable grade. Professor William G. Perry discussed this problem in his essay “Examsmanship and the Liberal Arts.” The essay is very much of its time and place, but it discusses what seems to be an ever-present problem: how to balance the need for concept-work and ideation with the need for an anchor in reality that a command of relevant detail provides. He calls familiarity with such detail “cow” to suggest a balance with the “bull” of empty ideas, and with tongue in cheek recommends a marriage in every good essay. Where an essay is inflated with senseless abstraction and other kinds of hot air, it needs deflation with a sharp red pencil. And, of course, a chaotic collection of snippets that never rises above recall and naming must be downgraded too.

In a prior posting I discussed the “montillation effect,”  in which a student can get  a good or even perfect score in a “test” of “knowledge” that that student doesn’t even understand. We have to be sure that in testing comprehension we are not just inviting students to mimic it, repeating what they have heard about the “montillation of traxoline.”

Another couple of problems come to mind when I think about vocabulary lists. Suppose that you have given your students some words with definitions to be learned: base – a number that is raised to a power; horse – large solid-hoofed herbivorous mammal; network – a thing reticulated or decussated at equal intervals with interstices between the intersections; reed – fibrous core of rattan used in basket weaving; syzygy – a pair of Gnostic aeons male and female. Now, I’ve exaggerated a problem in such lists: that it might leave a student lost—farblondzhet! How does one gain purchase on some of this stuff? The temptation will be simply to memorize the definitions without knowing what they mean. That temptation will be especially strong in a student who knows that he will be examined on his vocabulary by, say, a set of matching questions linking definition and word, a task that has nothing to do with understanding.

The second problem lies in coming to terms with context: even though syzygy is a great word for playing hangman, most of its uses will depend on a grounding that the student is unlikely to have or ever to get: astrology and Gnosticism. Many more words than syzygy will seem bewilderingly uncontextualized to a 9th-grader, a fact that teachers of “test-taking skills” may be reluctant to admit. Learned this way, such words will seem to students like beached monsters of the deep. The study of words learned in context is far likelier to lead to their being useful and used aptly in a carefully prepared essay question, which itself makes their use more likely. A properly set essay question will invite the use of vocabulary learned during the study of a text containing new words.

Setting good essay questions is harder than one thinks, but it is essential to elicit the best possible responses from students—responses that show they have been doing real live thinking while working up their answer. Studying for my teaching credential, I was given distinctions between essay and “objective” tests that were mainly of utility for the teacher: setting an essay test was “easy” but we would be paid back in having to grade responses. Harder was the construction of a good multiple-choice “objective” exam, but it led to more reliable and less troublesome assignment of grades. For the present I don’t want to get into the false distinction between an “objective” test and an essay test, but I do want to assert that forming a feasible yet demanding essay question takes careful thought. That sort of thinking does become easier with familiarity, unlike putting together a hundred multiple-choice questions, which will always require a certain amount of grunt-work. But the question, however long in formation, should demand thinking of all students while offering opportunities for imaginative, thoughtful, well-informed thinking and writing (or their lack) that will distinguish each grade of student from the others.

A word, or more than a word, might be said about letting tests give students a chance to learn as well as to perform. There are a lot of ways to ensure that this happens, but one way that can be really effective is to give out essay questions ahead of time and require that students work up notes to answers in the classes before the exam. If you are expecting to have time for two essays during the exam, set five or ten questions beforehand and give them out. Some questions can test understanding and application of material covered explicitly; some might require them to extend their thinking to deal with problems or issues not explicitly covered. Class then becomes a chance for work, solitary or in groups, silent or talkative at need, with the teacher intervening in individual or group cases, maybe even giving a special mini-lesson to the class as a whole. It is amazing how productive one or two weeks of such preparation can be. Let them bring notes taken in their own hand (but not printed or copied) to the exam, and watch the notes multiply, though students must be warned that the notes may not constitute, in effect, a pre-written essay. Choosing which questions to have them answer is fairly simple: roll a ten-sided Dungeons and Dragons die or use some other random process. That gets the juices flowing even more than—one hopes—they already were.

Implied in the use of questions requiring students to handle stuff that may in some respects be new to them is a rejection of the use of “study guides” before tests. High school students’ study guides should be their own notes. If they don’t learn how to anticipate intellectual demands and respond to them with a readiness they have achieved themselves, they are missing one of the main learning-opportunities an exam can provide.

So, yes, students planning on university or demanding work must learn to handle encounters with surprise. Having done so, they will have the salutary experience, as Professor Barzun says, of disorientation, bewilderment, or momentary dismay followed by a “rallying of forces.” If students’ response to an exam must oscillate between boredom and cave-in, something is wrong in the preparation that such an exam or the study for it  has given them.

Categories
Uncategorized

Added What?

Drive to the east of the Georgian Bay of Lake Huron and you will see some of the oldest exposed rock on earth. Drive to a certain small town there one day about thirteen years ago, and you would have seen some venerable outcroppings of humanity. Mr. S., the guest of honor at a party I attended, was retiring after thirty-five years of teaching the same grade in the same room of the same schoolhouse. One of the guests was the teacher he succeeded, who had taught him in that grade in that schoolhouse. Does it get any stabler than that?

By contrast, one of the schools I taught at had a 25 – 30% annual turnover in students. Its student body was drawn from families with parents on the move. In such circumstances one does one’s best with the students who appear in class only to vanish all too soon. (Well, some of them couldn’t vanish soon enough, but a teacher is nothing if not patient. I sometimes wonder whether being a farmer and cattle baron was Job’s second career, he having learned patience during a first career in the classroom.)

Schools in the U. S. operate across this spectrum, but in spite of the difficulty it would encounter working across such a spectrum, and in spite of other difficulties of implementation, schools and districts are moving towards the “acceptance” of a “formula” that “grades” teachers’ “effectiveness.” I am referring to “value-added learning,” which is not a formula, does not grade, does not measure effectiveness of teaching, and is meeting widespread (though not enough) resistance. It is not a formula because the variables are not commen­surable; it does not give an accurate grade because it does not evaluate things of an identifiable kind against a common standard; and it does not measure effectiveness because “effectiveness” here is the unjustified reification of sets of statistics into The Thing Effectiveness.

Take a number of students in a ninth-grade classroom in their first year of high school. At the beginning of the school year they take a test whose writers claim it can assess what they know in, say, English. In their second year, at the start of tenth grade, they take a test attached to  the same claim. It is in some cases the very same test, kept without provisions for security from year to year. Call the average score of the 9th-grade class AS1 and the average of their 10th-grade scores AS2. According to value-added learning, if AS1 is less than AS2,“value” has been “added” to the test-takers’ growing minds. This “value” is deemed by the proponents of this idea to have been “added” by their 9th-grade teacher.

There’s some illicit deeming going on here. The late great biologist Stephen Jay Gould in his book The Mismeasure of Man warned against reifying statistics, that is, taking numbers and turning them into “things” such as “intelligence” or “value.” Alfred Binet was allowed to finesse a concern about the nature of intelligence when, asked what intelligence was, he was reported to have said, “It is what my test measures.” We might be charmed by such insouciant superbity if we did not know the mischief caused by the misuse of intelligence tests since Binet invented the first one. This time we must be on guard against the mischief, and the first question for the guard to ask is On what basis or authority is it asserted that the score on a multiple-choice achievement test is “value”?

In any case, this view of value omits to consider the long-term good a teacher can do for students—what a former colleague of mine called “seed-planting.” She thought, perhaps naïvely or unbusinesslikely, that a good teacher’s benefit to a student can be lasting or can unfold only in the fullness of time. Professor Barzun thought it wise to give a teacher’s instruction ten or twenty years to see what it amounted to. The present-orientation of “value-added learning” discounts all that. To my former colleague’s pedagogy its question is, “Why are you planting all those trees and perennials—why are you cultivating that joyous and varied field—when you should be concentrating on a patch of what can be sown, grown, and mown in the same year?”

Cut grass lies frail;/Brief is the breath/Mown stalks exhale.—Philip Larkin

Cut grass has worth:/Better mown field/Than tree-clad earth.—Value-added Learning

It also asserts that all “value” is to be imputed to the teacher with no control for other variables. Did the student do independent reading? Did a newly invented gadget distract a number of students from their studies? Did a curtailed school lunch program leave students hungry and unready to learn? No matter, says “value-added learning.”

But even if we unwisely take the assertion of “value” as given, we still have two problems. One is that tests like these typically measure what they claim to measure only to a certain degree of accuracy. Where is the “value” in such tests? If someone wanted to say that “our tests allow us to make some tentative claims with a certain amount of confidence,” we might find in that statement a trustworthy caution. But that is not what these tests will be asked to do. They will be asked to show an exact and definite amount of “value” in a class, whose addition to prior values will in turn yield exact determinations of teachers’ effectiveness.

The other problem is what I will call Big Mac Immodesty. The Economist publishes from time to time its Big Mac Index to the relative value of the world’s major currencies. This index’s inventors assert with tongue firmly in cheek that the Big Mac hamburger is a mini-test of that value. They advise their readers to take the index with a “generous pinch of salt.” Do the proponents of “value-added learning” approach their instruments with the same becoming modesty? Not at all. In the Big Mac Index at least a cow is a cow and a bun is a bun, but in “value added learning” what bits of pedagogical produce are fungible across years? No matter: they will yield their numbers and we will use them. Our Big Mac index is to be swallowed salt-free and uninspected.

Then we get to the problem not faced by Mr. S before his retirement. In his school there was little or no turnover of students from year to year. What about schools whose populations turn over rapidly? Regardless of whether scores go up or down from year to year, they will not be measurements of the same kids. In such cases how can they be said to add anything to anything? If a statistical fluke leaves a school with an influx of poor students to replace an outflow of good ones, are its teachers to be blamed when “value” is “subtracted”? What if a school is reported to have poor test scores, prompting an exodus of students to neighboring or charter schools? Evidence suggests that these educational migrants tend to be brighter and more motivated than the students they leave behind. “Value-added learning” will not report an exodus; it will report a decline in “value,” and the teachers “responsible” for this “value-subtraction” will be deemed “ineffective.”

We also have problems with Campbell’s Law (See my posting “℞ Stone Tablets”): testing is causing corruption, with schools scheduling outings for their Special Ed groups on Testing Day, subjecting poor students to “counseling out,” etc.

Mr. S. briefly touched on his relief that he was retiring when he did, this even though he survived educationist vicissitudes from the early ‘60’s to the late ‘90’s. (Another former colleague, when trying to achieve detachment from the ups and downs of his career in teaching, would sometimes look at me with a smile and say, “Dem vicissitudes!” Indeed.)  I sometimes wonder what Mr. S. thinks of this nonsense, but more, I hope that he is having a delightful retirement, secure from depredations on his reputation and pay, and free at last of  the bad consequences of educationist baloney.

Categories
Uncategorized

A Problem of Fine Distinction

Most of us have seen descriptions of what the letter grades mean. My own favorites are the simplest: A is excellent; B, good; C, fair; D, poor but passable; F, failing. It might be worth examining our sense of the five grades by trying to understand what they mean in particular cases, especially quizzes and tests. I fear that teachers and other people in the field do not often do so. Since to give a grade is to make a judgment, we should have a sense of particular qualitative elements of each grade, or at least have the sort of expertise and connoisseurship that can explain itself. However we do it, in whatever subject, we should be able to make some distinctions between A work and B.

A good assessment must give us what we need to make these distinctions. If a student of English is to get an A, he or she should do things with a depth, a flair, a thoroughness, a subtlety that a B student doesn’t bring to the task. The assessment should therefore give students the chance to show these qualities. Of course it should allow an A student to demonstrate knowledge, but the knowledge is best set in circumstances that also allow the demonstration of skill or know-how and the display of uncommon understanding. Similar distinctions should be possible between the other grades, or why have them?

(In effect, some teachers and schools do not have them, for everyone who attends their classes gets A’s and B’s. I am speaking now of comprehensive schools, not specialized ones with selective admissions. I would be willing to bet that teachers and administrators at such schools giving mostly A’s and B’s cannot readily tell the difference between excellent or good work and fair work without baloney. Of course it is possible that, like the statistically improbable students of Lake Wobegon, the students at such a school are all above average. It is also unfortunately possible that such places are loci of pedagogical or intellectual scandal, as in New York, some of whose “proficiency” tests for promotion could be passed by random guesswork. But let us assume competence, good will, and normal distribution of aptitude for the sake of this discussion.)

A corollary of this idea suggests that awarding grades for tests on a numerical continuum may undercut qualitative distinctions between grades. Imagine a multiple-choice test of one hundred questions. In a standard procedure used by many teachers, a student who answers eighty-nine of them correctly will get a B+, while one who answers ninety will get an A-. On what basis can the teacher giving that test assert that the 90 was excellent work but the 89 merely good?

Perhaps the teacher has devised ten questions unlikely to be answered correctly by anyone but an excellent student. That is fine, but it is probably more than many teachers do, or many of the test banks that go with textbooks and randomly generate multiple-choice and matching questions. Furthermore, the teacher who undertakes the extra work of trying to gauge questions to qualitative distinctions could still be undercut by students’ lucky guesses. Where is the distinction between excellent and good then?

It is often hard to make. Let us take a sample question:

The musical composition called “Emperor” is a. an anthem b. a concerto c. a string quartet d. an aria e. an opera[1]

The answer to this question, combined with those to ninety-nine others like it on a test, might determine whether a student in, say, a music appreciation class (if such things still exist) was excellent, good, or worse.  What makes correct answers to ninety such questions excellent work but to only 89 merely good work? Put this way, I think the question has no defensible answer unless an almost incredible amount of forethought went into the design of the test. (Now, professional test-writers make two or three times as much as professional teachers, but I wonder if even they give this kind of thought to the questions they devise.)

We must also ask How is guesswork discounted? If it is like most multiple-choice tests given in class, the answer is not at all. But let us say that on this test the teacher subtracts .2% for each wrong answer from the total of right answers. A student might have reasoned that no opera would have the title Emperor without an article, that an aria is named after its first line, which Emperor is unlikely to be, and that an anthem would name something or someone more particular, leaving answers b and c.  If test-taking is like gambling—and many courses in “test-taking skills” make it so—the student has increased his odds of a right answer from .2 to .5, making a guess worth the chance. What does that guessed right answer—in effect a coin toss—have to do with music appreciation or even just musical knowledge? A further problem of this question is that the most obvious answer, b, is not the only correct answer, since that name is given not just to the piano concerto by Beethoven but also to a string quartet by Haydn.

If the student had been asked a short-answer question such as Identify a work containing variations on a theme, and name its composer, the student might have written “Haydn’s ‘Emperor’ quartet,”  and the teacher could have been confident that the student knew her stuff. The problem of bad questions would have been sidestepped, and the teacher could at least have been confident that guesswork had been eliminated. (Of course, the teacher would still have had to know that both Beethoven and Haydn wrote compositions called “Emperor,” and he would need a reason for supposing that this fact was somehow representative of the knowledge to be tested and therefore worth including.)

But the fundamental question remains. Does such a test offer a way to distinguish between excellent work and good? I have my doubts. It might at least certify that a mind can retain factual detail, which is important; but where is the assessment of skill and understanding?

The problem demands a solution, and I hope to touch on some in future postings.


[1] The idea for this example comes from Professor Barzun’s discussion of Banesh Hoffman’s book The Myth of Measurement.

Categories
Uncategorized

(No) Comment

Though it would take time for me to tell all the ways in which I gained from being a student in Kenneth Koch’s course of modern poetry, I want to mention one in particular: what he taught me about making comments on students’ papers through the comments he made on mine. That is by way of saying a few things about comments in general.

The first is that properly prepared students are avid for comments. I heard a contrasting view before my third year of teaching, when I attended the summer workshop of a “project” famous for “developing” the “writing process.” The leaders of this workshop contended that students do not read their teachers’ comments, so there is no point in the teachers’ making them. This claim, supposedly based on “research,” was repugnant and in my experience a demonstrably false bit of Bracknellian[1] nonsense, but I entertained it provisionally at least to try and understand in what circumstances it might be true.

I cast back to the team teachers of my 11th-grade English class, Mr. Z. and Mr. M. Any paper returned by Mr. Z. had comments such as “v. good” or “✔”. Those from Mr. M. had notes on my wordiness, my pompousness, or, sometimes, my concision and clarity. Errors of usage came under the red pencil, as did errors of grammar. The virtues he occasionally noted had specific names and were not subsumed with everything else under v. good.

But mostly I thought about Professor Koch (pronounced Coke), whose every returned paper was a course in writing. As you might guess of the teacher of a class in the modern poets who was himself a poet, much of our work consisted in writing poetry. If we studied Whitman, we would write a Whitman imitation, and Yeats, and Pound, and Lawrence. We also wrote a term paper in prose, a midterm essay, and final exam essays, one of which could be a poem.

Unlike many or even most schoolteachers who examine their students’ poems, Koch would subject our poems to genuine criticism, including particular praises and reproofs. The reproofs were as gentle as he could make them if he thought the writer had made a good attempt, but if something was “unWhitman-like,” it was unWhitman-like. He did not accept the notion that students’ poetic work was exempt from discipline and criticism, and most of his students left his class with a realistic estimation of their talents at poetry. He was clear, however, that he wouldn’t let lack of talent get in the way of a decent grade if the student did well on the paper and the essays and gave the poems a try. I was in the unusual position of having him like my poems more than my prose, so my advantage worked the other way around.

The prized comment was “very exciting,” always combined with particulars. It brings me to another important point about comments. Students value the comments of teachers who take them and their work seriously enough to be excited by it, or absorbed, or at least demonstrably occupied. “V. good” doesn’t cut it. If that is what students face when they get their papers back, then of course they would not value the comments. One 9th-grade student during my student teaching submitted a homework assignment early on that said in the middle of a prose passage, “Mr. V are you reading this?” I wrote in the margin, “Of course. I assigned it.” He came to me the day after I returned the paper and thanked me for paying attention to his work. I had his attention for the rest of the semester.

That brings us to a third point. Professor Barzun said that teachers who offer their students the criticism their writing needs and deserves will “work like dogs.” If that was true of his colloquium in important books with its twenty students, or of Koch’s course in modern poetry with its forty, it is terribly true for a high-school teacher with an unspeakably large number of students. Writing, a skill or talent, requires the teacher to be a coach and editor; coaching and editing must aim at particular people. Teachers with large loads of students are bound to have trouble managing this demand unless they have extraordinary fortitude and stamina. The difficulty lies in the quality and intensity of attention required and in the degree of detail that has to come under the teacher’s active notice. It took me a long time to develop that kind of editorial stamina, and even now I have to pace myself when grading writing, taking breaks to stay fresh and open, not burnt out. New teachers daunted by the job should know that the needed ability will probably come, but they must not suppose that it will be easy. Candidates considering jobs at schools where they will have large loads of students may have to inure themselves to dealing with an insufficiency of time and knowing that even with the best will, their work may be “not altogether satisfactory.” Perhaps teachers working in these conditions end up not offering the kind of comments that students take to, but through no grave fault of their own. How sad, then, if they end up moths in a flame!

The answer to the problem of students’ not taking their teachers’ comments seriously does not lie in abandoning comments. Rather, it lies in establishing conditions in which teachers know what comments are worth making and have the working conditions in which to produce comments worth reading.  On the students’ part it means coming to an assignment prepared and ready to seek and take advice, the way my 9th-grader could do once he had satisfied himself that I really recognized him.


[1] This coinage refers to Lady Bracknell, who says that “statistics are laid down for our guidance.”

Categories
Uncategorized

℞ Stone Tablets

Sometimes it’s a pity that a valuable dictum cannot be presented on a stone tablet by a prophet. Lightning and thunder might help it make an impression too. My dicta of the day are sometimes called Campbell’s Law or Campbell’s Laws, after Donald T. Campbell, a social scientist who died about fifteen years ago. They appear not on stone tablets but on page 49 of a paper he wrote he wrote in 1975:

“The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures, and the more apt it will be to distort or corrupt the social processes it is intended to monitor.”

The next few pages of the paper give examples of corrupted processes from law enforcement and government administration, but they end with a clincher drawn from the world of education. A program of compensatory education in Texas was corrupted by the private contractors hired to administer it. It turned out that success was to be determined by the performance of the targeted students on a test, and the contractors “coached” their students in order to produce good results.

Campbell reports that the contractors “defended themselves with a logical-positivist, operational-definitionalist argument that their agreed-upon goal was defined as improving scores on that one test,” but that their methods were generally regarded as scandalous. How far we have come since this scandal was reported in a 1971 paper can be gauged by the fact that not just the city of Texarkana but much of the United States since No Child Left Behind is doing the same thing. Campbell saw it coming and warned that “when test scores become the goal of the teaching process, they both lose their value as indicators of educational status and distort the educational process in undesirable ways.”

Education, he points out, is not uniquely corruptible: the problem lies in the very idea of gathering simplistic data, deciding that they represent or encapsulate a complex state or process, letting them become normative, using them to determine whether someone has done something right, and attaching these evaluations to a system of rewards and punishments. He was pessimistic about the possibility of circumventing his laws, and the growing “testing-&-accountability” fiasco is bearing him out.

Categories
Uncategorized

Reading and Wisdom

The Introduction to Great Books series was originally intended for use in high schools, though a visit to its web site suggests that its main readership is now adults in reading groups. It is excellent to find people of any age reading Kafka, Conrad, Dinesen, O’Connor, and Tocqueville, but sad to think that the joys and rigors of such encounters may not be a part of many high-school English curricula any more.

If students have been properly trained in reading, study, and discussion for a number of years (maybe having read the Junior Great Books series), they will be able to handle these authors and others even more difficult: William James, Thomas Kuhn, Sir Karl Popper. It is not just their vocab­ulary that will grow, but also their range of expression and thinking.

The vocabulary and understanding grow in a healthy and productive way because the students encounter words alive in the reading and not dead on a test-prep list. The health is threefold. It assures them that the words they are learning are used by real writers and so worth learning as part of a real and not a fake landscape of language that they are exploring. It sets up oppor­tunities for emulation or trying-on in writing and discussion, during which they can explore and test good usage and fit words to thoughts. And it gives them a chance to learn how to be rigorous and to increase their intellectual stamina.

William James reports that H. P. Bowditch, “who translated and annotated La­place’s Mécanique céleste, said that whenever his author prefaced a proposition by the words ‘it is evident,’ he knew that many hours of hard study lay before him.” He had enough humility and eagerness and a strong enough sense of obligation to the masterpiece he was encountering to be ready to tangle with it hard and long. Even­tually what had been evident to Laplace became evident to Bowditch too. The same process, on a less exalted but no less important level, occurs or can occur in high school with students who have effectively met a series of demanding but rewarding readings with study and careful conversation.

The sense that something is evident becomes stronger and more confident as it is tested, proved, probed, and exercised. And here we come to a benefit of a good course of reading and discussion that will escape capture by checklists of little skills and attainments, which embody the reductive fallacy in their pedagogical assumption (in this case the fallacy is that high-school reading is nothing but a string of discrete “competencies”). At some point, often but not always foreseeable, the student passes into an intellectual terrain to which James refers when he says that “the art of reading (after a certain stage in one’s education) is the art of skipping.”

But he makes that observation in an analogy whose second part is that “the art of being wise is the art of knowing what to overlook.” Here we have something diffi­cult or impossible to manage in a pedagogy aimed entirely at taking baby steps even on the verge of adulthood through a course of explicit, identifiable minor achieve­ments: how do we cultivate wisdom and understanding when they cannot be handled reductively? We come back to the problem Gilbert Ryle noted in his essay “On Forgetting the Difference between Right and Wrong.” It is that we are talking about something different from knowledge and skill.

If we don’t talk about it, we risk educating not a “mind of a high order” but someone like Funes the Memorious, the title character of a story by Borges. Poor Funes! He couldn’t skip or overlook anything. “My memory…is like a garbage heap,” he tells the narrator of the story. He could remember tens of thousands of futile details but had almost no ideas. James would say that he lacked the power of reasoning because he had learning but not sagacity. Of course, Funes would have been able to ace the kind of test that treasures the quick manipulation of learned detail. Of Funes and people like him one is tempted to ask, with T. S. Eliot, “Where is the wisdom we have lost in knowledge?/ Where is the knowledge we have lost in information?”

Categories
Uncategorized

By the Numbers

Statistics…are laid down for our guidance.—Lady Bracknell

Teaching is an act of perpetual discretion.—Professor Barzun

When Lady Bracknell first said that, Oscar Wilde expected his audience to laugh at her. One mark of how far our culture has moved from 1895 is the sense we get now that Of course they are laid down for our guidance! Why else would we have them? It is almost not a joke that the apocryphal woman with four children, hearing that one in every five births is Chinese, decided not to have another baby because she didn’t want it to be Chinese.

Nearly as mixed up as the Sinophobic mother is often the educationist or politician whose calls for action proceed from misused statistics and end in trouble. Educationists’ or politicans’ desire to give young people an education should be driven by the wish to see particular individual graduates who are knowledgeable, capable, and discerning. The stats should be an afterthought for the record only, especially if they are drawn from results on standardized tests.

A story that appeared recently in The New York Times will illustrate an aspect of the problem. The principal of an elementary school populated mainly by poor immigrants and other seemingly backward students, well liked by those students, their parents, and school-district officials, had to be fired in order for her school to qualify for funds under a certain Federal program. The reason was that the students, many of them fresh off the boat, scored poorly on standardized tests. The law required either the principal to be fired or the school to be closed down.

Now, that may sound all right to the firm-action enthusiasts, but it is not all right in a world that values discretion: not so much circumspection, though of course teaching often requires it, as separating, distinguishing, and using latitude of choice and decision properly and effectively (Merriam-Webster). Intellectually, the mandate is on a par with the Sinophobe’s decision to have no more kids.

The test for which the principal was fired required students to read a passage about the first moon landing aboard the Eagle spacecraft. One of the questions was whether the passage was fiction or nonfiction. One of the students reported answering that it was fiction because he reasoned that no one can actually ride to the moon on an eagle. Given his background, he had no idea that a space ship could actually fly to the moon, or that its name, in italics, is distinguished from a label, in Roman type. His not knowing the answer was not due to stupidity, nor was it due to a failure of school in his new homeland to teach him reading.

But there is a more troubling element in this test. Telling fiction from nonfiction is a rather sophisticated operation. Many adults cannot do it reliably, as some media recognize with pleasure and profit. If we think it important to ask ten-year-olds to make a distinction that those media happily depend on adults’ not making, we must do so with discretion and not a one-size-fits-all question.

I have not seen the test, but I guess that the students were given a multiple-choice question like This passage is a work of a) news b) history c) fiction d) persuasion. The question was probably worth one point like all the others. Though the answer can be graded easily by a machine, and though the answers to many such questions may easily produce something that could be called a statistic about educational attainment, there are problems. Such questions leave out of account all parts of a judgment except the result, and they prevent us from using our discretion to size up a student’s attainments. This particular question also omits to deal with the problem of classification that it presents without allowing an entirely satisfactory answer. The structure and requirements of a certain kind of standardized test actually rule out questions that would elicit thoughtful answers, forcing a sometimes unsatisfactory choice among a number of givens.

School districts and other government entities want numbers on the cheap and are impatient with demands for subtlety and discretion. They then use these results to decide on the awarding or withholding of money that districts or schools need for their programs. Also, teachers with very large numbers of students naturally look for ways to lessen their burden of work: there is a strong and understandable temptation (that should be resisted if possible) to use machine-scorable tests or their hand-graded simulacra. So it’s not surprising that these tests are widespread, but it is sad, and it is destructive of good education. At some point people will stop—have stopped—thinking of an education as something incidentally sized up by a variety of means including examination and instead think of an examination as the goal towards which an education proceeds. If what mattered in education were what can be examined by multiple-choice questions, much that is good in teaching and learning would be swept away. Even more would go if the resulting statistics were laid down for the guidance of those who must continue to teach and learn.

Categories
Uncategorized

Trips in Deep Water

Those of us who remember particular school trips fondly and who have subsequently become teachers now realize that there was much more to them than met our young eyes, and more than often meets parents’ eyes. The amount of planning most of them need would surprise someone not familiar with the work, but there is more: Even the most innocuous-seeming ones can turn difficult or perilous in an instant. Difficulty and peril can’t be entirely eliminated except by avoiding life or by shuffling through it in wrappings and shin-guards. Somewhere between this stance and a blithe disregard of caution lies the right way to take a school trip. Exactly where needs some teasing out.

So does the possibility that school trips are less than meets the eye; that they do not amount to much more than an excuse for a cut-up. In an earlier posting I said that school-wide play days, honestly so called, could serve this purpose. I think that trips should generally have more important goals. Some questions follow, which examine trips’ usefulness balanced against their potential for poor learning and for trouble.

Since no trip is risk-free, the first question must be What about this parti­cular trip makes it worth at least a minimum of risk? It must yield a modicum of value, to be assessed with a clear eye and no baloney. The Activity Director’s “Default Setting” should not be “yes;” it should be “tell me more.” His or her judgment should not be compromised by having to sidle up to a “core value” claiming that “our school values field trips for their own sake” or its equivalent in Edspeak such as “our school seeks out the intrinsic benefits of mobile experiential learning.” Though the trip might have many kinds of value, the most important kind would be curricular value: students will learn something the curriculum requires them to learn. Other values, such as “bonding” or “team-building,” might better find their cultivation in work and activity on the school grounds.

The next question is How does it fit in with other plans and activities, both curricular and extracurricular? An administrator must be ready, like Solomon, to decide at need between competing activities, or, like P. T. Barnum, he will end up with all of them in the tumultuous tent. Something is educational—or “miseducative” (John Dewey’s word)—partly intrinsically and partly as a result of its juxtaposition with students’ other opportunities—demands—distractions. Dewey thought that a whirligig of weakly connected or disconnected activities would be miseducative:  If the kids are in a whirl of things to do, the learning will be faulty, or they will learn the wrong thing.

Will the trip include too many attractive nuisances or too much opportunity for idle mischief—I mean inappropriate behaviors? Nuisances can become more attractive to students who are not absorbed by the main event. How sure are we that they will be? Long stretches of vacant or repulsive time are an invitation to the boredom whose ensuing choice is often subversion or trouble.  Rather have too much to do than too little.

What is the state of relations in general between parents and the school, in particular between parents and the teachers acting as chaperons? A school with healthy, trusting relations will have more leeway for trips than one whose relationships are mistrustful, adversarial, or litigious. This relationship should be the object of a clear-headed examination notwithstanding any claims to the contrary, such as a “core value” stating that, say, “Parent – school cooperation is foundational to our mission.”

If special understanding or expertise is advisable, does a chaperon have it? If not, will there be someone readily available who has it? A class going to a beach will want to know that a chaperon can read the surf, spot rip tides, and use a lifesaver’s can. If not, there had better be a lifeguard. How will availability of the needed expertise be ascertained?

Do all adults understand that they are not to play for popularity among students by disregarding rules, especially those that other chaperons are visibly enforcing? It’s surprising how many “grown-ups” will say to themselves and even to students, “Well, Mrs. Dust may think that’s a good rule, but I am a hero, so I will disregard it.”

Do the students understand procedures for gathering, for quieting down, and for receiving urgent instructions? Have these been reviewed and, if necessary, drilled? As an example, a group of students, no matter the size, should be able, when signaled, to become silent and attentive within seconds.

Some comments to follow the questions:

No first-year teacher should ever lead a school trip except under the direction of another teacher experienced in leading trips.

Chaperons other than teachers should be known to and approved by the school’s administration; otherwise, they should not “count” in determining the student – teacher ratio of the trip or be allowed in charge of small groups when the large one breaks up.

Students should not consider attending a trip their right irrespective of prior behavior and reliability. If deciding what this means ends up being a sticking-point among warring factions, it might be better to give up the trip than to have a stinker-student win a battle to go and then destroy a trip’s chance for peace or safety.

Though of course a student’s coming to harm is the salient disaster of any trip in which it occurs, a word might be spared for the teachers in charge, irrespective of blame. Having been assigned duty on a trip, maybe because no one else was available, an inexperienced chaperon on a trip that goes bad is like Pip during his first time in the whaleboat in Moby-Dick. One moment, things are going as planned; three minutes later, Pip is a castaway, feeling “an immense concentration of self in the middle of…a heartless immensity,” whose “ringed horizon [begins] to expand around him miserably.” That horizon, seemingly with only menace, guilt, and sorrow in the offing, must feel like the most distant in the universe.