Category: Uncategorized

In a posting last September I briefly discussed the notion of letting a test be a learning experience rather than simply an assessment. The three most important parts of doing so were 1) setting essay questions in advance, 2) letting students take time to guide themselves through preparation for an answer, with the teacher ready to pitch in at need, and 3) actually writing the essay after a momentary surge of excitement and interest caused by a lottery to choose which question to answer.

I want to return to that subject because of an article that recently appeared in The New York Times. It reviewed research showing that students remember what they “learn” better by being tested on it than by working up “concept maps” or by doing little review sessions. It is important to note the article’s summary of the experimental protocol: “Without the passage in front of them, they wrote what they remembered in a free-form essay for 10 minutes [emphasis added].”

A moment’s thought will tell us why an essay, even a “free form” one, is intellectually both more demanding and more rewarding than a concept map. A simple line drawn between two words or phrases suggests a relationship per se but tells us nothing about it. By contrast, even a “free form” essay is not so free as to dispense with the parts of speech, which means that it will link concept-words with prepositional phrases, clauses, and conjunctions that express in some fashion exactly what kind of relationship those words have. Drawing lines is by comparison a barren evasion, a mini-holiday from responsibility. Even less thought will tell us why an essay is superior to little review sessions, which so often amount to nothing more than students rattling off details to each other like the young George Orwell conning his history facts with black-negress mnemonics. How many minutes after the examination for the Harrow History Prize did he remember the names of the battles in the War of the Roses?

The article says that “why retrieval testing works is still unknown.” I have a couple of possible explanations. The first depends on my understanding of William James. According to him, “what interests us is real,” but the world of reality is actually “Many Worlds.” After taking an interest, however cursory, in something, we place it in one of reality’s Many Worlds using our judgment. He says further that we place these worlds in a hierarchy of importance that depends on our temperament and mental inclinations. In effect, some of the Many Worlds interest us more than others and are therefore more real.

Most people do not rank the world of abstract relationships high among the Many Worlds, so that when Bertrand Russell reports in his autobiography that learning geometry was like being in love for the first time, most people do not share his delight in that reality. They are not nearly so interested in abstract relationships as Russell, and we don’t fault them for it. Most people think of abstract relationships in connection with concrete or sensory problems when they think of them at all. It is strange in this light for educational psychologists or cognitive scientists see the “gold standard” for examinations in concept mapping, a technique that puts a premium on the establishment of abstract relationships, sometimes expressed in a line without a name and therefore without a “stinging term,” as James calls those interesting little bits of reality that compel our attention.

To understand why writing an essay test works better than a concept map, we should revise what we think of the “cognitive and affective domains” of learning that we heard about in teachers’ college. Cognition and affect do not have separate domains: they are the condominium of intellect. When we are interested in something, we cognize it better, more fully, and with greater relish than when we are not. The networks of interest that we establish run from their high points in the worlds we cherish down through other worlds to those about which we are indifferent.

Good teachers complement students’ predilections by establishing extrinsic patterns of interest to connect the Many Worlds, just in case not everyone wants, like young Russell, to fall on geometry with a tiger’s appetite or, like the young Samuel Johnson, to read thousands of books by his eighteenth year. One interest universally established is in the results of a test. Whatever else we have to teach high-school students, we usually don’t have to get them interested in a test grade. (The student too flat and feckless to be interested by anyone in anything is another matter.)

A teacher who sets essay questions has two things going for his students’ learning: one is the interest of the question itself, and the other is the interest provoked by the situation of the test. When the student works up notes on an essay answer and then writes it, he is propelled by test conditions and his interest in the question to establish the concept-work needed to give a shape to the details and in complementary fashion to use those details as the factual anchor of his ideas. He expresses relationships in language not lines, thereby getting them more exactly. He has chosen the language for an exercise in which he has an interest. All these things work together to make the material more memorable than it would be as a jumble of detail or a wind of unsecured ideas. Used correctly, testing doesn’t just examine thought and memory: it secures them.

But the testing-and-accountability people should not think this insight vindicates their approach. Their testing is not an invitation to the exercise of directed and participatory intellect characteristic of an essay exam with a generous run-up. The competency tests favored by them do not invite anything except pointing (at The Right Answer). They are entirely summative, not at all formative; and they are summative in a bad way.

The constructivist Howard Gardner rightly sees the results reported in this article as a challenge to constructivism, but surely a large mind is capable of recognizing the challenge as an opportunity to enlarge the possibilities of the classroom beyond the constraints of a single approach. Essay questions and other tasks set by teachers combine direction by teachers and construction by students in a synthesis worth pursuing. Surely there are more such possibilities?

Those of us who believe that memory is more than a test-taking skill also have a rooted conviction that it works with other mental powers, sometimes for simple delight but also as a guide. One of the oldest ways of grouping these powers was to call memory and its allies the Five Wits. Of those five, I want to consider imagination and common sense as helping us with our memories to take good educational decisions and actions and to avoid bad ones.

Memory not serving for much more than threescore and ten years, we have developed our imagination and common sense (could we in this case call it worldliness?) in the discipline of history. That, combined with the records of annalists and chroniclers, either as written or as shaped by need and culture for transmission, is what Richard Hofstadter called “the collective experience of the human race.”

When the collective experience of the human race tells us something, we ought to listen. The same with history. It is therefore exasperating to read that in Brooklyn a “new experiment” is in progress that appears to be ignoring these lessons. This “new experiment” is none other than an Open Classroom.

Against history, common sense, imagination, memory, and the collective experience of the human race are ranged a handful of Harvard graduate students and Joel Klein. They seem bent on confirming the critic George Steiner in his dictum that “American education is organized amnesia.” It must be, to have forgotten the open classrooms of forty years ago.

But when memory won’t serve, we could use imagination and common sense to supply the mental shortfall. The ostensible purpose of this open classroom is to duplicate the educational successes of the Phillips Exeter Academy, “where students in small classes work collaboratively and hold discussions around tables.” So they do, and well, but we must think and imagine carefully before bringing Exeter to Brooklyn.

First, there are physical and organizational differences between a classroom at Exeter and Grand Central Terminal. It is possible to conduct group activities in Grand Central, as a number of successful restaurants show, but I would not want to generalize from this success that uninstructed and incontinent six-year-olds in a cavernous constructivist environment will get their tasks as successfully as commuters headed to their tracks or as menu-driven diners at the oyster bar or the steak house.

Second, the students at Exeter were eminently successful pupils at other schools and have internalized many, many lessons in conduct, culture, and academics that a hangarful of first graders would not yet have managed to assimilate.

Third, classes at Exeter are generally limited to twelve students (not just any twelve, but twelve focused, highly motivated, extensively trained, and disciplined young—but not too young—people). Anyone who has taught classes of twenty-five or thirty and classes of twelve or fewer students, as I have, knows that there is a world of difference between the two. It is suspect, if not reckless, to assume that successes with a room of twelve in Exeter can be duplicated in a room of sixty in Grand Central.

Common sense and imagination tell us these things, but the open classroom also has the lessons of history against it. Open classrooms were tried, and they failed.

It is pointless to argue that they might have succeeded if they had been conducted by teachers who leap tall buildings, walk on water, and assimilate the findings of second-rate research: most schools don’t have many such teachers. That argument is another version of the old wheeze that the idea wasn’t misbegotten, but that instead the teacher was to blame.

The impetus to blame also seems to have governed another decision at this “academy.” I mean the assignment of students to single teachers in successive years. Is it done, as it ought to be, because there is an educational benefit from knowing whom you teach? (And there is.) No, it is done to increase teachers’ “accountability.” All that this assignment will end up showing is that bad thinking sometimes accidentally has good consequences. On a lark, here is bit of accountability-thinking: whenever Joel Klein or anyone else cooks up and imposes a scheme to improve learning and it doesn’t work, let them have their own salaries cut by 20%. But even better than waving paring knives around the schoolhouse would be genuine collaboration based on respect, the Five Wits, and the collective experience of the human race.

Diane Ravich reports that one special school in New York has had remarkable success in teaching disadvantaged children. Unlike the Grand Central Academy, this school has boarding students and a highly structured program in small classes. She notes what this school does not often report: that the assistance it receives from private foundations allows it to spend in the neighborhood of $35,000 per student per year.  This is about what the Phillips Exeter Academy charges. [Note: as of 2022, Exeter charges $59,000 per year.]

Common sense, imagination, and experience tell us that the commitment of ample resources to teaching in a program using methods that history shows have worked will be more productive than teaching on a shoestring in a hangar, which history shows has not worked. If I were a betting man, I’d rather put my money on Exeter and the Harkness Table than on Grand Central with beanbags.

Look, up in the office: it’s a bandwagon! It’s a juggernaut! It’s education by the numbers with accountability! This movement, combining the best managerial principles of Lady Bracknell[1] and Marshal Stalin[2], says that any statistics are better than none, that statistics are normative not descriptive, and that when a principal or teacher is shown by dubious statistics to be doing, or presiding over, a bad job, the best decision is excision.

But there are alternative and truly effective ways to produce good results, and so here are stories about people of my acquaintance—principals, other administrators, teachers, and, yes, even a consultant, who worked outside the Bracknellian-Stalinist paradigm to bring good education to their schools. Historians, anthropologists, and parabolists all know the educative and explanatory power of good stories judiciously told and used, so why not educators?

One day back in the 1990’s my principal and I went to the Orange Farm “informal settlement” (shantytown) southwest of Johannesburg. We were in a large truck carrying unneeded school supplies to the Leshata Secondary School, which had made news because its graduates had scored between 95 and 100% passes in the “Matrics,” South Africa’s comprehensive school-leaving exams. The results were unprecedented, but after we arrived, I came very quickly to understand what was happening.

The Principal, Mr. Moeketsi Molelekua, welcomed us at the gate. When we came in, we discovered that the students, all in uniform, were coming out to the school’s assembly-ground (no auditorium: too expensive) to greet us too. The students, most of whom lived in shacks without heat or electricity, stood in files to gave us a traditional black South African welcome with thanks: they sang in Sepedi, the predominant mother-tongue. The singing was joyous, full-throated, and beautiful. My principal and I briefly thanked them for their welcome, and they listened attentively and silently even though (we knew) most of them could not understand American-accented English.

After this welcome, than which a more moving one could not be imagined, Mr. Molelekua immediately gave a speech that turned out to be more moving. Within about ten seconds I understood why this school was getting 95% pass rates. Mr. Molelekua had an extraordinary charisma, forensic power, and eloquence, which he used in the service of his belief that education would transform South Africa and improve the lives of its people, in particular the students standing before him.

During out talks after the assembly (his office had his own modest desk, two plastic chairs for visitors, his assistant’s table, a small copy machine used sparingly, and a computer without an internet connection), I realized further that Mr. Molelekua’s belief was strong and genuine. He himself achieved a university education under the old regime in spite of economic and legal impediments while holding down a full-time job and supporting his family, and he pictured his students as in some very important ways exactly like himself.

He and the teachers kept the school open after hours into the evening on school nights so that the children, who often had no light or heat to study by and were crowded with their families into single-room shacks, would have a place to do their work. Believing that parents’ help and advice was essential to the operation of a good school, he would regularly see up to forty parents a day, seeking them out when they didn’t come to him. He invited parents to help him choose new teachers, and he accepted their advice. He visited the neighborhoods served by his school, where he was well known to parents and other members of the community.

In short, he was an educational leader, not a businessman. While I doubt that any administrative credential could confer on anyone the qualities that Mr. Molelekua brought to his calling, I feel saddened and upset that in the U. S., states and school districts seem to be moving away from the belief that educational leadership, however recognized, is what is needed in a principal or a superintendent, and instead that a programmer or a publisher with a plan and a knife will know better how to run a school or a district.

(The sad turn on “nothing gold can stay,” if my information is correct, is that Mr. Molelekua is no longer at Leshata and that the school’s results have sunk to ordinariness. I hope it is not true. Part of the reason that “bliss was it in that dawn to be alive” in 1990’s South Africa was the unpretentious but white-hot conviction of (extra)ordinary people like Mr. Molelekua that their decisions and actions would make a difference in their and their children’s lives, and that these beliefs, not statistics, were their guidance. So dawn goes down to day.)

But good education need not be the result of decisions by administrators. I had the happy good fortune to work at a high school that over a period of eight or ten years developed a remarkable productive collegiality among teachers. It was as if a roomful of Messrs, Mmes, and Mss Molelekua found ways to transpose their convictions into good teaching and learning without a single administrative ukase. Our principal generously kept to a laissez-faire style of running the high school because that was exactly right given the conditions. Some of the credit was due to the ingenious consultant Mr. Martin Skelton, for his approach respected teachers and gave them the time they needed to embrace his suggestions, which included a plan for the evaluation of teachers not by administrators but by their colleagues. We also devised a program of “writing across the curriculum.” We had a wonderfully varied offering of “activities” for our students in addition to the core of academics. My own contributions included help in the revamping of the English curriculum, work on the adoption of reading and writing assessments that we devised and graded ourselves (we used them to assess our programs, not our students and certainly not our teachers), supervising the adoption of “writing across the curriculum,” and running an outreach program to a nearby orphanage. I also set up a “TOK trip” on which students taking Theory of Knowledge camped by a riverside and mixed fun with work on their culminating “TOK Presentations,” which were given in a conference setting outside the classroom. Ms. DM, the I.B. Coordinator,  taught physics, rode horses, and climbed Mt. Kilimanjaro with students when she wasn’t running the I.B program. Mr. JW established an international Model United Nations conference. Mr. BK, the Activities Director and P.E. teacher, helped run the MUN conference, set up a program in which NBA players met with local disadvantaged students in a kind of basketball clinic, and organized invitational tournaments in I forget how many sports. Mr. OF, an Olympic medalist with a Ph.D., made student government real for many years, except one, during which XX, a clever and energetic stinker, was the student body president.

It is important to see that the good work described here has nothing to do with strategic planning or business models. It has nothing to do with statistics. It has nothing to do with elimination of undesirable elements. Instead, it has to do with educational leadership, whether by inspired principals or by devoted teachers. It has to do with verve in the service of deeply held beliefs. It has to do with respect and nurture that regard the materials they work on as live and fragile and not as lumber to be hammered, nailed, or burned.

---

[1] “Statistics … are laid down for our guidance.”

[2] “No man, no problem.”

Now that students in three Chinese cities have gone to the head of the international class in the PISA tests, we are starting to see coverage of Chinese education, much of it tinged with a trace of sour grapes. 1) “Schools in Shanghai are better than the average school in China.” 2) “Chinese students all learn by rote.” 3) “Chinese students are concerned only with the test.”

1) This must be meant to suggest that China’s scores wouldn’t be so high if they included test-takers from smaller cities and rural areas and that picking and choosing cities gives an unfair picture of Chinese education. That’s true, but it’s beside the point. Comparing China and Europe: we have focused on Finland’s schools as establishing a world-class standard of education without worrying that other schools in Europe are not as good as Finnish ones. Finland’s five million people run better schools than the rest of Europe, so they may have lessons for us. What about Shanghai’s twenty million? Or is the point of PISA testing a competition?

2) This may be true, except when it isn’t, but it bears more investigating. In a pedagogy that recognizes the differences among knowledge, skill, and understanding, rote learning has its place and should not be sneered at per se. The question should be Is rote learning used appropriately and with discretion? A refinement of that question: Is it complemented by coaching (skill) and Socratic probing (understanding)?

3) It is probably true, though not universally. The poet Du Fu (712 – 770) took the Civil Service Examination twice, failing both times. The failures didn’t keep him from writing over fifteen hundred poems, and at one point he even worked as an education bureaucrat, writing test questions for exams, a job he hated. One explanation of his first failure is that he refused to modify his dense style to suit the tastes of his examiners. The generally accepted explanation of the second failure was a case of result-rigging by a suspicious office-holder. But even if Du Fu is an exception that proves a rule, how different is that rule from the one governing test-preparation in the Land of No Child Left Behind?

Let me add some observations that are the result of having taught at a high school in China:

•       Chinese students are strongly inclined by upbringing to respect their teachers. September 10 is the national Teacher Day, and it is widely observed. It is impossible to explain briefly how this one attitude washes away mountains of crap from schools.

•       They are highly mindful of their studies. Electronic entertainment and distraction are making some inroads, but generally, when Chinese students study and pay attention, they do so successfully. This success is the result of good habits’ being instilled early and proved often.

•       Their parents are generally the allies of their teachers, supporting teachers and encouraging their children to do well. At parent conferences the student often attends, sometimes translating English into Chinese—accurately, even when the teacher has a reproof to deliver.

•       They tend to see schooling as a mission to accomplish rather than an annoyance to be endured.

•       They are sixteen in many of the same ways that high-school students are sixteen around the world.

May your classroom be full, but not too full, of eager students.

May your students not be jaded.

May they have had a good night’s sleep.

May they greet you when they come in and bid you goodbye when they leave.

May they look you in the eye but not get in your face.

May they never say “whatever.”

May they get their work done—by themselves.

May their parents appreciate what you do for them.

May your lessons fall on keen eyes and ears.

May your bag of tricks be bottomless.

May the only added value in your life be the value added to your abundantly deserved retirement accounts.

May your administrators be educational leaders and decent human beings.

May they appreciate teaching more than a business plan.

May your professional relationships with your colleagues be fruitful and productive.

May nothing in your building leak.

May your school’s network work.

If these wishes do not entirely come true, may you be possessed of the serenity to accept the human condition and the keenness to relish what you have.

As the seniors were winding up the writing of their applications to college, an article came out in The New York Times wondering whether going to an “elite college” was worth the cost. The main determinant in the answer appeared to be how much money a graduate could expect to make ten years after getting out.

Well, I am a teacher, and so I can’t have been worried about that. Maybe my values were skewed because “the cost” to me meant something different from what it usually means to a student (and his parents). One reason was that thanks to an extraordinarily generous scholarship, most of my expenses were paid; the other was my need to pay the rest myself. You may make what you like of this disclosure, but I can say confidently that I never worried about what I would be making ten years after getting out of the “elite college” I went to.

That seems to be a minority outlook today. The problem is that if Mommy’s Little Bean Counter wants to know the “return on investment” of a university education, there is little definitive to say. Yes, people who go to “elite colleges” make more than people who don’t, and the differential appears to be growing, but that is not the end of the story. According to the Times article, statisticians have also discovered that high-school students showing broadly similar quantifiable markers of ability and promise seem to get broadly comparable amounts of money in their careers regardless of the college they went to.

This got me thinking. When I entered college, the population of the U.S. was roughly two hundred million. Now it is a bit more than three hundred million: it has increased by fifty percent. Compare that with the growth in “elite colleges”: of the twenty-five national universities and twenty-five liberal arts colleges ranked highest by U S News, only five have been founded in the last hundred years, and four of those are in a single consortium, the Claremont Colleges in California. Not a single one has been founded in the fleeting years since I went to college.

My students look at me as if I were an antediluvian monster when I mention the gadgets that didn’t exist when I was their age, but I am positively up to date when I say that College X had a great reputation when I was a boy. There are half again as many kids to apply to the same fifty schools. Did they expand their classes? Some did, but not by 50%.

Add to that increase the number of kids who have succumbed to the Brand Anxiety Disorder (university strain), and these fifty must be far more selective than they were when I was trying to choose among them. The students they accept now are so extraordinarily accomplished that even when dim descendants of bright old names are factored in, these colleges’ classes would comprise individuals having generally much higher markers of promise and ability than formerly. So even the increase in income their graduates eventually register could be due entirely to demographics and not “elite” education. People who want return on investment would apparently be well advised to become fireballs and apply to a relatively cheap college with a good football team and strong alumni support.

The strange thing about the Times article was therefore how small a part anything but “return on investment” played in the discussion of which university to choose or whether “elite colleges” were worth the price. It did mention that within any university some departments are much stronger than others—so much so, said one former admission officer, that “there’s more variability within schools than between them.” That should add some complexity to an already baffling business. It suggests that even though some universities or departments may have better facilities than others, that explains only part of the difference between “eliteness” and “non-eliteness,” and maybe not the biggest part.

Universities and their departments, like high schools, seem to fly, or to sink, in new buildings or old, though of course, all other things being equal, great facilities would confer an advantage. But what are these other things? When I was applying to colleges, the University of Cambridge’s physics department was still housed in the old Cavendish Laboratory, built almost a hundred years earlier. I didn’t apply to Cambridge, but at the university I ended up attending, the physics department, one of the best in the country at that time, was housed in a building constructed forty years before.  What is true of physics is a fortiori true of pencil-and-paper departments.

An exchange occurred at Columbia in the mid-1940’s. Dwight Eisenhower, the new president of the university, referred in an early communication to the faculty as “employees of the university.” The physicist I. I. Rabi answered him, “Sir, we are the university.” Eisenhower accepted the correction.

This recognition suggests another, better, reason for attending an “elite college” than that it will produce a “return on investment.” Rabi was known for chalk talks with coffee in paper cups among professors and students. What I have read about them suggests that they were among the best conversations it was possible to have at that time in physics. The attendant “technology” was not advanced nor the facilities elite, but instead primitive and demotic (chalk, coffee, boiling water, paper cups) and didn’t make much difference. People had these conversations not because they expected a “return on investment” but because the talks were exciting and because they advanced learning, developing it and shaping—educating—the people who did the talking. It is hard to imagine Rabi being energized, improved, or made more effective by being subjected to a value-added learning audit or being appreciated because of participating in a horizontal study of the earning-potential of his graduate students. I think a good college or department, like the Porch of ancient Athens, becomes distinguished in time by a growth and maturation that cannot be predicted or forced, though it can be given opportunities to grow and mature in an environment where the faculty is treated with the respect due to it as being truly the school. I believe that a kindred organic complexity, also largely unpredictable, is true of high schools; they too can be nurtured in the hope that a desired efflorescence might take place.

It may be argued that high school teachers are in a different position[1], and it sometimes feels very much as if we are, but the palpation-and-command crowd would do well to take a careful look at how “elite” schools became elite. It had nothing to do with business models or five-year plans. The studies referred to in the Times article suggest that what makes them valuable is not value-addition, return on investment, business plans or branding but something less tangible and, to judge by the slowness with which schools establish themselves as “elite,” less subject to command than to patient alertness and nurture.

---

[1] See my first definition of “position” in my posting The Devil Made Me Say It

That is the arresting title of a paper by John Ioannidis, an epidemiologist at Stanford. Even more fascinating is Professor Ioannidis’s discovery that of thirty-four randomized controlled studies published in three medical journals and later replicated, the results of forty-one percent “had either been directly contradicted or had their effect sizes significantly downgraded[1].” It turns out that experimental results are subject over time and replication to “the decline effect,” whereby initially astounding or superb results become less remarkable or even unremarkable in successive tests. If all this weren’t enough to induce terminal modesty in experimenters, there is in addition the widespread problem of publication bias and other forms of prejudice that lead experimental research wrongly in expected or desired directions.

I would not want to say that research is useless—perish the thought[2]!—but I would  say that it is subject to many of the same kinds of error as “less scientific” ways of thinking, and a few that these less scientific kinds are proof against. William James describes a psychological problem he calls “mental vertigo,” in which the sufferer enthusiastically embraces propositions without any of the caution that a good education and wide experience can confer on someone with a generous natural endowment of common sense and critical intelligence. Because “research” is commonly and incorrectly supposed automatically to provide authoritative results derived by foolproof techniques, it actually can set up credulous people for a case of mental vertigo.

It is therefore unsurprising yet deeply disturbing to contemplate the exuberant and uncritical acceptance accorded by educationists to research findings of dubious sensibility or even doubtful sanity, followed by the findings’ vanishment in the same kind of silence as the kind that greets a faux-pas at a garden party. Do you remember the research showing that students don’t learn any more in enriched classes than in ordinary ones? Do you remember the research showing that teachers have no discernible effect on the learning of the students in their classrooms? Do you remember the research showing that classrooms should not have walls? Do you remember the research ramifying the dominions of the left and right brains? Do you remember the research showing that there is no transfer effect? Do you remember the research endorsing “whole-language” learning? Do you remember the research showing that team-building activities are effective, and the research showing that they are ineffective? Do you remember the research showing how English was supposed to be taught as a second language twenty years ago (and do you remember the whirligig of successive acronyms to describe the students who are learning it)? Do you remember the research showing that everyone should be taught to write as if he or she were a gifted and talented writer? Do you remember the research supporting “New Math” instruction? Once thought earth-shaking, these research results have fallen into oblivion.

A sign in an Austrian restaurant where I used to go during my college years said, “Ve get too soon oldt und too late schmardt,” of which a special case consists in getting excited about experimental results that, older and wiser, we laughed out of the room. But some educationists, ostensibly mature, “get never schmardt.” Off they go behind the latest Pied Piper, ready to jump again into the River of Educational Innovation. Unfortunately, they are not just eternally young and foolish; they are also undead, and they keep coming back, compelling teachers to adopt the next fad.

And what will that fad be? America’s dikeless Low Countries of Learning seem to attract their unfair share of inundations, I mean innovations, and I have dealt with a few of them in prior postings. But one potential Eternal Truth of the Year is suggested by a study in which the researchers spent $45,000,000.00 to discover that students can tell a good teacher from a bad one.

I am glad to hear that the expenditure of $45,000,000.00 has ratified a truth that I have known since I was nine years old, for now I can repose in the stability of research results. Or can I? Though the subtitle of the New York Times article in which the study appeared was “Ask the Students,” that is not what the researchers actually did. Instead, they required them to answer questionnaires by checking/ticking canned comments, identifying whether the comments applied to their teachers. We are assured that a Harvard researcher who has spent ten years refining student surveys is the designer of this one and the author of the potted replies.

That set off warning bells. How do the researchers guarantee that the choices allowed for students’ response are not tendentious? What does the reporter mean by “refine”? Does he mean “get bugs out”? What are these bugs that need ten years to get out, and how do we know that they have been got out? Does he mean “subtilize”? How do we know that crudities and gaps do not remain in the picture the questionnaire draws of the effective teacher? How do we know that it identifies more than—or no more than—a few middling marker-techniques of quality?

This last is important if we are to avoid question-begging. What marks the techniques as effective? If we say that it is their success in “adding value” to teaching and then show a correlation between them and value-addition in order to validate them as components of a questionnaire, we are making a circular argument. It is also important if we are to avoid publication bias, one of the intellectual vices described in the New Yorker article, or if we are to avoid a walk in the Garden of Interlocking Assumptions, where so many of education’s mutually self-confirming studies take us. And it is important if we are to establish a truth that does not wear off.

Roger Bacon said that experiments are necessary because they “put nature to the question.” This remark is usually quoted approvingly even though “put to the question” means “torture.” One thing we should have learned in the seven hundred twenty years since Bacon and the three hundred seventy years since the abolition of the Star Chamber is that torture guarantees only that we will hear the answers we want to hear from the tortured victim. Now, Nature, when questioned experimentally, does not necessarily scream answers on the rack. But as the New Yorker article suggests, sometimes she does when asked in a biased or loaded way. I think it reasonable to assume that the intellectual vices that sometimes vitiate the results of experimental science can also vitiate those of experimental education. This was one reason why Richard Hofstadter was leery of accepting the results of experimental psychology over the “collective experience of the human race,” something that historians, not experimentalists, are qualified by temperament, training, and experience to discover.

Such generalists are also qualified to recognize a good teacher. If school administrators had received a sound liberal arts education completed in a place that requires academic residency with its attendant humanity, instead of training as a specialist in educational research—or, more commonly now, business and finance—they could sniff out good and bad teachers. It also helps not to corrupt the process of evaluation. If it is true, as the Times article states, that most teacher evaluations consist in giving full marks with only cursory awareness of what a teacher does in the classroom, then it goes against my own experience, but it also argues, perhaps vainly, for a remedy at the administrative level of reliance on such old-fashioned intellectual virtues, established by the collective experience of the human race, as honesty. In a climate of educational corruption in which single schools can graduate nine valedictorians and teachers can spend all their teaching time prepping their kids for standardized tests, maybe we should not be surprised, though we should be ashamed, that they harbor shoals of perfect teachers when their students cannot muster even average scores on the PISA tests. This is not a problem of bad teaching; this is a problem of educational leadership. Unfortunately, the remedy proposed in “value-added learning” says that these same administrative structures that cannot sniff out a bad teacher, will know how to remediate one when he or she is identified by a statistical technique of dubious value. I guess that they will end up doing more termination than remediation if Diane Ravitch’s discussion of New York’s District Two[3] is any guide. Just what is needed to attract the teachers of tomorrow: make schools like Stalin’s Ukraine. Why didn’t we think of that before?

But if, as the Times article suggests, we have the evaluation of teachers by their students to look forward to, we can also expect another instance of the applicability of Campbell’s Law of corruption. We have seen how schools have responded to No Child Left Behind by gaming test preparation. We can therefore anticipate what will happen when Educational Science, with all its intellectual shortcomings, has spoken and the student has been installed as the evaluator of his teachers. To go full circle: it is suggestive that Professor Ioannides, mentioned at the beginning of this posting, is an epidemiologist.

---

[1] If you subscribe to the digital edition of The New Yorker, look at the December 13, 2010 issue on page 56 under an article by Jonah Lehrer called “The Truth Wears Off.” Otherwise, ask your subscribing friend to print the article for you.

[2] Consider, for example, Jerome Bruner and Neil Postman’s brilliant experiment using playing-cards with red spades, cited approvingly by Thomas S. Kuhn in The Structures of Scientific Revolutions in his explanation of how shared paradigms (like the Garden of Interlocking Assumptions?) actually shape our perception.

[3] See Chapter 3 of The Death and Life of the Great American School System.

When a teacher gets sick, as I have just done, he sees his work in a special light. Examine a teacher’s job and consider what happens when the teacher has to take time off, and you will discover, no surprise to teachers, how difficult the vacancy is to fill or the teacher to replace.

When I was doing my student teaching, there was a kind of teacher disparagingly referred to as Mister Ditto. This teacher had cabinets full of potted lessons in “ditto masters,” an obsolete kind of printing now superseded by photocopying. Every day Mister Ditto would take out a master, print a class’s worth of worksheets, and hand them out. The students would quietly fill in the blanks on the ditto and turn them in.

I am sure that only Mister Ditto could calmly foresee an absence from class because only he could conceive of lesson-planning in this weakly constituted way. My own reaction to the prospect of an absence is always less serene. The reason is that unlike Mr. Ditto, I conceive of learning as of three kinds—knowledge, skill, and understanding—of which only one, knowledge, responds to the Mr. Ditto treatment. When I am gone, who will do the needed coaching in skill? Who will probe with Socratic questioning for an understanding? Who will take on the class as a live work in progress?

Who can mark writing, knowing just where every student is and what kind of encouragement and reproof he or she responds to? Who can work against the inevitable tendency of students to regard substitute teaching as a holiday?  Who will adjust the lesson plans to account for the reality of the learning of the day before?

I referred to one of my substitutes as the Visiting Fireman. He actually was a fireman, and he subbed when his duties at the firehouse allowed. Nice guy, and he had a great story about responding to a fire alarm at the Seven Seas Bar, where the customers would not leave their drinks to evacuate. But not much could come of the lessons he supervised. Another sub was the Reverend. A preacher, he used his substitutions as a chance to transmit the Wisdom of the Ages to “his” classes. The lessons remained undone.

My lesson plans during an absence usually ended up being a lot of Mr. Ditto stuff (or its equivalent in more modern technology), with skill and understanding on hold. Even knowledge became a poor relation to simple classroom management. At some level the students would recognize this. The worst of them welcomed the holiday from productive work and learning, but many, maybe even a majority, had a sense that under normal conditions what they did in class was more worthwhile than what happened during a teacher’s absence. This is probably the explanation for students’ welcoming me back warmly when I returned from my illness. It also suggests the importance of a regular human teacher in learning. Mechanized lessons may be handy, but they do not satisfy students the way a real live teacher can do.

Anyone who has graded essay exams knows that not just educators are capable of baloney[1]. Our own dear students sometimes lay it on with a trowel.  We might pass off baloney as a bit of survivalist exuberance except that we rely on essays, or should rely on them, to help us gauge what our students have learned. How, then, shall we insure that our students do not write it?

We begin with the inculcation of the belief that writing, like all communication, is for an audience and that the audience must be satisfied with our production in order to credit it. In the case of an essay, students should have in mind a teacher or some other respected or admired person as the audience. The respect helps propel students in productive directions; they should thus come to a writing task experienced in respecting teachers so that it doesn’t feel like an irksome pose struck awkwardly on special occasions. It is more difficult to produce baloney for someone we respect than for someone to whom we are indifferent or whom we despise.

(I suspect that one reason for the horrifying content of some material written for or posted on the Internet is that it has been detached from a sense of particular audience and the reaction such an audience might have. A converse problem is with the writer who has three audiences and doesn’t keep them straight: the person the posting is about, a claque of admirers, and  the entire wired world. It therefore helps not just to have respect for a particular audience but to ground writing in a sense of decorum before a general audience too.)

Also essential is the belief that it matters whether or not the teacher is satisfied. Indifference aborts good writing until a student has become so proficient in both writing and respect that he can do without the affective assistance of a particular reader before his mind’s eye. I have said in earlier postings that the use of pointed and particular comments is a very good way of showing that we are engaged by a student’s work: that show of engagement encourages the student to work harder at writing. A test therefore seeds improvement in its successors if the teacher has reacted to it properly: it stops being just a test and becomes another lesson.

When the teacher’s satisfaction comes from a student’s good understanding and grip on factual detail as displayed in good writing, we have an ideal setup for the evaluation of an essay. This threesome should be the basis of evaluation of any answer, and a good essay should show all three. Nonetheless, though we may break writing down into its components for analysis, satisfaction is unitary and should result in one grade, not separate ones for “content” and “writing.” Good writing does not just ornament or frame what is said; good writing is what is said. It is not just that “writing is an act of courtesy”—though writing is certainly that—but also that writing is an act of realization. Our satisfaction as teachers comes from seeing that our students have gained knowledge and understanding and that they can communicate that understanding skillfully.

If we have done so, we will fulfill another essential condition, for we will not be bamboozled as was a “section man” at Harvard by the essay of the Abominable Mr. Metzger, who larked an exam in a course he hadn’t taken and got an A- on the essay. Though Mr. Metzger was clearly an accomplished thinker and writer of a kind, his “section man” could have—and should have—smoked out the imposture by insisting that he make his understanding operate on real, particular, factual stuff, and should not have been satisfied with airborne generalities.

William G. Perry, reporting on Mr. Metzger, called the story an “amoral fabliau,” and that characterization points us in the direction of how to regard baloney. I would go a little farther and call Mr. Metzger immoral, for writing baloney is a kind of fraud: the writer is pretending to the appearance of understanding, knowing very well that he does not have the real thing. Though Mr. Metzger’s sin was venial—he was not writing in order to gain official credit for a body of knowledge he did not possess, the scandal was a passing phenomenon not permanently destructive, and, as Perry points out, Mr. Metzger brought to the question a considerable mitigating nimbleness and power of expression that we are unlikely to find in garden-variety high-school baloney—it was wrong for him to have done it. For us and for our students the rule should also be that it is wrong to produce such essays.

I have spoken before about the need for some kind of emotional or affective tie between student and teacher as enabling certain kinds of understanding[2]. Among the most important is the understanding of the need for satisfactory work. I think this virtue is taught in part by the kind of personal connection—and by this I don’t necessarily mean a connection of friendship— a good teacher and a good student can establish. If the good teacher has somehow made it clear that he enjoys reading good essays and dislikes reading baloney, the student has one more motive for producing them. Eventually students who have produced good essays for teachers they like will be able to produce good essays for teachers who leave them cold or inspire dislike. I tell my students that I’d like an evening or a weekend of good reading—who wouldn’t?—with the result that I don’t get too much nonsense, at least not after they get to know me. The one or two padded horrors produced from bad motives are far outnumbered by the better productions of students who start by wanting to please their teacher and learn thereby to express themselves well.

If the right preliminary beliefs and feelings are in place, students can then begin to gain an understanding of what a good—and a bad—essay is. High-school students are ready to see and learn from exemplary work, and by that I mean examples of both good and bad writing. One piece I used with great effect was an article from The Economist in the early 1990’s about the effort to eradicate the Guinea worm. It began with horror  and ended with hope, proceeding along the way to discuss clearly and intelligently how the worm propagates and how it might be fought. Another, for advanced students, is the paragraph from A.C. Bradley’s lectures on Macbeth that begins, “Darkness, we may even say blackness, broods over this tragedy.”  Other good examples can be found without much effort, but examples of baloney are regrettably too easy to get—some even from teachers! And the examples can come from students themselves (I mean good examples here). I had a bulletin board marked GOOD WRITING! where I placed fine essays. Students were usually delighted to get their work posted there because they knew I didn’t put up poor work; and everyone would leaf through them to see “what so-and-so did.” I found a stamp of a fist with thumb up and would put “two thumbs up” on posted essays, a mark that some students came to desire too.

Finally, a teacher, in order to complete the lesson, must have a certain moral courage in order to say to a student, “This is not very good.” We must proceed from Dr. Johnson’s maxim that “he who praises everybody, praises nobody,” and in a clear but humane way let students know how their work falls short. It may even help to reprove sharply the author of a clearly careless nonsensical mess. I have occasionally drawn a red line across the page between two lines of a very bad essay and said, “I stopped reading here.” If a student expects a teacher to read an essay, then a teacher may expect a student to produce an essay that is readable: expectation must be reciprocal.

A great way, if time permits, to minimize baloney is to supplement a written examination with an oral. It is difficult to pass off baloney to a teacher’s face and impossible to prevent it from being questioned. If time doesn’t permit, another possibility is a post-essay conference with a student in which the teacher asks, for example, “What do you mean by ‘insights into the contingent possibility of structure’?” Few students will write nonsense if they think they might be called on it face to face. If one of them is an Irwin Corey, we may laugh, but not raise the grade.

---

[1] See my postings Baloney Bingo and The Devil Made Me Say It.

[2] See Understanding Understanding

In his dark, bitter, and wonderful essay “Such, Such Were the Joys,” George Orwell reports on his boyhood prep school[1], St. Cyprian’s, which he attended before and during World War I. His stories about bullying, poor food, shabby facilities, and overbearing administrators suggest a slight but disquieting kinship between his Edwardian school and the schools of our own time, yet in one respect they are close kin indeed.

That is in their teaching to what we now call high-stakes tests. In his day they were the entrance exams for Eton College and the Harrow School or the exam for the Harrow History Prize; in ours, proficiency exams mandated by No Child Left Behind and RAce to the Top. In both cases I think we may use Orwell’s excellent description: “a sort of preparation for a confidence trick.” The trick is that both Orwell and his modern counterparts are gorged like geese with “a series of unrelated, unintelligible but — in some way that was never explained to us — important facts” (Orwell’s words) that might, with some happy guesswork, lead to a satisfactory score on a test without regard to whether a coherent subject was actually being taught. The reason it is a confidence trick is to persuade the test marker that you know a coherent subject, when in fact you only know a bag of facts.

In the history chapter of The Paideia Program Professor Barzun proposes that in teaching history we start with “once upon a time” and “many years ago,” proceeding in the study of history until, at the age of 13 or 14, we start to draw on “casual noises from the big world and unite them into an increasingly coherent recital of their meaning, which is to say their origins and purposes.” At the same time, while keeping the element of the story in history, the course shifts to “the essence of genuine history,” which is continuity.

These qualities of history were ignored or undercut in the kind of preparation Orwell received, which consisted mainly of going over old exams and conning answers from them irrespective of storyline or continuity–in much the same way people “prepare” for high stakes tests today by learning shovel-loads of data. Orwell reports that the initials of the mnemonic “A black Negress was my aunt: there’s her house behind the barn” correspond to the initials of the battles of the War of the Roses (1455 – 1485). Does the study of detached mnemonics differ in any important respect from the discontinuities experienced by contemporary children studying test-preparation books? Not really. The main difference I see is in what courses are shortchanged in testing mania. In today’s climate it means that history, any history, is likely to be shortchanged in favor of English and math, the two playing surfaces on which the Three-card Monte of testing is now being rigged.

(That some subjects give way to others was the same in Orwell’s day and in ours. It is not just that the “black Negress” mnemonic would not be used today, but that we are throwing out history itself along with such mnemonics. In Orwell’s day science was thrown out—science! This in the youthful heyday of the Cavendish Laboratory. Changing fads of specialization suggest that the transfer effect is valid and that early specialization is not as necessary in education as some people and programs demanding it lead us to think. Let me therefore plug the idea, now increasingly alien in practice, of general or liberal education. Besides, if the kind of specialization implied in heavy “test preparation” worked, wouldn’t the U. S. have better results in English and math to show for it? If you think we do, look at PISA’s comparative results.)

Orwell, using his judgment to handle material he experienced directly, arrives at an insight generalized many years later by a social scientist. One of the “scandalous” facts reported by Donald Campbell in his paper “Assessing the Impacts of Planned Social Change,” which led him to formulate Campbell’s Law[2] in that paper, was that a private profit-making contractor, hired to administer a program of compensatory education in a public school system, gamed the system with a gamey program of test preparation (I discuss this in my posting ℞: Stone Tablets). The Texarkana schools couldn’t have known that they (or Orwell’s Flip and Sambo for that matter) were pioneers on the low road followed by the New York City Schools and other school districts of today in their efforts to “prove” student “achievement” in intellectually and morally suspect programs of testing-and-accountability.

A quick postscript on the accountability part of the terrible twosome. The Los Angeles Times has published lists of L.A. teachers found unsatisfactory in “their” efforts at “value-added learning[3],” and The New York Times is said to want to do the same for teachers in New York. One value that might be added to the consciousness, if not consciences, of people who want to give teachers such publicity, is a disdain for the kind of public shaming without trial that one associates with the Great Proletarian Cultural Revolution. Are published lists much different in their effects from yellow robes or dunce caps?

---

[1]That is, a grade school preparing its pupils for English public schools such as Harrow or Eton.

[2] “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”

[3] Dealt with in my posting  “Added What?”