Archive for June, 2011

Towards the Pebbled Shore

Friday, June 24th, 2011

 

Last week I examined the perplexity facing students caught between the demands of solid reading and the push to make learning into a kind of Quick Quaker Oats®. Since there is no way to resolve the conflict between these demands, I came down in favor of 1) complexity and 2) taking time, as my imaginary students did in that posting.

A major educationist publication[1] has also recently plumped complexity. To understand why this is both good and bad news, we must examine what it says. Briefly: educational testing shows that the strongest “differentiator” between students who met or exceeded “benchmark” scores and those who didn’t was the ability to answer questions “associated with complex texts.” It was not the ability to answer questions related to “cognitive processes, such as determining the main idea or determining the meaning of words and phrases in context.” Part of the bad news is that these are the kinds of question usually found in Testing and Accountability instruments and Aptitude or Achievement Tests.

What helps to make bad news of this is that some of our commonest “measures” of complexity miss the boat. Consider how “complexity” is often “measured” by “quantitative means”[2]. Most of these “measures” work by the mysterious substitution of “proxy values” for the actuality of complexity. These mysteries are arcane—some would say arbitrary—and they lead to questionable “measurements.” Take as an example Lincoln’s Second Inaugural Address, in which this sentence appears: If we shall suppose that American slavery is one of those offenses which, in the providence of God, must needs come, but which, having continued through his appointed time, he now wills to remove, and that he gives to both North and South this terrible war, as the woe due to those by whom the offense came, shall we discern therein any departure from those divine attributes which the believers in a living God always ascribe to him? I ran this speech through my MS Word Flesch-Kincaid Readability Tool, which analyzed it as having 8th-grade readability. F-K also rated Shakespeare’s Sonnet 60 (“Like as the waves make towards the pebbled shore”) as having 2nd-grade readability. Uh-huh.

The solution to problematic findings of readability like these should lie in the abandonment of a wrongheaded mythology of measurement in favor of connoisseurship and the cultivation of finesse and its application to the selection of reading, but that is not what our Appendix proposes. Instead, it claims that what we need is something called a Coh-Metrix report on readings, weighing them against more than sixty indices. Or, if sixty are “daunting for the layperson or even a professional educator unfamiliar with” them, further research may identify and “isolate the most revealing, informative factors.” I will await this research without bated breath.

Consider another issue visited by our Appendix. Showing how education research results occasionally leads to a valuable conclusion, it claims that students must approach their complex reading “independently,” that is, without lots of scaffolding, guidance, slicing, dicing, and “scanning for discrete pieces of information.” The Vegematic® approach to reading instruction so commonly used deprives them of this opportunity. If, as I said last week, reading is like coming to an unexplored country, we will not learn to be explorers when our trip is a Cook’s tour laid out in advance.

Consider by analogy the unexplored country of Pieter Brueghel’s Landscape with the Fall of Icarus. Well, it is perhaps unexplored by your students, who, let us say, must make something of it. If they are to explore independently, they may not run to the web for a study guide, because in doing so they give up their independence. On the other hand, they might need to hear the story of Icarus and to have their exploratory glance discreetly turned in a productive direction. Give them two more items: Matisse’s The Fall of Icarus and William Carlos Williams’s poem “Landscape with the Fall of Icarus.” After some quiet time of Google-free looking, reading, note-taking, and thinking, bring them together for a session of Shared Inquiry about the three items. By the time it is finished, the students will have done some independent but guided exploration of that country. What they produce may not be Meyer Schapiro or Simon Schama, and that is what will be good about it in the context of education: it will be their own best-exploring selves testing independent conclusions by discussing them with their classmates and you. You will know whether your students have struck gold during their explorations if they can say why Icarus might be the center of the Matisse picture while almost invisible in Breughel. But not every quest strikes gold, and that fact of life must be respected. There are times when the guide, if he is also a teacher, must forbear to yak about the lie of the land. (This does not mean the teacher is a constructivist: the teacher may guide exploration with tips and advice, propose ancillary readings, and subject findings and understanding to Socratic questioning.)

If the analogy holds, as I think it does, a course in complex reading allows students to encounter something unexpected and to deal with it in their own terms. With the added confidence and sophistication that experience brings, they will achieve a finer and solider understanding because they will have done it more or less independently. The I.B. course in Theory of Knowledge has been many things to many people. It has been many things to me! One that it has been to me is a course in complex reading. For many years I began and ended it with readings by William James. I had my reasons, but one of them was to prove something to the students. Every year one or more students would invariably oblige me by commenting that the last reading in James was much easier than the first one. I could then gratify them by saying, “No, they were equally difficult, but in the intervening year you became far better readers.” That is the good news.

 


[1] Common Core State Standards, Appendix A, p. 2, “Why Complexity Matters”

[2] ibid, p. 7

 

Homework Bites

Friday, June 17th, 2011

Flannery O’Connor didn’t have much patience with people who wanted to summarize fiction, including hers. If someone could “tell what it was about,” she thought, what was the point of writing it in the first place? To one audience she offered, as a parody summary of her story “Good Country People,” that it was “about a lady Ph. D. who has her wooden leg stolen by a Bible salesman whom she is trying to seduce.” The point was taken in laughter. To another audience she said that if the wooden leg was going to end up symbolizing anything, the symbolism would have to grow on readers as a part of their encounter with the story and not because someone had told them in a class. She knew that, her comments notwithstanding, stories usually do have subjects and themes and sometimes also symbols, but she knew that an encounter with a good story is radically different from, and better than, an encounter with a list of “elements.”  She was arguing for a kind of reading that places the work first and comes to it the way a traveler comes to an unknown country.

O’Connor died in 1964, the year I first saw Cliff’s Notes (first written in 1958 in Nebraska and nowhelpfullycalled CliffsNotes) at the local book store. I remember thinking it peculiar to buy one book whose purpose was to “tell me about” another book. In the years since, I have looked at a few of those “study guides,” but never finished one. Sometimes a summary can be good and funny. Take for example Desmond Skirrow’s “Ode on a Grecian Urn Summarized”: Gods chase/ Round vase./ What say?/ What play?/ Don’t know./ Nice though. That standard is rarely met.

It is sad to think that summaries and notes, already sweeping whole works aside, will become even more deeply entrenched in education, but that is likely to be one result of a movement gaining some currency to reduce homework loads in school. Under the formula commonly propounded, it should be limited to ten minutes per night per grade. This would mean that, for example, 8th-graders would have eighty minutes of homework a night and 12th graders two hours. In addition, no homework is to be assigned on weekends or over holidays.

In an anthology I have used with 10th-graders appears “Everything That Rises Must Converge,” a short story by O’Connor. Like Wallace Stevens’s unseen nightgowns, this story is strange and wonderful, though dark as O’Connor’s work usually is. It ends with Julian, the protagonist, running and crying “Help!” for his stricken mother against darkness that “seemed to sweep him back to her, postponing from moment to moment his entry into the world of guilt and sorrow.”

Let us imagine a tenth-grader at home facing this story. She is taking five academic subjects and is at a school with the ten-minute-per-grade homework policy. That means that, roughly speaking, she is “allowed” about twenty minutes at English homework per night. The story has about 6500 words. Assuming she can read 250 words per minute—and that exceeds the speed at which quite a few tenth-graders can actually read—it would take about twenty-six minutes of steady reading merely to get through the story, already well into two nights of homework. Having interrupted her reading directly before the crisis of the story, she backtracks and does a bit of rereading, actually taking the full two nights’ allotment to read the story.

To read it once, not twice. To read it without taking notes. To read it without ruminating on that last sentence: how can the darkness sweep Julian back to his mother? How can it postpone his entry into the world of guilt and sorrow? To read it without being able to use the reading to support her own answers to any of the five questions that appear in the anthology at the story’s end. If we want to allow our tenth-grader two hours to read and reflect on O’Connor’s story, we must allow six school nights for the completion of the assignment. Since homework is not to be assigned on a weekend, it would take her more than a week of often-interrupted work allotments to study the story. What is more, we have not even spoken about other work that might take place concurrently, such as writing or grammar. Let us therefore recklessly imagine as an alternative that she exceeds her time limit because she is fascinated by the story and, as a result, succeeds in working out for herself the relationship in the story of darkness, guilt, and sorrow.

Let us also imagine her elder brother, a twelfth grader allowed a generous twenty-four minutes per subject. He has been assigned to report on the arrest of Samuel Pepys (pronounced peeps) in 1679 for participating in the “Popish Plot” to unleash terror in England, including the assassination of the King. How many days or weeks would it take him to come up with the needed factual detail and a synthesizing assessment? Let us imagine him working overtime to do so because he too has become interested in the subject instead of his clock.

He discovers that there was no “Popish Plot.” He discovers that Pepys was not even Catholic. He discovers that in the court trying the case Pepys was not allowed to confront the lying witnesses against him or to impeach their testimony. He discovers that Pepys could be held indefinitely without charges and was in fact held without charges for weeks. He discovers that if convicted of the charges finally brought against him—without due process of law—Pepys would be hanged, drawn, and quartered. He discovers what that punishment was.  He ends up giving a report not just on Pepys but also on background to the U. S. Constitution’s protection of habeas corpus and due process of law.

Much of the good that our two students get out of their extra work would be lost if, constrained by unreasonable time limits, they had to resort to “study guides” in order to get “the basics” of the O’Connor story or “a lesson” in civics. These guides are not the same thing as what they purport to “study.” They do the students’ thinking for them, and they deprive them of the chance for a bracing encounter with salutary complexity. They are like food that has been chewed by someone else, given to students with the instruction not to take the time to chew it themselves.

 

Looking for Learning; Looking for Teaching

Friday, June 10th, 2011

Some years ago a school I then worked at adopted a program called Looking for Learning developed by a consultancy in England. Two of the principles at the heart of the program were that pedagogy must focus on the learning that is occurring and that the best people to evaluate teachers are their peers. There were many reasons to admire this program, including the lucid, jargon-free presentations of Mr. Martin Skelton, but one of them requires some explanation.

The program took the long view: slowly slowly catchee monkey. We teachers had two monkeys to catch. One was to learn to recognize learning when it occurred in a classroom, and the other was to trust each other enough to give and take praise and criticism honestly and helpfully. Tricky monkeys.

Recognizing learning takes more than one might think at first. To suggest why, consider typical classroom visits by administrators. They take a seat in the back of the classroom, watching what the teacher does and taking notes. If they favor the method of record-keeping called “scripting,” they are so busy writing that they hardly even look at the teacher. Consider by contrast the evaluation that an old established high school of my acquaintance gives to applicants for teaching positions there. Short-listed applicants are required to teach a lesson. Two observers, an administrator and a teacher, attend. One of them watches the teacher, and the other watches the students. Afterwards they discuss what the teacher did and how well students were learning as evidenced by their behavior in class. They jointly arrived at a hiring recommendation, which the headmaster typically accepted.

Now let us go back to the monkey. During the first two years, what did the teachers at my school learn from Mr. Skelton and his associates about looking for learning? Almost nothing: instead, and more important, they learned how to trust one another, an ability they needed to cultivate first. They—we—did so by adopting a teacher-driven improvement plan and then seeing it through without the oversight of any school administrator. Of course the ostensible object of the plan was to improve the school, but the hidden objective was to bring the faculty over a period of years to the position of being able to work trustfully and productively together. What we teachers did not know at the time was that Mr. Skelton was also working with the administrators and encouraging (or developing) their ability to trust teachers to evaluate. They showed their learning by accepting our plan and, later, when we started reviewing each other, by letting it proceed without hindrance.

The result was a system of teachers’ watching one other and their students, of sizing up what was happening in the classroom, and of making recommendations. In short, it was what we sometimes called “formative evaluation.” As the best formative evaluations always do, these at their best combined the essential ingredients of trust, judgment, and finesse.

Finesse! Or, as Professor Barzun has it, “perpetual discretion,” the ability always to make and use fine distinctions and to size up situations flexibly and accurately. How different in approach, process and effect from sizing a teacher up by using a value-added learning equation. It admits fine differences and distinctions. It is human and humane. It produces more than the three results OK, not OK, and fired. I think that “judgment” has turned into a dirty word in education partly because people have talked themselves into feeling exempt from judgment and partly because they have experienced judgments made according to unclear, arbitrary, or unfair principles. And trust is the great enabler that binds the other two.

It was therefore encouraging and gratifying to find trust building up at my school during the operation of the first part of our program, and it is encouraging to know that it is identified as an essential ingredient in Finland and the Montgomery County (Maryland) Public Schools. Two qualities of the Montgomery County program that stood out for me were the parity of teachers and administrators in the Peer Assistance and Review (PAR) panel they have and the emphasis they place on having developed the trust needed to make the panel work—a seemingly Finnish degree of trust, but grown and developed on American shores. As in our school’s program, trust took some years to develop in Montgomery County.

How much is that trust worth to Montgomery County? Though I hope they consider it priceless, it can be valued at $12,000,000—the amount the schools rejected from the federal RAce to the Top (RAT) program. To accept it, they would have had to abandon trust, finesse, and judgment, adopting instead a scheme of teacher evaluation based on value-added learning as assessed by test scores and formulas. Dr. Jerry Weast, the Superintendent of the Montgomery Country Public Schools, said, “We don’t believe the tests are reliable. You don’t want to turn your system into a test factory.”

You don’t want to turn your schools into any kind of factory, including a test factory. You don’t want to produce your teachers as if at a tool and die shop. You don’t want teachers graded by an on-off switch. Rather, you want to have schools characterized by the qualities constitutive of Looking for Learning and Peer Assistance and Review: trust, finesse, and judgment.

 

Ardors Wrenched Here

Saturday, June 4th, 2011

High schools, even those whose missions aim to produce “motivated global contributors” and “inspired life-long learners,” usually try to prepare their students for college. But what should a student be prepared to do in college?

In many cases, the answer appears to be “not much.” Louis Menand reports in The New Yorker on a study showing that the average college student studies thirteen hours a week and that a third of college students study less than five hours a week. If these incredible figures are true, they explain why nearly half of the students in the study did not engage in significant learning during their first two years of college.

(Now this study was itself the object of a blistering critique of its statistical methods, which included, as such studies so often do, an arbitrarily chosen standard of “significance.” Even one of its proponents notes that selection bias has not been controlled for and that the data are not randomized, and he notes criticisms of using tests as the basis for making “consequential decisions[1],” but he recommends using the test anyway. Why? I recommend that it be pushed away with a barge pole.

(I would also add that the Collegiate Learning Assessment, the basis of the study, is deeply suspect as an “instrument” for “measuring” learning or, as the test’s proponents themselves have it, “added value.” It asks students to demonstrate how they have learned in college by pretending to make an aircraft-purchasing decision on the basis of examining FAA reports and stories about plane crashes. I have said elsewhere that the transfer effect is real, but I have also said that it can take a long time for an education to ripen. Are the only fruits of education that matter the watery ones whose growth has been forced? And will colleges under the gun to produce students who can “think critically” on this test start requiring courses in procurement administration?)

The news flurry over the “worthlessness” of college hid an issue worth discussing: What kind of college should high school prepare its students for? The study mentioned above, flawed though it may be, notes that liberal arts majors “learn more” than majors in other fields like business, engineering, and computer science. Menand notes that liberal arts majors tend to have had better preparation in reading and writing than majors in other fields, and that the ability to read and write well tend to fit one for success in learning. Thus, in the “debate” between Bill Gates and Steve Jobs about what kind of education to pursue, Jobs seems to be right: the collegiate place to be is the College of Arts and Letters.

It is possible that some Colleges of Arts and Letters are suspect as loci of learning, but I don’t want to address that issue in any detail now. It is also possible that education in the US should stop thinking that one sort of education is suitable for everyone. Instead, it might consider adopting the European model of multiple tracks of education with schooling for some that culminates in A Levels or Baccalauréat or Arbitur, though my own preference would be for culmination in a variety of types of assessment such as the I.B. program offers.

In that case it would follow that academic high schools should offer preparation for a solid and rigorous liberal arts education. That would mean a comprehensive program of many complementary elements, and one that prepares its students to read and write well. It should be generous in the assignment of reading, writing, and other homework and inventive and various in its techniques of assessment.

All the prevailing emphasis on education’s instrumental value overlooks, however, an important result less easy to fit into the voc. ed scheme. When my students ask me what I thought of going to college, and I think they will not suspect my answer of being humbug, I tell them that it was like being born a second time. You can probably guess that this sentiment was not due to my having learned at last how to interpret FAA reports. Closer to the mark might be a line from Whitman that I first read in my poetry class with Professor Koch. It said of listening to an orchestra, “It wrenches such ardors from me I did not know I possessed them.” I could say that about some of the material I experienced in college. And just as Whitman must not have expected his every hour to be like an orchestral concert, I did not expect that kind of response to every item I encountered as an undergraduate. Professor Barzun notes that a simple count of buildings and their types will not reveal that Manhattan’s dominant building is the skyscraper. By analogy, a statistically based test of “value addition” will not reveal the dominant type of learning in a good college education. If I were to place a sign above the gates of an excellent college describing in five words or less what lies ahead, I would propose this one: “ARDORS WRENCHED HERE.”


[1] Such as that they violate Campbell’s Law