Probing the Depths of Students' Historical Knowledge
Samuel S. Wineburg, March 1992
They are a "generation at risk," hordes of youths "gravely handicapped by their ignorance" who lack the "perspective to separate what is important from what is trivial" or "what is durable from what is ephemeral." This characterization of today's teenagers comes from Diane Ravitch and Chester Finn, whose book, What Do Our 17-Year Olds Know? chronicles the "shameful" performance of America's youth on the "first national assessment of history and literature." Ravitch and Finn, who helped design the multiple-choice test used in this assessment, found that 66 percent of the students tested could not place the Civil War in the proper half-century; close to 40 percent thought the Missouri Compromise settled a border dispute; and only 45 percent could identify Samuel Gompers. They warn that unless we do something—and fast—the present generation will be unable to "stand on the shoulders of giants" because they can't tell "who are giants and who are pygmies."
If Ravitch and Finn are right, what can we do to save students from the ignominy they will face as adults, as citizens, and as parents? The Bradley Commission on History in the Schools and the National Commission on Social Studies in the Schools both recommended one way: require students to take more history. Ravitch and Finn recommended another: compel students to demonstrate "substantial amounts of historical and literary knowledge" before passing them to a higher grade. Through both proposals runs a common notion: If students possessed more historical information, they would be able to "make sense of what they see and hear" and "discern patterns in trends and events." In short, they would better be able to think.
But would they? Suppose that instead of studying "typical" or "representative" students, we purposely sought out "good" students, those who take history even when given the chance not to, who can date the Civil War or state the tenets of the Missouri Compromise, and who know that Samuel Gompers' association with the AFL had nothing to do with football. Would such students be able to bring their knowledge to bear in thinking about problems too messy to be scored by an automatic optical scanner?
I wanted to find out. With funding from the Spencer Foundation and the help of some high school teachers in the San Francisco Bay area, I recruited a group of highly capable seniors and had them work through a series of historical documents. To select students, I first administered a pretest composed of items adapted from the national assessment. (I chose only items that stumped at least two-thirds of the national sample.) Stringent criteria guided my selection of students: only those who had taken four years of history, who scored 50 percent or above on the pretest, who planned to attend a four-year college, who maintained a GPA of 3.0 or more, and who were reading at or above grade level were considered. I then relied on teacher recommendations to help me identify eight "top-notch history students" who together had an average GPA of 3.5 (two students had perfect 4.0s) and average SAT scores of 1268, far above those of "typical" college-bound seniors. And unlike student profiles emerging from the national assessment, these eight students said they spent about two and one-half hours a night on homework and read for pleasure in their spare time. To be sure, these were no ordinary kids.
Inspired by a pamphlet from the "Amherst History Project" of the 1960s, "What Happened at Lexington Green?" I culled eight documents for students to read—eyewitness accounts, diary entries, a deposition, a textbook excerpt, and a passage from Howard Fast's novel, April Morning. But before giving these to students, I taught them to "think aloud," a procedure that is being used increasingly in cognitive psychology and artificial intelligence. Essentially, "think-aloud" asks people to verbalize their thoughts as they solve complex problems or read sophisticated texts, and it departs from more traditional "experimental" research by focusing on the ongoing processes of cognition, not just on its outcomes. While the artificiality of this technique has drawbacks—for many, "thinking aloud" is a contradiction in terms and takes some getting used to—it offers a window into thinking provided by few other methodologies.
I purposely gave students contradictory documents that yielded no "right answer" and told them that their goal was "to understand what happened at Lexington Green on April 19, 1775." To better evaluate their responses and to provide a touchstone for judgment, I recruited a comparison group—eight historians from universities in the San Francisco Bay area—and engaged them in the same task. Among these historians, six were seasoned veterans trained at either Wisconsin, Stanford, or Harvard. To leaven the group, I included two graduate students from Berkeley who had recently completed their comprehensive exams and were in the middle of their dissertation research. Four of these historians had specialties in United States history; the other four had specialties ranging from British penal history to French art, from Asian history to the influence of Islam on the Karites, a breakaway sect of Jews in the Middle Ages.
When we compare how historians and students read these documents, we see dramatic differences on practically any criterion we select: the historians excelled in the elaborateness of understandings they developed, in their ability to pose alternative explanations for the events at Lexington, in their use of corroborating evidence, and so on. By themselves, though, these are rather trivial findings; of course historians would do better on such a task—they know more history. But what counts as "knowing more history"? True, the four Americanists knew more than the students about the events surrounding the confrontation at Lexington. But what about the Asian specialist or the medievalist?
To better gauge background knowledge, I quizzed each historian on factual items from a leading U.S. history textbook (Todd and Curti's Rise of the American Nation), items similar to those appearing on the national assessment: "Who was George Grenville?" "What was the Quebec Act?" "What was Fort Ticonderoga?" "What was the Battle of Saratoga?" "What was the Olive Branch Petition?" As expected, three of four Americanists knew most of the answers, but the fourth scored less than 60 percent. As for the non-Americanists, their average score was 35 percent, with the Asian specialist and the medievalist scoring less than 20 percent! With such low scores, how can we explain the fact that even these historians were able to, in the words of Ravitch and Finn, "discern patterns in trends and events" when a group of bright high school students got lost in congeries of detail?
The differences between historians and students had little to do, I think, with the discrete bits of information captured by a multiple-choice or short-answer test. Rather, they reflect systematic differences in conceptualizing the past, different casts of mind that transcend particular names and dates and even particular specializations and inclinations. The act of reading history does not go on in a cognitive vacuum but is informed by a rich network of beliefs, a powerful if rarely articulated epistemology of historical inquiry. A brief example will illustrate my point.
After students and historians had read the eight documents, I presented them with three pictures, each claiming to depict what went on at Lexington on that fateful April morning. I asked participants to say anything that came to mind as they reviewed these pictures, and when they were finished, I asked them to select the picture that "most accurately depicted what went on at Lexington." Historians typically bristled at the thought. They hemmed and hawed, changed their mind, and then changed it back again. Some even resisted making a choice, and wouldn't do so without appending strings of qualifications to their answer. The medievalist cautioned me that he would decide, even though it was impossible to decide. The British social historian told me the pictures were inconsequential and told him nothing about what was most important, namely, why were the British there in the first place? An Americanist selected one of the pictures, but first reminded me that "there are some inconsistencies, mind you" and proceeded to lay them out in excruciating detail—and so on down the line. Because I was persistent and because the historians I recruited were a fairly agreeable lot, all of them did, finally, select a picture. But their selection was more a suggestion than an answer, more a forced choice from flawed alternatives than a committed decision executed with resolve.
For students, the picture evaluation task was a different story. Rarely did they sift through the written documents, puzzle about the intentions of the artist, or reflect back on what they had read. Students generally sized up the pictures and made a selection without regret or qualification. For historians, the picture selection task was an exercise in exploring the limits of what could be known. For students, it was a multiple-choice test. Even when those in both groups agreed on the same picture, it was often for opposite reasons. For example, one of the depictions shows the colonists perched steep above a hill on Lexington Green. The specialist in Asian history selected this picture but qualified her answer by noting that this hill was nowhere mentioned in the documents. One of the students also selected this picture, but rather than qualify his choice because of the hill, this feature emerged as central to his understanding: "The minutemen are going to be hiding behind the poles and everything, rather than staying out and facing (the British). ... You know there's got to be a hill, and they're thinking they got to hide behind something, get at a place where they can't be shot." For this student, the confrontation at Lexington was simply an eighteenth-century version of television's China Beach.
Beliefs about history are also implicated when we compare how historians and students understand causality. The historians I studied saw events as caused by a multiplicity of factors, some part of carefully worked out systems of goals and intentions, others less planned, the result of fortuitous circumstances, unpredictable forces, or plans turned awry. Manifestations of these beliefs were legion in their comments—indeed, five of eight historians drew analogies to either Kent State or the 1968 Democratic National Convention, events like Lexington in which human intentions became mangled by fears run wild.
Students' beliefs about causality sometimes took the form of what might be called teleological fallacy, or the tendency to equate human intentions with historical outcomes. In this kind of thinking, outcomes stem from logical strings of antecedent conditions that make the final result well-nigh inevitable. We see an example of such reasoning in the comments of Darrel, an ambitious college-bound senior with a 4.0 GPA and SAT scores of 630 Verbal and 790 Math. The most articulate of the eight students I interviewed, Darrel "knew" the most history in the conventional sense.
Let us examine Darrel's comments in response to a letter sent by Joseph Warren to Benjamin Franklin, the colonial representative in London, soon after the events at Lexington. These comments show that Darrel has a keen sense of chronology—he knows, for instance, that Lexington comes a year before the Declaration of Independence; moreover, he knows his Declaration of Independence, particularly its long list of grievances against George III. From the start, though, Darrel distrusts Warren's claim that, "We profess to be (George III's) loyal and dutiful subjects," responding that, "This might be appeasement here, trying to say that we are loyal and stuff but be separated." Because the colonists do, eventually, declare their independence, Darrel sees this as their intention from the start. So, when Warren claims that "These (events at Lexington), brethren, are marks of ministerial vengeance against this colony, for refusing, with her sister colonies, a submission to slavery. But they have not yet detached us from our Royal Sovereign," Darrel wags the finger of hypocrisy: "It's really confusing here what he's trying to say; he's a tyrant and stuff, but they have 'not detached us from our Royal Sovereign.' Maybe that's why Britain really didn't take them seriously, because they did not state their real intent until the Declaration of Independence." In other words, independence was a foregone conclusion, not the result of a change of heart, which would mean that things could have been otherwise: the result of chance, which would grant fortuity a key role in human affairs; or the result of irrationality, which would acknowledge that we so often ignore what is in our best interests. In addition to being an example of flawed historical reasoning, there is another reason Darrel's reading should give us pause: It is an instance in which a student's background knowledge, rather than paving the way to understanding, actually gets in the way of it.
There is much to criticize in Darrel's reading, but we shouldn't be too harsh, for it is actually one of the more sophisticated examples among students. Darrel does something rarely noted among high school students: he constructs a willful author behind the text, an author who uses language to achieve purposeful social ends. Among students, this was a minority position; for most, reading history was not a process of puzzling about authors' intentions or situating texts in a social world but of gathering information, with texts serving as bearers of that information. Given materials they were supposed to learn, most students did what came naturally: they sat down and "learned the material."
And so, another curious finding: in almost every opportunity to do so, historians, when first given a document, looked immediately to its attribution, whereas students usually began with the first word in the upper left and never stopped until they reached the last word on the bottom right. For most students, the text's attribution carried no special weight; it was merely the final bit of information in a string of textual propositions. Furthermore, authors and their accounts were only loosely connected. So, when one student evaluated the excerpt from Howard Fast, he knew something was wrong: "You can't really believe exactly what they're saying. It's going to be, the details are going to be off." But by the time he reached the last document, his reservations about Fast had fallen by the wayside, as elements from this source were clearly present in his understanding. An Americanist, on the other hand, paused when he encountered the claim that the colonists were drawn up in "regular order." Remembering that an earlier document described the battle formation, he flipped back to Howard Fast's excerpt and then burst into laughter: "Oh, that's from Fast! Forget it! I can't hold on to Fast, I can't do that. But it's funny, it stuck in my mind." So here we see the opposite case: a detail is remembered but the historian cannot remember its source. Reunited with its author, the detail is rejected, for this historian knows that there are no free-floating details—only details tied to witnesses.
The last thing I asked my participants to do was rank each of the written documents in terms of its trustworthiness as a historical source. Naturally, historians with diverse specializations showed many differences of opinion. Nonetheless, the average correlation among historians was striking—.65, a high degree of consensus by any standard. On the other hand, students' rankings were all over the map; their average correlation was a scant .14. All eight historians rated a diary excerpt from a British officer who admitted he lost control over his troops (R.H. Dana Jr., "A British officer in Boston in 1775," The Atlantic Monthly 39 : 398) as the most or second most trustworthy document, whereas no students rated it this high. On the other hand, historians rated the textbook excerpt (Samuel Steinberg, The United States: A Story of a Free People, Boston, 1963) dead last—even less trustworthy than Howard Fast's novel. Three students, however, put the textbook at or near the top, even though it contradicted primary accounts from both sides. One student characterized the textbook as "the facts" and "really straightforward." Another called it "straight information," an "objective" account of the events at Lexington. For such students, the textbook, not any of the eyewitness accounts, emerged as the "primary source."
What accounts for the fact that a group of able high school students displayed such a rudimentary sense of how to read a historical text? How could they "know" so much history and still have so little sense of how to read it? These are not simple questions, and their answers lie beyond the scope of this essay. But at the very least, we can point to the types of texts students generally read in their history classes. Textbooks dominate history classrooms and, as Peter Schrag once noted, history textbooks are often written "as if their authors did not exist at all, as if they were simply the instruments of a heavenly intelligence transcribing official truths." Avon Crismore, a specialist in discourse analysis, provided documentation of Schrag's claim in a 1984 article in the Journal of Curriculum Studies. In a comparison of history textbooks to examples of academic and popular historical writing, she found that "metadiscourse," or indications of judgment, emphasis, and uncertainty, were used frequently in historical writing but were rare in conventional textbooks. For example, historical writing is replete with "hedges"—words such as "may," "might," "perhaps," "possibly," and verbs such as "suggest," "appear," and "seem"—all of which convey the sense of historical indeterminacy. But Crismore found that most textbooks abjured hedges, providing no indication that human judgment had anything to do with the words on the page.
Perhaps the portrait I've painted here is little cause for alarm; perhaps students' naive beliefs about history will simply be sloughed off when they get to college. Most of us know better, though. Indeed, James Lorence, in a study of how college freshmen read documents ("The Critical Analysis of Documentary Evidence: Basic Skills in the History Classroom," History Teaching 8 : 77–84), found beliefs similar to those I've described. Students, Lorence wrote, "expect a document to reveal something which they may regard as 'the truth' ... . They persist in seeking a definitive conclusion on the reliability of the sources before them." Likewise, Christina Haas and Linda Flower of Carnegie Mellon University had undergraduates think aloud as they read a series of polemical texts and found that, like the good readers I interviewed, college students could easily decipher the basic meanings of texts and formulate the gist of what they read. However, "these same students often frustrate us, as they paraphrase rather than analyze, summarize rather than criticize texts" (College Composition and Communication, May 1988, p. 32).
Would teaching students more facts help them do better on the task I designed—a task that had no right answer, a task that throws students into a sea of details and asks not that they "solve problems" but that they find them? Would having students memorize E. D. Hirsch's Dictionary of Cultural Literacy guarantee that they would be able to, in the words of a recent AHA task force, "weigh the validity of arguments, assess the soundness of historical judgments, and otherwise practice the art of critical thinking characteristic of discerning minds"? I fear the situation may be more complicated than even Diane Ravitch and Chester Finn imagine. It is not that students don't know enough history; they do not know what history is in the first place.
Are changes likely in the foreseeable future? As long as issues of coverage and sequence dominate discussions of curriculum, I have my doubts. The issue is not whether we should restrict 11th-grade U.S. history to the modern era, as California has chosen, or to follow New York's lead by infusing the curriculum with multicultural perspectives. In order to produce students who, in addition to possessing a store of historical facts, possess ways of making sense of them, we must take up a set of fundamental questions—What counts as historical thinking? How do we assess students' ability to use facts in the service of argument? What should we teach about the relationship of argument to evidence?
In conferring the status of historian on "Mr. Everyman," Carl Becker pointed out that all of us, not just Clio's retinue, are called on to engage in historical thinking; called on to see human motive in the texts we read; called on to mine truth from the quicksand of innuendo, half-truth, and falsehood that seeks to engulf us each day; called on to brave the fact that certainty, at least in understanding the social world, remains elusive and beyond our grasp. If Becker was right, then school history possesses great potential for teaching students to think and reason in sophisticated ways. Whether we exploit this potential, however, is another story.
—Samuel S. Wineburg is an assistant professor in the College of Education at the University of Washington. He is an educational psychologist whose research focuses on the learning and teaching of history.