Mining the Treasures of the Valley of the Shadow

Frances M. Clarke | May 1, 2002

For almost a decade, the Valley of the Shadow project has been hailed as an exemplary online archive. Created at the University of Virginia's Center for Digital History, this web site ( provides access to masses of Civil War-era materials from two communities on opposite sides of the Mason-Dixon line. Visitors to the site have often pressed the project's founder, Edward Ayers, to analyze and draw conclusions from the vast body of data he and his team have amassed. Yet Ayers had declined, insisting that the project was intended mainly to provide access to the raw material of history and to allow readers to formulate their own questions and answers.But as if to say that the rich archival treasures of the electronic El Dorado he had helped to create were too alluring to resist, Ayers has undertaken—together with William G. Thomas, the director of the Center for Digital History—an innovative and sophisticated analysis of the data using new computing technologies. At an NEH-sponsored e-Humanities lecture held in Washington, D.C., on February 27, 2002, Ayers and Thomas gave a sneak preview of their multilayered digital article that will shortly be published online and, if accepted, in (modified) print form in the American Historical Review. In a playful, dialogue-style presentation, in which both authors took turns speaking, Ayers said that they undertook the article to promote a discussion of the analytical possibilities allowed by new computing technologies and also to suggest the potential forms that digital scholarship might take. Hoping that their article would foster a discussion about the best ways to create and evaluate online publications, the authors noted that such discussions were made timely by a rapid expansion in the number of online publishing initiatives in history. And they were made critical by the need to demonstrate the potential of online scholarship to skeptical academics—especially those in charge of hiring and tenure decisions. Speaking to a standing-room-only crowd, Ayers argued that what we need now are new ways to showcase the possibilities and assess the merits of digital scholarship in order to secure its place in the academy.

The digital article that Ayers and Thomas gave a preview of is a far cry from the average print or online scholarship that's currently available. The article draws from several hundred thousand documents, maps, and images that comprise the Valley of the Shadow project in order to compare social, economic, and political conditions in Augusta, Virginia, and Franklin, Pennsylvania. Ultimately, the authors aim to discover why citizens from these relatively similar communities ended up on different sides of the war, and to place their discoveries in historiographical context. Rather than offering a single, linear narrative, however, the authors have developed a multisequential analysis that also allows readers to construct their own multiple versions of the article, according to their interests. The article is constructed around different topics or "modules." Each module contains a statement explaining the authors' main findings, their supporting evidence, and a discussion of related historiography. A reader interested, for example, in slavery and its causal role in the war would enter "slavery" in the article's search engine. Those parts of the authors' analysis relating to this topic—including maps detailing the geographic distribution of slave residences and places of work in Augusta, a discussion of relevant historiography, and a bibliography—are then drawn together into a coherent article. Similarly, readers curious about, say, comparative economic development or political life in the two communities could construct their own versions of the article. And these are merely a few examples of the way readers can manipulate the data that the authors have assembled.

To allow such interactive reconstruction of their article, Ayers and Thomas employ the latest web-enabled technology to create a complex piece of scholarship. First among the authors' technological innovations was to use XML ("extensible markup language") instead of HTML for encoding their text for electronic publication. Most web documents are created using HTML, a simpler computer language that uses tags to mark up documents so that a web browser knows how to display them. But the downside of HTML, Ayers explained to his audience, is that it limits the connections that can be made between different parts of a document, only allowing readers to call up one web page after another in sequence. In contrast, Ayers noted that XML enabled far more sophisticated links among the various pieces of data in their article.

The advantage of digital scholarship for Ayers and Thomas is not simply that it permits innovative presentations of text and narrative. Instead, they want to prove that new computer technology can facilitate "historical arguments of enhanced intricacy, depth, and connection" by enabling historians to analyze larger quantities of data in more sophisticated ways. The traditional historian toiling away in the archives is simply incapable of processing data to discern patterns and make comparisons in the way that a computer can, Ayers asserted.

Although the audience appeared to accept this argument, a few remained skeptical that individual historians would be able to produce similar scholarship. As one questioner pointed out, Ayers and Thomas are backed by substantial technical, academic, and financial support. Indeed, both the Valley of the Shadow project and the digital article they are creating received significant funding from the NEH and other foundations, while dozens of technical support staff and research assistants working with the Center for Digital History have contributed to the success of both projects. Ayers admitted that these digital ventures were "deeply collaborative," and required a good deal of time, money, and technological savvy. But as Thomas and Ayers were at pains to point out, theirs is simply one example of how to use their massive and yet malleable data set. Their hope is that their digital article will serve not so much as a model but as an inspiration, encouraging other scholars to use their data to make their own forays into the world of online publishing.

Frances Clarke is a research associate for the AHA's Research Division.

Tags: Archives


Please read our commenting and letters policy before submitting.