Assessment has become a hot topic in higher education, with even the White House ready to create a ranking of top universities. We are not alone in the rush to evaluate. For the past few years, our colleagues in Britain have been absorbed in a national assessment exercise called the Research Excellence Framework, 2014. The REF, as it is popularly known, is a government-mandated assessment of every higher education department in England, Scotland, Wales, and Northern Ireland. Required every six years, it is managed by the Higher Education Funding Council for England (HEFCE), a government department. The rewards for successful universities (chosen from 154 participants) include a share in $3 billion of research funding for each of the next six years, as well as the prestige that comes to highly rated departments.

To conduct the assessment, HEFCE created four main panels: Life Sciences, Physical Sciences, Social Sciences, and the Humanities and Arts. The main panels then created disciplinary subpanels, whose chairs also serve on a main panel. The Humanities and Arts main panel, for instance, produced 10 subpanels. Observers representing various constituencies, including international members, made up the remainder of the main panels.

The first major task for the main panels was a complicated one: creating rules for the assessment. For instance, every scholar included in the review had to submit four items published (not just submitted) by November 30, 2013. How that rule might apply to pregnant scholars, newly appointed professors, or those suffering from a major illness or disability remained unclear; these and other varied circumstances were left to individual universities to mediate. Another issue facing the Humanities was the method for calculating published items. A book counted as only two of the required four submissions, no matter how long or complicated.

History was included among the Humanities, and 83 departments participated. Three factors determined a department’s overall score: the quality and importance of the department’s academic Output (or published research) made up 65 percent; the quality of its research Environment (research funding, support of graduate students, conferences, centers of activity, etc.), 15 percent; and the reach and significance of its research (or Impact) beyond academia, 20 percent.

The final area, Impact, had not been included in the 2008 review and generated the most controversy. Depending on their size, departments had to submit two or more examples of research that carried into the public domain—civil society, the economy, culture, public policy, public discourse, or innovation. These broad categories created confusion, as did the question of how to judge the worth of a public program. Eventually, criteria were agreed to—how many visitors viewed an exhibition, attended lectures, heard a radio program, or watched a television show—requiring each submission to include an account of its original research, the program that resulted, and an assessment of the audience. History departments submitted a wide variety of projects, exhibitions, public programs, and websites. Large departments fared best.

Once departments had compiled records of faculty research (Outputs), reported on details of life in the department (Environment), and set out case studies of Impact, university administrators then forwarded the results to HEFCE for distribution. In all, 1,785 historians submitted approximately 7,000 publications. The history subpanel had 24 members and called in outside assessors when needed. At least one member of the panel (often two) reviewed every Output. The panels also graded the Environment and assessed the case studies for Impact. HEFCE mandates four scores: 4*, world leading for originality, significance, and rigor; 3*, internationally significant, but short of the highest standards of excellence; 2*, a useful contribution of some influence; and 1*, nationally important, but of minor significance. Achieving coherence over so large an enterprise required considerable calibration in the subpanels, amounting to an immense exercise in peer review.

A score for each department’s whole effort (its combined grades for Output, Environment, and Impact), added to every other department’s, determined a university’s total score. The way data was compiled has since allowed 38 institutions to claim a place in the “top ten.” A number of top-ten lists have surfaced for history departments, all counting (or manipulating) different factors. One list of universities has this top ten: Cambridge, Durham, London School of Economics, Oxford, Saint Andrews, Warwick, University College London, Exeter, Bristol, and Leeds. Aggrieved departments that did not make this list can cite others on which they did appear. University publicity departments have creatively selected which lists to broadcast and which statistics to manipulate.

An enterprise this large—with prestige and funds riding on the results—inevitably produces unintended consequences. Administrators attuned to the rewards of success have in some cases signed famous scholars to short-term contracts in order to use their bibliographies during review. At the very least, they have rallied their campus to ensure a good outcome, appointing officers to manage the campus response, interpret complicated rules, and advise faculties on their roles and reports. An enormous amount of faculty time was spent judging Outputs, sustaining the department’s Environment, and figuring out programs that would have Impact. One campus reported that between 50 and 75 faculty and staff work years were required to prepare the REF report.

Having survived the experience, faculties are already being alerted to the demand for four new published works by November 2019, which will require difficult choices. Should scholars aim for four articles, or publish two articles and one book? Can a book be in print by the deadline (no small matter in history, where monographs abound)? Should one tackle a difficult research theme or take an easier path with assured results? Faculties will be tasked with assessing the work of every department member and deciding whose will be submitted, while some administrations will hire outside assessors with experience in prior disciplinary subpanels. Because HEFCE announced that it would not fund any work graded below a 3*, some administrators insisted that only work scoring 3.5 or better be submitted. An outside assessor giving work only a 3.3 could thereby have a profound effect on a scholar’s career prospects.

Furthermore, even departments with good results may reap few benefits. Research funds earned by one department could be funneled to others in order to raise a university’s overall profile in 2020. Mr. Chips may be long gone, but a world dominated by metrics and assessment makes one nostalgic. Criticism mounting since the completion of REF 2014 is unlikely to stall the movement toward 2020, but it should perhaps give pause to Americans eager to emulate our colleagues overseas.

Robert C. Ritchie is a senior research associate at the Huntington Library. He was an international observer on the Higher Education Funding Council for England’s Humanities and Arts Panel.