What’s Wrong with Ranking Legal Scholars?
Mary Dudziak has an important post at Legal History Blog objecting to 
Brian Leiter‘s methodology of ranking legal scholars according to the number of times they are cited in the Westlaw JLR database of legal scholarship This, according to Dudziak, will under-count citations (and therefore under-count the impact) of inter-disciplinary scholars whose work is cited in journals not included in the Westlaw database. Dudziak is surely right to point to out this misleading aspect of Leiter’s rankings. However, Leiter could remedy this problem, if he wished to expend the resources, by having a special ranking for inter-disciplinary legal scholars. Accordingly, the real problems with such rankings lie elsewhere.
Indeed, the culture of ranking suffers from two further defects. First, it fortifies the dogma that empirically established rankings reflect an interesting account of the true impact of legal scholars. But to ratify this methodology one needs an evaluative criterion against which to compare the empirical results. Yet, if we had such an evaluative criterion, there would be little need, if any, for purely empirical evidence in the first place. Moreover, the enduring value of such rankings is highly problematic. Consider an example from another discipline. David Hume, reputed to be one of the most important philosophers, if not the most important, ever to write in English, during his life time was known more for The History of England than for his masterpiece, A Treatise on Human Nature. In fact, Hume himself recognized the Treatise’s failure to have an impact at the time in ruefully observing that the Treatise “fell stillborn from the press.” How would Hume’s philosophical reputation have fared if during his lifetime there existed a philosophical ranking system comparable to Leiter’s?
More important, ranking presents a unique problem for legal scholarship. Most legal scholars, without experience or qualifications in other academic disciplines, accept without question, the so-called “two-tiered” system of law review craftsmanship. Pardon the hyperbole, but there are three central imperatives in legal scholarship, especially for the fledgling scholar: citations, citations, and citations. By contrast in philosophy, citations are generally frowned upon. A philosopher will use citations 
for purposes of attribution and for some brief explanations which would appear out of place in the text. Indeed, “attributions” themselves are also frowned upon in philosophical scholarship. No, not because philosophical culture encourages plagiarism, but rather because a philosophical article is supposed to be original; the author’s article is supposed to stand on its own. In philosophical scholarship, there’s no need to try to impress editors with how many other philosophers you can name or even how much you know about a topic by dropping citations like seed. Your competence–and the worth of your article–is determined by how well you frame the issue, how cogent your criticisms of important alternative approaches are, and your solution to the problem. Citations of tens or hundreds of other philosophers are simply unimaginable.
Back to legal scholarship. Any astute beginner in legal scholarship will, if savvy, look at the scholars on Leiter’s list and make sure to cite at least some of the scholars on the list, in his or her first article (and even in his or her so-called “tenure” piece). I’m not suggesting that these scholars don’t read the articles. But there does seem to be a propensity to skim, shall we say, the relevant passage and then cite it in order to demonstrate to the student editors just how erudite you are. In philosophy, as I recall, citing someone without thoroughly reading the relevant work (and understanding it) was considered, if not duplicitous, surely irresponsible. That’s not, unfortunately, the practice in legal scholarship. Many lawyers turned scholars seem to think that they can master just about any area, no matter how complex and no matter how intertwined its foundations are with other academic disciplines. In-depth analyses of legal issues, even issues in legal theory, with few exceptions, seem to be absent from their vocabularies.
Additionally, rankings such as Leiter’s encourage members of the scholarly community to make prominent scholars more prominent independently of whether there work continues to be first-rate. (Yes, it is true that early success in scholarship does not guarantee continued success. Although, come to think of it, in legal scholarship the question of whether a sensational young phenom lives up to his or her potential is almost never raised.) The result is that less prominent scholars are virtually ignored. (“The rich get richer and so forth.”). It’s bad enough to require an abundance of citations in legal scholarship. But to use citations as the basis of a ranking system of a scholar’s impact or the inherent worth of a scholar’s article compounds the problem. Legal scholars can be (at least hopefully) creative intellectuals whose work should be judged by the content of its character not by the number of its citations.
One final note,
Lieter’s intent might be to simply list those scholars whose work is cited more than others with no further evaluation attached to the ranking. However, such a project is conceptually predicated on a sharp distinction between the descriptive and the evaluative. Even if such a distinction is viable, Leiter’s ranking overlooks the entrenched psychological predilection to automatically see rankings as identical to lasting impact and inherent worth. To perseverate over rankings based on the number of citations is likely to distort the creative enterprise of legal scholarship, encouraging junior scholars to remain fixated on rankings and thereby abdicate the importance of deciding for themselves whose work is worth studying. With all due respect to Leiter, whose scholarly output is formidable, descriptive rankings in any intellectual domain, but especially in law, though intuitively appealing, should be resisted.
Credit for Erasmus Image
Credit for Hume Image
Credit for Langdell Image
No comments yet.