More on Ranking Legal Scholars
Brian Tamanaha, at Balkinization, has an important response to Brian Leiter’s insistence that
citations–with the appropriate qualifications, of course–measure the true impact of legal scholars. Some of the key points in Tamanaha’s post, as I understand it, are: (1) It is a non sequitur to insist that citations can be a “proxy” for impact even with elaborate qualifications, (2) The practice of citation, unique to law, requires an inordinate number of citations and “over time, stock or standard citations develop, which are cited again and again, with no guarantee or even indication that the authors cited are read or even if they are read that the cited piece had any significant impact on the field” and (3) An alternative would be to count only citations where the source was actually discussed. Although even here, I would add, discussing an article might not prompt anyone else to notice, or if someone does notice, to cite or extend the discussion. The true value–and therefore, probably (and hopefully) the true impact–of law review articles consists of the importance of the question posed, knowledge of previous answers, comprehensive analytical and critical examination of these answers, a proposed solution, and, of course, writing style. Seeking to ascertain an article’s value cannot possibly be achieved by counting its citations even with qualifications. One can, of course, qualify a thesis to death.
Generally, the inclination to rank involves two elements: a class of items we think should be compared and evaluated and criteria for doing so. This is a perfectly legitimate instinct. When we compare and evaluative two novels, we’re ranking them in a sense. But comparing and evaluating, though typically essentially contested, can be interesting and productive. Such a practice, in principle, can illuminate critical issues and help us discover important questions not obviously raised by our initial discussion of the novels.
The question is whether “ranking” does the trick with novels or with any other complex value-items, including the “impact” of legal scholars.[1] At its heart, ranking seeks an uncontroversial or virtually 
uncontroversial (empirical?) criteria when we don’t agree over the comparisons or evaluations conducted on an intuitive or normative level. We, therefore, seek an independent (“objective?”) standard that will, with appropriate qualifications, take the contention out of intuitive and normative disagreements about which items (in a class) have greater value of some kind or another. But any uncontroversial intersubjective criteria will be unable to explain why what we value has value. And any controversial intersubjective criteria will not achieve consensus and reliability if that is what we seek. Surely, consensus and reliability in ranking the impact of legal scholars are two of the salient elements we design ranking to achieve.
We’re mixing frameworks here. We should have learned as much from G.E. Moore’s “naturalistic fallacy” and David Hume’s dictum concerning the relationship be descriptive and normative (in Hume’s case, prescriptive discourse.[2] I’m endorsing neither Moore’s “naturalistic fallacy” nor Hume’s puzzlement over deriving and “ought” from an “is.” Evaluative judgments–prescriptions, normative claims, and so forth–might be derivable from some types of descriptions, but the character of such a notion of “derivability” will inevitably be complex and tentative. That brings me to Tamanaha’s terrific and pithy conclusion: “Our culture suffers from a ferocious ranking fetish. Leiter’s citation study feeds the beast, when we should instead be starving it.” Bravo! Bravo!
_________________________________
[1] “Impact” or “true impact” is significantly ambiguous. They might refer to the prevalence of a legal scholars works in the literature upon which consensus can be achieved. By contrast, it might refer to the effect on the tenor, character, and direction of the field of scholarship in question. The first sense is generally uninteresting but verifiable. The second sense in fundamentally important but doesn’t stand a chance of achieving verification or consensus.
[2] I am not unaware of objections to and re-interpretations of Hume’s dictum that one can’t derive an ought from and is. But Hume’s succinct statement of the relationship between descriptive and prescriptive reasoning is worth pondering just the same.
In every system of morality, which I have hitherto met with, I have always remark’d, that the author proceeds for some time in the ordinary ways of reasoning, and establishes the being of a God, or makes observations concerning human affairs; when all of a sudden I am surpriz’d to find, that instead of the usual copulations of propositions, is, and is not, I meet with no proposition that is not connected with an ought, or an ought not. This change is imperceptible; but is however, of the last consequence. For as this ought, or ought not, expresses some new relation or affirmation, ’tis necessary that it shou’d be observ’d and explain’d; and at the same time that a reason should be given; for what seems altogether inconceivable, how this new relation can be a deduction from others, which are entirely different from it.
David Hume, A Treatise of Human Nature (1939-40).
No comments yet.