"Methodological Advances and Empirical Legal Scholarship: A Note on Cox and Miles's Voting Rights Act Study"

Nancy C. Staudt and Tyler J. VanderWeele have posted this reponse to the Cox and Miles study at the Columbia Law Review Sidebar. Here is the synopsis:

One notable difference between early empirical legal scholarship and the more recent sophisticated contributions to the literature is scholars' goal of identifying cause and effect relationships. Professors Cox and Miles's recent study of judicial decisionmaking provides a terrific example of this new-generation work.[1] The authors investigate whether personal attributes such as ideology, race, or gender cause judges to favor (or disfavor) plaintiffs' claims under section 2 of the Voting Rights Act. The study is a valuable contribution to the emerging body of empirical scholarship exploring causal relationships, and to the work on judicial decisionmaking and voting rights litigation in particular.[2]

Causal inference, as opposed to making claims about mere correlations, is, of course, an ambitious undertaking. Investigators must spend time and energy exploring the underlying relationship between and among the variables of interest in order to identify possible bias and confounding in their data and, importantly, to address these perceived problems with appropriate conceptual and statistical methods.[3] If bias and confounding exist but are not--or cannot be--remedied, scholars must exercise humility in reporting empirical results: They may point to interesting correlations in the data, but causal claims would be completely unjustified.

In this Response, we use Professors Cox and Miles's study of judicial decisionmaking to explore what is at stake when legal scholars present empirical findings without fully investigating the structural relationships of their data, or without explicitly stating the assumptions they make in order to draw causal inferences. We do not intend merely to identify the limitations of Cox and Miles's work (and by implication, those of many other empirical studies published in the extant legal literature); rather, we plan to introduce a new methodology that is intuitive, easy to use, and, most importantly, allows scholars to systematically assess problems of bias and confounding. This methodology--known as causal directed acyclic graphs--will help empirical researchers identify true cause and effect relationships when they exist, and at the same time posit statistical models with appropriate controls, in order to better justify causal claims. While this methodology has become popular in a number of disciplines--including statistics, biostatistics, epidemiology, and computer science--and is widely believed to be a valuable tool for empirical research, it has yet to appear in the empirical law literature. Accordingly, our goal is to offer a brief introduction of the method and to initiate discussion as to its worth in empirical legal studies.

Posted by Rick Hasen at May 31, 2009 09:01 PM