www.elsblog.org - Bringing Data and Methods to Our Legal Madness

10 May 2013

Empirical Legal Scholars and "Special" Responsibilities

A recent post by David Schwartz (Chicago-Kent)--wondering whether empirical legal scholars should shoulder "special ethical responsibilities"--ignited a fascinating (and timely) discussion over at Concurring Opinions. Two reasons prompt Schwartz's concerns. "First, nearly all law
reviews lack formal peer review. The lack of peer review potentially
permits dubious data to be reported without differentiation alongside
quality data. Second, empirical legal scholarship has the potential to
be extremely influential on policy debates because it provides 'data' to
substantiate or refute claims. Unfortunately, many
consumers of empirical legal scholarship — including other legal
scholars, practitioners, judges, the media, and policy makers — are not
sophisticated in empirical methods."

Schwartz's concern focuses on what he calls "weak data." By that he means "reporting [results from] data that encourages weak or flawed inferences, that is not
statistically significant, or that is of extremely limited value and
thus may be misused." Specifically, "[t]he precise question I
have been considering is under what circumstances one should report weak
data, even with an appropriate explanation of the methodology used and
its potential limitations."

Whether you agree with Schwartz or not, he raises an important question that warrants attention.