Legal Analytics: A New Moneyball for Lawyers?

February 13, 2018

Have you heard of CaseCrunch, the litigation predictor program that beat human lawyers with a margin of almost 25%?

We don’t believe computers will soon, or ever should, replace lawyers entirely. But it does seem the legal industry is pondering utilization of artificial intelligence where it can bring access to justice for more clients and streamline certain phases of the litigation process. Where computer programs can provide previously-unascertainable outcomes with acceptable (even super-human) accuracy, do you see areas of the law where an AI approach to litigation prediction can prove beneficial to human lawyers and clients alike?

Insights and Wisdom
“Any other team wins the World Series, good for them. They're drinking champagne, they'll get a ring. But if we win, on our budget with this team, we'll change the game. And that's what I want, I want it to mean something.”

That was Billie Beane in the movie Moneyball, where data-driven insights were vetted up against the hard-earned wisdom of scouts, managers, and team coaches. Those insights proved capable of providing a margin of victory. If legal analytics programs such as CaseCrunch can prove similarly-capable of providing an edge over the hard-earned wisdom of human lawyers, it will definitely mean something. While the CaseCrunch competition was structured under very strict confines among UK lawyers in the fall of 2017, the results were almost alarmingly surprising.

The CaseCrunch Program

With more than 100 London commercial lawyers competing, making over 750 predictions over the course of a week based on very specifically-defined questions—i.e., Was this complaint about Payment Protection Insurance misselling upheld or rejected by the Financial Ombudsman Service?—the CaseCrunch program scored an accuracy rating of 86.6% where the human lawyers fell short with a rating of only 62.3%. The CaseCrunch developers were, themselves, very forward in reiterating the experiment was not designed to show computers are more accurate than lawyers at predicting outcomes. Rather, the intent was to show when very specific questions are asked, litigation predictors like CaseCrunch, with proven accuracy, can be used to supplement or streamline certain elements of the legal process such as client intake and education.

CaseCrunch program scored an accuracy rating of 86.6% where the human lawyers fell short with a rating of only 62.3%

The CaseCrunch experiment was not designed to show computers are more accurate than lawyers at predicting outcomes.

Imagine if a potential client could fill in a form on a law firm’s website, providing facts and details about their case, and a program like CaseCrunch could be used to advise the client of their likely outcome (with an obvious disclaimer that the program is not 100% accurate). The program could then ask the client if he or she wants to proceed and, if so, then sets up a meeting with an attorney. This would also assist the lawyer in more quickly honing the relevant facts and potential problems with a new client’s case in advance of their meeting. This would mean use of the program, not as a lawyer-replacement, but an improved workflow and developmental tool in the legal realm. It does get one thinking outside the box.

Conclusion
The CaseCrunch scenario reminds us very much of LegalZoom and the cookie-cutter, fill-in-the-blank forms many citizens have found very useful and affordable for addressing basic legal needs. Legal analytics seems to take this to the next step by mining data in case dockets and filings, and then aggregating the data to reveal trends and patterns in past litigation that can be used to inform legal strategy. And we do say only “inform.” At IMS ExpertServices, we know and rely upon the sharp minds of both lawyers and experts every day that can only be honed by years of experience litigating, consulting, and testifying. But we do see a future for legal analytic programs like CaseCrunch to assist and help supplement certain elements of the litigation process and law firm logistic challenges. What do you think? Are AI litigation predictors the new Moneyball for lawyers, or a slippery-slope to excessive artificial interference?