Recently, cable news personality Sean Hannity commented that the Ninth Circuit is the "most overturned court in the country." Politifact rated that claim as "false." But Politifact's analysis is seriously flawed and suffers from selective analysis of the evidence, and misrepresentation of the evidence in other respects.

I recently had the opportunity to appear on NPR's AirTalk to support a proposal from Arizona Senator Jeff Flake's office to split up the United States court of Appeals for the Ninth Circuit. The "Judicial Administration and Improvement Act of 2017" would keep California, Oregon, Hawaii, and some U.S. territories in the Ninth Circuit. It would create a new Twelfth Circuit out of Arizona, Nevada, Montana, Idaho, Washington, and Alaska.

One reason for splitting the circuit is systemic dysfunction in the Ninth Circuit. It has 29 active judges, (but four vacancies at the moment) nearly the size of the 30-member Arizona Senate (to speak on terms for those from the Grand Canyon State). That's nearly twice as large as the 17-member Fifth Circuit. It's little wonder that decisions take longer (often much longer) to issue from the Ninth than anywhere else. Splitting the circuit would help create a new "Mountain Circuit" that would function fairly effectively, and the new Ninth Circuit would remain the largest circuit in the country.

More judges might help the Ninth speed along cases, but it would not help its high reversal rate. That's because the court lacks the ability to self-correct with true en banc procedures; the entire 29-member court can't really assemble, and it's left instead to a lottery of some subset of these judges to correct errors from three-judge panels. And in a lottery of three judges among 29, some combinations are sure to be greater outliers than others. Splitting the circuit would allow it to have true en banc procedures and minimize reversal rates.

So, to Mr. Hannity's claim that the Ninth Circuit is the "most reversed." While there are arguments raised in the Politifact piece that "most reversed" may have limited normative significance, that's a separate argument. (But, Politifact can't help itself to weigh in on the normative claim, concluding, "More broadly, experts say this statistic is a poor way of comparing courts.") Instead, as a matter of pure math, is the Ninth Circuit the "most reversed"?

Politifact says no. The answer is resoundingly yes.

Politifact chooses a cohort of data from 2010 to 2015 to conclude that the Ninth Circuit is the third-most reversed, behind the Sixth and Eleventh Circuits. But why pick this window of time?

I used the same analysis with slightly different data--Politifact uses the SCOTUSBlog Stat Pack, which is slightly simpler and less comprehensive in time than the Harvard Law Review statistics, which I opted to use (and may result in some slight variations of the numbers). Because of changes to the Harvard Law Review statistics system, I also didn't have ready access to Eleventh Circuit data between OT1994 and OT1996.

The chart below displays the cumulative reversal rate, which includes opinions from the Supreme Court that reverse a court, vacate an opinion from a court, or reverse in part or vacate in part. It is the cumulative reversal rate based on the term listed: so, for instance, OT2014 data is the cumulative reversal rate for OT2014 & OT2015; OT2013 is the cumulative reversal rate for OT2013, OT2014, and OT2015. I ran the figures for the Sixth, Ninth, and Eleventh Circuits.

It would be hard for Politifact to manufacture a window that more perfectly enables it to refuse Mr. Hannity's claim. If you begin to include any earlier terms, the Eleventh Circuit quickly falls behind the Ninth in reversal rates; by including a 12-year window back to OT2004, the Ninth Circuit is the most reversed, and holds that trend back through cumulative data to at least OT1994.

Now, there are, of course, many ways to slice such data. Is the Ninth Circuit relatively better than it has been? (Answer: yes, but not by much--its cumulative reversal rate since OT2012 was around 80%, not much better than the rate of around 82% since OT1994). Is the Ninth Circuit not as bad as other circuits in recent years? (Answer: yes, but only with a fairly limited window of time.)

But for Politifact to so brazenly rate Mr. Hannity's claim as "false" displays its choice to evaluate his claim through a single and precise window--perhaps to achieve a result that the fact-checker desired to reach.

Below are charts for the 9th Circuit and 6th Circuit reversal rates dating back to OT1994.

9th
Circuit

rev/vacate

total

rev rate

OT1994

14

17

82%

OT1995

11

13

85%

OT1996

27

28

96%

OT1997

14

17

82%

OT1998

14

18

78%

OT1999

9

10

90%

OT2000

13

17

76%

OT2001

14

18

78%

OT2002

18

23

78%

OT2003

19

25

76%

OT2004

16

19

84%

OT2005

15

18

83%

OT2006

19

21

90%

OT2007

8

10

80%

OT2008

15

16

94%

OT2009

11

15

73%

OT2010

19

24

79%

OT2011

18

24

75%

OT2012

12

14

86%

OT2013

10

11

91%

OT2014

10

15

67%

OT2015

8

10

80%

Total

314

383

82%

6th
Circuit

rev/vacate

total

rev rate

OT1994

4

7

57%

OT1995

2

4

50%

OT1996

2

3

67%

OT1997

3

3

100%

OT1998

2

4

50%

OT1999

3

4

75%

OT2000

5

7

71%

OT2001

9

10

90%

OT2002

5

7

71%

OT2003

6

8

75%

OT2004

7

11

64%

OT2005

6

8

75%

OT2006

4

7

57%

OT2007

2

3

67%

OT2008

5

5

100%

OT2009

7

7

100%

OT2010

5

6

83%

OT2011

5

5

100%

OT2012

2

2

100%

OT2013

9

11

82%

OT2014

4

5

80%

OT2015

3

4

75%

Total

100

131

76%

Additionally, and inexplicably, the Politifact analysis includes this absurd claim:

We also found that the 9th Circuit never had the highest reversal rate in any individual term between 2004-15. (That’s the farthest back we could go.)

Just as it would be wrong to look at the total number of reversals--the Ninth Circuit is the largest circuit, and we would expect it to have the most raw reversals (and the most raw affirmed opinions) over a period of years, such as 314 reversals in a little over 20 years, which dwarfs all others--it would be just foolish to look at a single year's data for which circuit had the highest reversal rate. While it sounds impressive that the Ninth Circuit was "never" the "highest" in a single year for 12 years, Politifact's own reporting in this same piece explains why a single year's data is a silly metric:

In 2014, for instance, the 2nd Circuit had a reversal rate of 100 percent, which sounds pretty bad until you find out that the Supreme Court only heard one case from the 2nd Circuit that entire season.

There are plenty of years where a single circuit's record is 0-1, or 0-2, giving it a 100% reversal rate; meanwhile, the Ninth Circuit, with an appeals load of at least 10, and often more than 15 cases, it almost assuredly guaranteed at least one decision affirming what the court did.

Politifact's fact-check, then, is false.

It's worth emphasizing that what bearing this particular claim has on the merits of a decision to split the Ninth Circuit into smaller courts is a different matter. There are many good reasons for dividing the Ninth Circuit up, which I discussed in my AirTalk interview, and which Mr. Flake's offices will surely raise to his constituents.

On the precise point raised by Mr. Hannity, however, he is, at least in some measure, quite right--the Ninth Circuit is, in recent history, the most reversed federal court of appeals.