23 September 2006

Grades and the mismatch effect - Sander 6

In my third post, I summarized the “grades” argument: preferences cause black students to have much lower grades; low grades lead ineluctably to a variety of bad outcomes; and grades matter more than school prestige in predicting outcomes. See that post for more detail and the theoretical context of the argument. Here I’d like to give a few empirical examples and discuss the critics.

The two tables below provide a nice intuitive illustration of how low grades produce the mismatch. The first breaks out, for each law school tier, the relationship between law school grades and the odds of graduating and passing the bar on the first attempt. For example, 27% of the students in the bottom 5% of Tier 3 (Duodecile 20) graduated and passed the bar on their first attempt. It’s pretty obvious from this table that the bottom of one’s class is a lousy place to be.

(Note that in both Table 1 an 2, below, the lowest grade demidecile is 20; smaller-numbered demideciles mean higher grades.)

Table 1. Proportion of matriculants, by school eliteness and grade demidecile, who graduate and pass the bar on their first attempt, BPS data

Grades demidecile

Tier (1 = most elite)

6

5

4

3

2

1

20

10%

10%

22%

27%

33%

59%

19

21%

36%

40%

51%

68%

74%

18

19%

42%

56%

67%

71%

81%

17

26%

48%

67%

77%

73%

84%

16

31%

54%

70%

84%

81%

85%

15

35%

58%

82%

83%

82%

90%

14

30%

72%

79%

88%

87%

96%

13

41%

73%

86%

85%

88%

95%

12

45%

88%

84%

91%

89%

96%

11

43%

81%

90%

94%

91%

99%

10

62%

85%

89%

94%

96%

98%

9

67%

90%

93%

95%

93%

98%

8

77%

92%

93%

93%

96%

97%

7

79%

92%

93%

96%

95%

98%

6

77%

87%

95%

94%

96%

100%

5

90%

88%

96%

97%

97%

98%

4

86%

89%

96%

98%

97%

96%

3

86%

94%

97%

97%

97%

98%

2

93%

94%

96%

96%

98%

98%

1

97%

95%

97%

98%

99%

100%

This next table (Table 2) contains the same categories as Table 1, but each cell shows the average admissions index (a weighted average of LSAT and undergraduate GPA) of students with those law school grades at that school. (Both of these tables omit students who graduated but never took a bar.)Obviously, there’s a relationship (though a far from perfect one) in each tier between index and grades.

Table 2. Average index of matriculants, by school eliteness and law school grade demidecile, BPS data

Demidecile

Tier (1 = most elite)

6

5

4

3

2

1

20

503

567

627

628

680

745

19

538

603

650

683

723

793

18

545

630

667

712

749

805

17

535

637

675

730

753

818

16

509

642

687

752

768

853

15

514

647

694

751

766

846

14

586

645

702

762

784

843

13

565

649

707

759

781

853

12

528

647

711

771

783

855

11

563

647

711

769

765

862

10

565

662

716

770

791

857

9

610

669

723

785

803

858

8

557

660

722

781

801

873

7

598

672

722

788

807

876

6

618

683

732

789

817

864

5

599

684

739

797

814

874

4

583

701

737

799

822

876

3

642

692

748

799

823

877

2

660

720

755

812

819

863

1

676

727

777

827

829

854

Here’s the point: compare someone with the same index at differing tiers (in Table 2), and look at the outcomes of those cells in Table 1. For example, the cohort with an average index of 680 in Tier 2 schools has, in the aggregate, a 33% chance of graduating and passing the bar on the first attempt. A similar cohort (683) in Tier 3 has a success rate of 51%. A similar cohort (675) has a success rate of 67% in Tier 4, and the closest cohort in Tier 5 (683) has a success rate of 87%. In each case, lower tiers are associated with better outcomes for what appear to be similar cohorts.

This doesn’t mean, of course, that it’s always better to go to a less elite school – once one’s cohort has an index close to a tier’s mean index, the benefits (in terms of graduation and bar passage) of going to a less elite school are small. But a cohort associated with low grades at a given school will do much better at a less elite school.

There have been two sorts of critique of the “grade” argument. The first was from Dan Ho, who argued here that my regressions comparing the power of law school grades and law school eliteness were invalid, because one’s grades are influenced by the eliteness of one’s school. I think Ho simply didn’t realize that law school grades in the BPS dataset are standardized by school (so my measures of GPA do accurately measure performance relative to one’s peers within each school); if one digests that fact, the argument goes away. Moreover, one can use many different methodologies to demonstrate that grades trump school eliteness in predicting outcomes – Tables 1 and 2 above, for example, as well as analyses here and in "Mismeasuring the Mismatch" (posted here).

The other critique of the grade argument is that credentials are poorly correlated with grades; thus, there’s little if any relationship between the admitted “credentials gap” caused by racial preferences and the “grade gap” between blacks and whites.

This really shouldn’t be a debatable issue anymore. No one has tried to dispute that, for groups (as opposed to individuals) the correlation between credentials and grades is extremely high (.88 and upwards; see Systemic Analysis, Part IV). No has disputed Anthony & Liu’s finding (which can be found here , report TR-00-02) that black underperformance is a very small factor in explaining black grades (their finding implies that about one-tenth of the black-white grade gap might be due to underperformance). Moreover, it appears that when one takes into account differences in college quality, even that small underperformance diminishes or disappears.

Many critics won’t concede this issue because, if they do, it concedes the entire debate on the mismatch effect.