Black females are 17 percentage points more likely to attend college than black males, making the gender gap among black youth larger than the black-white racial gap in college enrollment (14.7 pp). We estimate a sequential model of schooling and arrests to assess the major contributing factors to the gender imbalance in educational attainment within racial groups. First, we find that differences between males and females in measures of early behavior account for the majority of the gender gap for each racial group. Second, despite the fact that 50% of black males were arrested at least once before age 25, we find little evidence that arrest outcomes influence educational attainment, and that the negative correlation of educational attainment and arrests is entirely attributable to the same behavioral factors that explain the gender gap in education. Finally, we find that black males have the largest response to improvements in family background characteristics, such that equalizing the distribution of family background characteristics for black and white youths reduces the gender gap in college enrollment among black youth by 50% and completely eliminates the black-white racial gap in college enrollment.