According to an analysis from Bloomberg Markets, loans are clustering around the highest rated buyers, with 51% of mortgages originating in the first quarter of this year going to people an with Equifax Risk Score of 760 or higher (the scale goes from 280 to 850). That’s up from 23% of mortgages going to people with credit scores of 760 or higher in 2004. Just 4.6% of mortgage loans in Q1 went to people with credit scores between 620 and 659—fewer than half the 9.7% given to the same group in the first quarter of 2004.

Observers worry that making home loans unavailable to people with less-than-pristine credit will make home ownership—traditionally the primary vehicle through which middle-income families are able to build wealth—unavailable to middle- and lower-income Americans. As it is, banks are approving fewer loans to blacks and Hispanics, who tend to be less affluent.

Part of the cause of fewer mortgages going to people with bad credit is that banks are simply more skittish about giving out loans than they were in the mid-2000s, when banks handed out loans too freely to people likely to default on them, setting the stage for the global financial crisis of 2008.

Bloomberg notes that rising housing prices may simply be putting homes out of reach for low- and middle-income buyers. The lingering effects of the financial crisis are also likely making it harder for prospective home buyers to enter the real estate market—according to the Urban Institute, 69% of the 7 million Americans who went through a foreclosure from 2004 to 2015 still had a foreclosure on their credit reports as of last month.

With mortgage markets tightening, if you have a credit score lower than 700 you’re probably better off focusing on improving your credit rather than seeking a loan, as anything on offer for people with lower scores will have extremely high fees and rates.