Think rates have risen? Looking back on 18.63 percent loans

While mortgage rates have been rising the past few months, they are still historically low compared to the trend over the last four decades, Freddie Mac reported.

But rates as low as they were in November 2012 - when the 30-year fixed-rate mortgage reached an all-time low of 3.31 percent - aren't likely to return any time soon, the mortgage giant said.

Still, Freddie assures borrowers that the all-time record high of 18.63 percent reached in October 1981 isn't on the horizon either. (At 18.63 percent, monthly mortgage payments on a $200,000 loan would be $3,117, compared to $992 a month at today's 4.32 percent average.)

With mortgage rates at 4.32 percent, 123 of the 157 metros that Freddie Mac tracks remain affordable to households earning the median income. In order for affordability to be hampered in the majority of markets, interest rates would have to reach 7 percent, according to Freddie Mac.

"Stubbornly high unemployment over the last several years coupled with stagnant income growth exacerbates declining affordability in a rising interest rate environment," according to Freddie's blog post. "More jobs and income growth would help blunt the effects of higher interest rates and make buying a home more accessible. While jobs and income have shown some improvement in recent months, they continue to be challenged."

Here's an overview of mortgage rates in the past four decades, as well as the approximate payment on a $200,000 mortgage and how it changes with the rise and fall of rates, according to Freddie Mac.