Compared to the 1980s, when mortgage rates hovered above 10%, today’s rates remain relatively low. In early May, the 30-year, fixed-rate shot up to 4.46%, before settling back to 4.29% last week, according to Freddie Mac.

However, the recent pace at which they’ve been climbing has many potential homebuyers hesitant to buy a home.

At the end of June, right after rates rose sharply, Trulia($33.12 -0.2%) surveyed more than 2,000 people to see what their biggest worry would be if they were to buy a home this year.

Of all the consumers surveyed, 41% said their top fear is that mortgage rates would rise before they could actually buy a house. Second to rates, 37% of consumers said they were worried prices would rise before they could buy, and 36% said they wouldn’t find a home for sale that they like.

So how high will rates have to get before consumers become too discouraged to buy a home? Among consumers who intend to buy a home someday, 13% said that mortgage rates of 4% were already too high for them to consider buying a home. Rates had already climbed to 4% at the time of the survey.

Another 20% of consumers surveyed said they’d be discouraged from buying a home if rates reach 5%, while another 22% said they’d be discouraged from buying a home if rates reach 6%. Combining these groups, 56% of consumers who plan to buy a home someday would be discouraged from doing so if rates reach 6%.

But are consumers right to worry about the effect of mortgage rates on housing costs? According to Trulia, yes. Higher rates will raise the monthly mortgage payment for a loan.

For example, with rates at 3.35%, the monthly payment on a $200,000, 30-year FRM is $881. However, once rates hit 4.46%, that payment jumps to $1009 — a jump of 14% in the monthly mortgage payment.

“This means a consumer can afford less house for a fixed monthly payment, which – all else equal – should reduce housing demand and home prices in the long term. In the short term, however, if consumers expect rates to rise further, some might rush to buy, which could boost sales and home prices temporarily,” said Jed Kolko, chief economist at Trulia, in a report.

Surprisingly, the recent run-up in rates has not greatly affected prices or home-purchase mortgage applications as of yet. According to the Trulia Price Monitor, asking prices only rose 1.5% month-over-month in June. Additionally, the Mortgage Bankers Association index for home-purchase mortgage applications in June rose 2% month-over-month.

“With price gains still going strong, there are few signs that the rise in rates will derail the housing recovery,” said analysts atCapital Economics.

So why has the effect of rising rates on the housing market been limited thus far? Mortgage rates are rising alongside a strengthening economy, which is subsequently boosting housing demand. And while demand is on the rise, a tight inventory is forcing many would-be buyers to wait to buy.

Additionally, rising rates could lead to expanded mortgage credit, as refinancing demand dries up. Banks might look to expand their home-purchase lending to replace the refinance activity they have lost.

Barry Habib, chief strategist with Residential Finance, said he’s never experienced anything like what is seen here.

“In fact, it’s the largest percentage rise in interest rates and as rapid a period as we’ve seen in 53 years.”

Habib said he’s seen purchase activity drop slowly. When we see a normal lull is in July’s numbers. September’s numbers will be more telling, he said.

But demographics remain strong. The case for buying a home has never been stronger, with rent rising and affordability near its all-time best.

“You can make a strong case that housing should be strong moving forward,” he said. Habib noted that affordability is still 1% below the average for the past 10 years and 2% below what it’s been for 20 years.

Matt Weaver, senior mortgage banker at WCS Lending, told HousingWire that the main effect he’s seeing from the rise in rates is an increased sense of urgency.

“The interest rate increase hasn’t affected any homebuyer that I’m dealing with at this point in time,” said Weaver. “Has it affected their amount of monthly payments? Certainly. But it hasn’t taken them out of the game.”

He added, “Because it happened so fast, it almost didn’t even allow enough time to think about ‘should I pull back and not look for a home?'”

Weaver said May 22 marked the start of the rate volatility. In fact, the mortgage banker said for the first two weeks in June, if he had a client come in to make a mortgage application in the morning, he would have to increase the interest rate on the application by the time the process was finished nearly an hour later.

Luckily, Weaver said he had not seen a transaction as of yet to where an interest rate stood in the way of making the purchase.

“Going forward, from some of the studies that I read, I think that rates are going to take a much more gradual approach as opposed to this erratic behavior we’ve been seeing,” he said.