Banks usually let borrowers pay extra mortgage principal whenever they want, allowing budget-conscious homeowners to throw in additional money now and then to take years off of a loan's repayment period.

For instance, making 13 payments a year instead of 12 on a $250,000 30-year-fixed mortgage carrying today's average 3.41% rate will cost you an extra $1,110 annually -- but knock 44 months off of your loan's term. You'll also save $20,280 in interest by paying the mortgage off in roughly 26 years instead of 30.

McBride says that in today's low-interest-rate environment, though, retiring a mortgage early has more drawbacks than advantages.

"Most people have much higher financial priorities than accelerating repayment of a low-cost, tax-deductible mortgage," he says.

The expert believes you shouldn't even consider making extra mortgage payments until you first:

• pay off all high-interest credit-card balances;

• build up emergency savings to cover six months of living expenses if you lose your job or suffer some other serious setback;

• make sufficient annual contributions to 529 plans or other college-savings vehicles to cover your or any dependents' future educational expenses;

• make the maximum allowable contribution each year to your and your spouse's 401(k) and individual retirement accounts. For most married couples, that means putting $17,500 into each spouse's 401(k) and another $5,000 into each person's IRA (the maximum the Internal Revenue Service will allow as of 2013). People age 50 or older can also add another $1,000 "catch-up" IRA contribution, and sometimes put an extra $5,500 into 401(k)s.

"Needless to say, very few people can tick off every single one of those items," McBride says.

But even if you can cover all of the above, McBride still thinks paying off homes early has downsides.

For instance, you can't easily take out the extra equity you've put in if your finances suddenly go south.

"If you lose your job, then what?" McBride says. "All of that money that you've poured into your house isn't something you can get to when you need it."

The expert adds that even if you have lots of spare cash, setting aside retirement money over and above 401(k) and IRA limits will give you more chance to enjoy compound investment returns.

"Time is what harnesses the strength of compounding -- the longer you save for retirement, the better off you'll be," McBride says. "You're better off savings for retirement today and paying off the mortgage later -- not the other way around."