While most of the U.S. housing market sank deeper into depression, 2010 was a banner year for the Boston luxury condo market. Brokers expect 2011 to be even stronger, with the biggest challenge the dwindling supply of new units.

In the fourth quarter of 2010, there were 174 sales of luxury units in Boston, which was up 38 percent from the fourth quarter of 2009, according to Listing Information Network, a Boston firm that tracks the city’s real estate market. Overall, annual sales numbers were up 22 percent over the previous year and the average selling price of $1.03 million was a 22 percent jump from 2009. In the past decade, only 2008 had a higher average selling price for luxury units, according to LINK.

“If you look at the numbers, there looks like there was no interruption in values [in the luxury market] during the market drop,” says Kevin Ahearn, president of Otis & Ahearn, one of the largest Boston real estate firms. “It did better than pretty well — 2010 was the second best in data in the past 10 years.”

According to Ahearn and Debra Taylor Blair, president of LINK, inventory is the biggest hurdle.

“Our development cycle in Boston is quite lengthy and condo developers aren’t able to secure financing,” says Taylor Blair, adding that there have been no new developments in Boston in three years.

The dwindling supply will continue to drive prices up and create a competitive housing market downtown.

“Inventory is dropping like a brick,” says Ahearn. “It is a tight market.”