A handful of large mutual fund firms were hit hard when selling swept the US junk bond market last month, underscoring the challenges facing the asset management industry in 2016.

iStock

Among the firms suffering large investor withdrawals and declines in their net asset values: Third Avenue Management, whose decision to bar redemptions at a troubled credit fund marked the acute phase of the December panic; Loomis Sayles, which had more than $1 billion of investor withdrawals from its largest bond fund; and Ivy Investment Management, the Waddell & Reed Financial unit that runs one of the high yield bond funds that in 2015 had the industry’s largest outflows.

Many investors expect flat returns this year for many asset classes, following sharp increases in stock and bond prices after the financial crisis. Also likely on tap: rising volatility as policy makers in the US, China and elsewhere pare back the stimulus widely credited with fuelling the post-crisis rise in asset prices.

Some investors have warned that soft economic growth, volatile markets and fund holder anxiety could provide a combustible mix in 2016, potentially fuelling large outflows at some asset management firms whose funds have performed poorly.

“We had a lousy year,” said Dan Fuss, manager of the Loomis Sayles Bond Fund, which lost about 7% during 2015, hit by bad currency and interest rate bets as well as widening investment grade and high yield credit spreads.

“People ask if it’s like 2008, and it’s not. That was a bang, the bids went out of the market and the prices were marked down. This one erodes; it just sort of eats in,” he said of bond market weakness.

Related

Investors pulled $9.4 billion out of mutual funds that specialise in junk bonds last month, according to Morningstar. The redemptions marked the second largest monthly outflow from high yield mutual funds, after a $12.1 billion outflow in June 2013. Total assets managed by junk bond funds tracked by Morningstar declined 13% last year to $237 billion in December.

Third Avenue Management said it managed $6.3 billion in assets at December 31, down 21% from the $8 billion at the end of last September. A spokesman for Third Avenue declined to comment.

Investors pulled $624 million out of the Ivy High Income Fund in December, equal to some 10% of the junk bond fund’s assets at the start of the period, according to Morningstar data. The exodus reflects mounting fears among individual investors of the risk of sharp losses in mutual funds specialising in hard-to-trade securities including junk bonds.

Assets in the Ivy fund have halved since August 2014. Assets were about $5.3 billion at the end of December. Investors began redeeming shares in large quantities in 2014, when two portfolio managers left and falling oil prices triggered losses in the junk bond market.

“In December, every fund of significant size in the category had outflows,” a Waddell & Reed spokesman said. “Singling out the Ivy High Income Fund misses that broader point.”

Redemptions in December more than doubled from $290 million in November after the junk bond fund operated by Third Avenue Management began liquidating and halted investor redemptions. The move sparked concerns about the health of other high yield bond funds with large concentrations in especially risky securities with credit ratings of triple-C or lower. About half of the Ivy fund’s assets fell into that riskier bucket as of September 30, according to Morningstar.

Assets under management at the Loomis Sayles Bond Fund declined about 30% during 2015 to $16.9 billion at the end of December, according to Morningstar. It said investors pulled $1.5 billion from the fund in December, bringing net withdrawals for the year to $5.8 billion as performance suffered.

Fuss said he had been “dead wrong” in forecasting interest rate rises and was stung by currency bets related to the Norwegian krone and New Zealand dollar. He added that outflows at the end of the year were accentuated by year-end tax and capital gains-related sales.

Franklin Resources, another large bond fund manager, said Monday its total fixed income assets under management fell to $309.2 billion at the end of December, from $319.2 billion at the end of November and $354.5 billion at the end of 2014. A spokesman for the firm said Franklin reports flows on a quarterly basis during earnings releases.

Write to Sarah Krouse at sarah.krouse@wsj.com, Matt Wirz at matthieu.wirz@wsj.com and Gregory Zuckerman at gregory.zuckerman@wsj.com