Despite healthy gains from the ongoing stock market rally, public pension funds are still badly underfunded and the shortfall continues to widen.

To try to close the gap, many states have shifted pension fund assets into stocks and alternative investments like hedge funds. But in doing so, they face a greater risk of being able to meet their long-term promises to pay retiree benefits, according to a new report from the Pew Charitable Trusts.

In the short run, the heavy investment in riskier assets has been paying off. After big losses during the financial collapse in 2008, large funds posted annual returns of more than 12 percent for the fiscal year ended last June, according to the report.

But the gains haven't come close to making up for years of underfunding by states that simply failed to set aside what their pension accountants told them they'd need to keep their retirement promises to state workers. As of 2012, the latest data available, states had set aside only $3 trillion to meet the more than $4 trillion in benefits earned by public workers.

"It is function of bad policies and bad budget practices," said Gregory Mennis, head of the Public Sector Retirement Systems Project at Pew. "There are states where the funding policies are reasonably sound—New Jersey is a perfect example—but the state simple simply chooses not to make the payments that their own policy recommends making."

Read More Report: 85% of pensions could fail in 30 years

In New Jersey, Republican Gov. Chris Christie, a potential 2016 GOP presidential candidate, recently announced a plan to divert $2.4 billion in pension payments to close a $2.7 billion budget gap. In 2012, the state came up with just 39 percent of the annual contribution required to meet its estimated $47 billion pension liability.

New Jersey isn't alone. About half the states kicked in at least 90 percent of their annual required contributions in 2012, the latest data available.

As a result, most states have set aside far less than they'll need to keep their promises to current and future public retirees. Only Wisconsin has fully funded its pension plans.

Some 14 states have saved at least 80 cents for every dollar they'll need to cover their pension liability. Illinois has set aside only 40 percent of what it owes; Kentucky (47 percent), Connecticut (49 percent), Alaska (55 percent), Kansas (56 percent), Louisiana (56 percent), and New Hampshire (56 percent) have the worst-funded public employee pension plans.

That widespread underfunding helps explain why—despite recent pension reforms and benefit cuts in dozens of states—the gap between public funding and long-term liability continues to widen in many states.

"The problems that occur as a result of underfunding don't necessarily show their impact immediately - so result people can kick the can down the road and not feel the pain until the future," said Mennis. "That's what allows for bad budget decisions to be made today."

To help close the funding gap, states have shifted their pension assets away from relatively safe investments like bonds into higher-risk holdings like stocks and hedge funds, according to the Pew reprot. In 1982, nearly 80 percent of public pension assets were held in bonds and cash; by 2012, the share had fallen to 25 percent.

More recently, funds have shifted to riskier, so-called "alternative" funds that include private equity, hedge funds and real estate. Between 2006 and 2012, those assets, as a share of overall investments, more than doubled to 23 percent.

The shift to those higher-risk assets has helped states boost their projections on investment returns—but has also put those funds at greater risk of losses when the markets pull back. Those losses would further strain state budgets—leaving taxpayers on the hook to make up future pension fund investment losses.

The shift to heavier reliance on stocks and other riskier investment has also added a new layer of cost in the form of higher fees paid to hedge funds other investment managers. Those fees jumped by 30 percent between 2006 and 2012, according to the Pew report.

But it's not always clear that taxpayers are getting their money's worth for the added investment cost, said Mennis, because information about pension fund fees and investment performance is incomplete.

"This all points to the need for addition information on a more consistent basis so that stakeholders understand how well their funds are performing after all the expenses are taken in to account," he said.