Amid growing talk that China may take steps to stimulate a slowing economy soon, the country's vice finance minister told CNBC that Beijing would be careful not to repeat past mistakes.

Last week, China's Premier Li Keqiang said the government should roll out measures as soon as possible to stabilize growth and boost domestic demand, China's state news agency Xinhua reported.

Vice Finance Minister Zhu Guangyao told CNBC in an exclusive interview on Sunday that authorities would take a cautious approach and was mindful of the negative impact stimulus has had on the economy in the past.

"We don't want to repeat some mistakes before [that] we've made," said Zhu, adding that in the past China had stimulated "too much" and this had hindered sustainable economic growth.

China pumped billions of dollars' worth of stimulus into the economy following the global financial crisis in 2008 in a move that many economists say contributed to problems such as inflation and over capacity.

Zhu said growth had been too high back then and today the focus was more on achieving "quality" growth, an objective Beijing planned to achieve through increased investment in environmental protection and infrastructure building, especially in more remote and poorer areas.

"We think a 7 to 8 percent growth rate is in line with Chinese economic potential and in line with our real capacity," he added.

At the National People's Congress meeting this month Chinese leaders said there was some flexibility around the 7.5 percent gross domestic growth target for 2014.

Asked whether China would accept more defaults in the future, Zhu said such incidences would be determined by the market.

"[It's] not the government [that] decides, the market decides," he said.

"We must balance the moral hazard issue and how to make real reforms, to take away the risk of bubbles bursting and causing some systematic impact. We must rebalance that," he said.

ChinaFotoPress | Getty Images

Zhu added that recent weakness in the yuan was also driven by markets and reflected weakness in Chinese economic data.

Policy makers widened the yuan's trading band to 2 percent from 1 percent earlier this month, paving the way for greater volatility in the Chinese currency.

"That [the move lower] is something related to the market. Because the February [export] figure showed some negative trade and we have a long, long history of a trade surplus with a large amount... so the market reaction is normal," Zhu said.

He added that one of the biggest risks to the global economy was U.S. monetary policy and urged policy coordination between global central banks.

"[The issue is] how to deal with this challenge we never met before [the end of quantitative easing]. That's a test for policymakers not [just] in the Federal Reserve, but all central bank governors should take consideration of this challenge," Zhu said.

The Fed started unwinding its monetary stimulus program at the end of last year, triggering volatility in emerging markets.