Sanaa, Yemen (CNN) -- Dozens of civilians may have been killed by Yemeni forces since May while fighting an armed Islamist group in Abyan province, the activist group Human Rights Watch said in a report released Saturday. But a spokesman for Yemen's ruling party disputed the findings.

"As Yemeni government forces battle armed militants in Abyan, civilians are paying the price," Joe Stork, deputy Middle East director at Human Rights Watch, said in the report. "Both sides need to be doing much more to protect civilians from harm, and the government should be investigating possible laws-of-war violations by its forces in Abyan."

Human Rights Watch said it did not visit Abyan province because of security concerns but received accounts of attacks from 30 witnesses who fled the clashes. The group cited several reported incidents, including what some described as indiscriminate attacks on civilians with "artillery shrapnel or small arms with no visible fighting or military targets nearby."

But the ruling party in Yemen refuted the report.

"The Yemeni government insists that the Human Rights Watch report is 100(%) inaccurate," said Tareq Shami, spokesman for the ruling General People's Congress party.

"Only al Qaeda targets are killed by the government, and there is no proof of the government being involved in the killing of civilians in Abyan."

He added, "Foreign fighters are also involved killing and attacking civilians in Abyan, not the government. We are defending the people and will continue doing so."

Abdu Ganadi, spokesman for the Yemeni government, said "all wars have casualties."

"Yemen is trying to limit the civilian deaths," Ganadi said. "Our condolence(s) to the families who lost a loved one."

Yemen has been consumed by unrest for months as protesters have demanded an end to the rule of President Ali Abdullah Saleh's rule. Saleh and other senior officials were wounded June 3 in an attack on the mosque at the presidential palace. Saleh is being treated in Saudi Arabia. Officials have said he will return when he has recovered.

In recent weeks, government troops have battled both tribal forces and Islamic militants, including al Qaeda in the Arabian Peninsula.