The circumstances surrounding the death of Muammar Gaddafi must be properly investigated, and the murder of more than 50 people by forces opposed to the former Libyan dictator in a hotel in Sirte could amount to a war crime, senior British defence officials have said.

Serious questions about the conduct of anti-Gaddafi forces were raised at a meeting of the cross-party Commons defence committee about the conflict and Britain's role in it. Asked about the killing of Gaddafi, the defence minister Nick Harvey said the government had made it clear to Libya's National Transitional Council that it had to be "properly investigated".

Responding to suggestions that it amounted to an extrajudicial killing and that the convoy in which Gaddafi was travelling was composed largely of civilians, Lieutenant General Richard Barrons, deputy chief of defence staff in charge of operations, said: "Did we know Mr Gaddafi was on this convoy? No."

It was a matter for Nato commanders to make a judgment as to whether the convoy amounted to a legitimate target and was part of the Gaddafi regime's "command and control apparatus", the general added.

Asked about the 53 deaths at the Mahari hotel in Sirte and whether it amounted to a war crime, Harvey said "potentially" there was a "prime facie case". However, he said, it was "virtually impossible for us [Britain] to investigate".

Some of the dead in Sirte had their hands bound together, and bloodstains and spent rifle cartridges indicated that they had been summarily executed. "The UK government deplores mass killing in whatever circumstance it took place", Harvey told the MPs.

The New York-based Human Rights Watch (HRW) group has also demanded an inquiry, warning of a "trend of killings, looting and other abuses" by those who triumphed over Gaddafi with the help of British and other countries' armed forces.

Peter Bouckaert, a HRW researcher, said the Sirte hotel had come under the control of fighters from Misrata before the killings took place. He said a massacre there seemed "part of a trend of killings, looting, and other abuses committed by armed anti-Gaddafi fighters who consider themselves above the law".

Harvey conceded there had not been a clear exit strategy for British forces when they embarked on the bombing campaign, and that there was a shortage of Nato surveillance, reconnaissance, and intelligence-gathering aircraft.

Sir Mark Stanhope, the head of the navy, told the committee that the aircraft carrier Ark Royal and its jumpjet Harriers would have been used to bomb Libya had they not been axed. However, he said, RAF Tornados and Typhoons would still have been needed because of their longer range and more accurate weapons.

The MoD estimates that the operational cost to Britain of the Libyan conflict is about £160m and the cost of replacing bombs and missiles an additional £140m. But Britain had helped to avert a humanitarian disaster and "saved countless lives", said Harvey.

Britain's intervention in Libya's revolution will not set a new precedent for military action in foreign countries, General Sir Nicholas Houghton, vice-chief of the defence staff, said at Buckingham Palace after being knighted by the Prince of Wales.

"The skill is to make sure we spend the money we've got on the right things," he said. "There's no doubt about it that some of our historic inventories were built around previous threat scenarios."