Commons select committees must be clearer about their objectives and how they measure success, according to former cabinet secretary Lord O'Donnell.

In a continuing spat between politicians and the senior civil service over who should hold whom to account for public services, O'Donnell, who was cabinet secretary until December 2011, said that while civil servants were "incredibly accountable", select committees lacked scrutiny of what they were trying to achieve or how they measured success.

Giving evidence to the Lords constitutional committee, which is holding an inquiry on civil service accountability, O'Donnell said that there were "enormous things" that should be done to improve accountability. Many suggestions – all "sensible stuff" – had been made as far back as 1991 and has been reiterated in the recent civil service reform plan, he said.

The most important issue, he added, was making managers accountable within a clear framework of responsibility. "It's defining the clarity of outcomes that's important," said O'Donnell, commenting that select committees themselves should be under more scrutiny to ensure they are clear about their outcomes and how they would measure them.

O'Donnell said that some form of code of practice for select committees "could make a real difference" and in a dig at high-profile committee chairs, suggested that committees' choice of topics for their inquiries lacked a clear focus. "A cynic would say it is to do with the public profile of those on the committee," he said. "Couldn't we get a bit further than that? I've had some fantastic committees where real improvements were made and some where frankly they never got on to the important stuff."

In February 2012, it emerged that before he stepped down, O'Donnell had written to Labour's Margaret Hodge, chair of the Commons public accounts committee, to express dismay at her treatment of officials and her drive to make civil servants accountable directly to MPs, rather than to departmental ministers.

O'Donnell also said that there was an issue about the accountability of arm's length bodies, saying he would welcome a minister setting up a framework for an agency, with objectives, and would expect the permanent secretary of that department to account for it in front of select committees. "That would be great," he said. "All I'm after is clarity."

Despite much progress, too many central government organisations still lacked clarity about their own outcomes, said O'Donnell. "You could write a book about it and call it 50 shades of grey," he said, to laughter from the peers.

O'Donnell was one of four former cabinet secretaries giving evidence to the Lords committee on 11 July. He followed Lord Armstrong, who was cabinet secretary from 1979-1987, Lord Wilson, who held the role from 1998-2002, and his immediate predecessor, Lord Turnbull.

Dave Penman, general secretary of the FDA, the union that represents senior civil servants, also gave evidence, telling the peers that his members were concerned about the "tone and nature" of the scrutiny they had come under from some select committees.