Donald Rumsfeld's colourful and controversial public career is bookended by two big appointments: in November 1975 he became the youngest US secretary of defence; in December 2006 he was fired as the oldest holder of that office. In the interim, he served several presidents, came close to vice-presidential appointment and was actively engaged in commercial life. By any standard, his has been a rich and interesting life.

Yet his memoir is infected with a bitter tone, stemming from close association with the continuing difficulties of Afghanistan and Iraq, and the torture legacy. This is surprising for a character whose public image is of unbridled optimism and straight talk. Characteristically, he takes on former colleagues from within the Republican establishment, including Vice-President Nelson Rockefeller, Colin Powell and John McCain. His real ire, however, is reserved for the lawyers, senators, congressmen, retired military, judges, academics and "self-proclaimed human rights activists" who have blighted his twilight years.

This over-long memoir is essentially an advocacy document. It has achieved some attention for the one big regret that is offered: his failure to resign when the images of abuse at Abu Ghraib became public. Beyond that, there is no apology for Iraq, Afghanistan or detainee abuse, and considerable hubris. His words may have been "ill-chosen" (for example in relation to the "stuff happens" comment about the looting in Baghdad), and he may have made the occasional "mis-statement" or "unintentional" comment. But for the most part he is unrepentant: the Bush administration basically "got it right" on the big issues, including Iraq, even if the policy cost a "high price" in terms of the lives of thousands of American servicemen and women and "hundreds of billions of dollars". The silence as to the costs on the lives and security of others is telling. He hints at responsibility, but can't actually articulate it.

There is also a defensive posture, evident from the opening pages. Like a good litigator, he opens with his weakest point: the notorious 1983 photograph showing him in a smiling handshake with Saddam Hussein. Not easily explained away, the image becomes an opportunity to bash the French (who were much closer to Saddam than he ever was). The difficulty is that we cannot know whether the conversation he shares is accurate or complete, and therein lies this book's greatest problem. The reputation for competence and rigour is by now so shot to bits that we don't know what is fact and what is unfact.

This is particularly telling in the extended efforts to address detainee abuse. In November 2002, he signed a notorious one-page memorandum authorising 15 interrogation techniques for use on Mohammed al-Qahtani, accused of involvement in the 9/11 plot and held at Guantánamo. The techniques violated the Geneva Conventions and the US Army Field Manual, and their application over 54 days amounted to torture. (An allegation that his techniques migrated to Afghanistan and Iraq has been toxic, and damaged his reputation to the extent that foreign travel is risky.) He could have avoided the issue, and deserves some credit for taking it on. But his account is hugely inaccurate. He boldly asserts, for example, that "[a]t the time, my approval . . . was uncontroversial". That is completely wrong, and he must know it; prior to approval his office received clear objections from all four branches of the US military: the air force expressed "serious concerns regarding the legality of many of the proposed techniques", some of which could be construed as torture, and similar views were expressed in writing by the army, the marine corps and the navy. Rumsfeld was warned of the risks of criminal investigation before he signed off, and still he signed off. Some would call that criminal recklessness.

And he still claims that the Abu Ghraib images "had nothing whatsoever to do with interrogation or intelligence gathering", since no fewer than 12 reviews and investigations found no evidence "that abuse had been encouraged or condoned by senior officials in the Defense Department". The problem is selectivity of evidence: he just ignores the inconvenient December 2008 US Senate Armed Services Committee report – unanimous and bipartisan – which concluded that his authorisation "contributed to the use of abusive techniques . . . in . . . Iraq".

A reasonable reader should therefore treat every factual assertion with caution. That applies to the partial, self-serving account of the run-up to the 2003 Iraq war, a policy made on the hoof and without the application of critical thinking as to consequences. His case rests on such thin ice that he even invokes the Guardian for the proposition that there was broad support for the possibility of Saddam's involvement in 9/11. No citation is provided for that particular allegation. True, he recognises a failure of "deliberate, systematic" post-war planning, but this was the responsibility of others. There are nuggets that may be of interest to bodies such as the Chilcot inquiry. Astonishingly, he tells us that just one week before war began the administration finally agreed on an Iraqi Interim Authority, but the "exact timing and execution were left to be worked out". We also learn that President Bush "made the decision to go after Saddam Hussein" as early as 11 January 2003, making it clear that all that followed at the UN was a sham.

Had he been running a fairground stall Rumsfeld might be forgiven for his lack of attention to detail, a mind lacking in rigour, and for the telegenic soundbites that are the stuff of theatrical plays. But for more than five years he ran one of the largest organisations in the world, the US Department of Defence; in his hands it wrought mischief, mayhem and misery, without making the US or the world a safer or a better place, in the face of very real threats and dangers. That is his tragedy, and also ours. He and a small coterie of compliant associates ignored knowns and unknowns, and for that we are all worse off.