NGOs can learn from mistakes made in setting up projects. Photograph: KK Arora/Reuters

NGOs battle for media attention, devoting considerable effort and energy into getting that crucial eyeball contact. Usually that means making the message as stark and sensationalist as possible, with the implicit message that the NGO knows exactly how to sort out the problem – whether that is tax havens, Aids or educating 10-year-olds in Tanzania.

So I'm full of admiration for a Canadian NGO that is breaking all the rules by publishing a failure report. Engineers Without Borders has bravely catalogued various mistakes in its projects. Project officers come clean in a series of snapshots of what they did wrong . So Owen Scott confesses that he thought he knew exactly what was needed in Malawi, where he was working on a water project. He secured the funding and got it sorted: an updated survey. But it only postponed the problem, which was that the district government didn't have the money to regularly update the survey. Scott admitted "prioritising tangible activities" and effectively using money as bribery.

Another project officer, in Ghana, confesses that she had a "humbling realisation" when she saw that she was just adding to the existing problem.

While working in a project in Zambia, Mark Hemsworth thought his task to support local enterprise was straightforward – a carpentry business needed a planing machine. He supplied the machine but it was badly damaged when fitted. Like plenty of unused machinery lying around rural Zamiba, parts and repairs were hugely difficult to arrange.

This is brave stuff. Anyone who has ever worked in aid projects will recognise all of it. The confidence with which aid workers can think they know what they are doing, plunge in and make countless mistakes. But this is the knowledge that NGOs keep well clear of their marketing departments. It's an ugly dishonesty that runs through almost all aid work, a painful underbelly to the very obvious idealism and good intentions.

Engineers Without Borders is boldly suggesting that we learn from failure, and by putting itself on the line, it is hoping to encourage plenty of other NGOs to do the same. It has set up a website admittingfailure.com and has already got promises from three NGOs to share their failures over the next week. The idea is to kickstart a more honest conversation with the public (who, after all, provide a lot of NGO funding) about what their money is doing.

In his contribution to the report, a long time development worker, Ian Smillie, points out that "development enterprise is notoriously risk averse; donors demand results and punish failure". Other experts argue that monitoring and evaluation is often so inadequate, that we simply don't know what works.

The current trends in UK and US aid agencies is to insist on value for money and clear evidence of results. So here is a counterblast which says that we must recognise the truth of that old proverb, that we can learn from our mistakes. But it's risky. What happens when an aid sceptic such as Bill Easterly starts using these cameos of aid failure to buttress their arguments? This innovative approach could provide fuel for the increasingly powerful anti-aid lobby.

On balance, it's probably a risk worth taking. A more grown-up conversation about NGOs and their work is overdue. BBC Radio 4's Ed Stourton did a very thought-provoking programme last week, arguing that "the aid sector is facing a crisis of identity" over its impact and effectiveness. In 60 years of aid, it hasn't delivered what people expected, he said. A range of interviewees pointed out how NGOs have grown into bureaucratic organisations that are often very distant from the people they claim to be empowering. Many have "internalised the logic of the marketplace", keen to get their hands on as much money as possible, creating tensions with their ethical commitments. One of the interviewees was Linda Polman, whose excellent book War Games raises many of these issues, and in particular shows how the media and NGOs use one another – and in the process often disastrously distort the reality on the ground.

So are we ready for a grown-up conversation about what NGOs do? Are they the force for good portrayed in their marketing? Or are we all colluding in wanting to believe their wild promises … saving babies' lives for a fiver?