Nick Hurd: open data will provide the third sector with a powerful tool. Photograph: Felix Clay

Government officials, charity leaders and senior figures from the open data movement said on Monday that the UK must foster an environment of collaboration and transparency in order to share best practices from the public and private sectors with nonprofit organisations.

"Part of the value of civil society is holding power to account, and if this can be underpinned by good quality data, we will have a very powerful tool indeed", said Nick Hurd, Minister for Civil Society.

"The UK is absolutely at the vanguard of the global open data movement, and NGOs have a great sense that this is something they want to play a part in.

"There is potential to help them do more of what they do, and to do it better, but they're going to need a lot of help in terms of information and access to events where they can exchange ideas and best practice."

Third sector has for too long been misread as third place when it comes to nonprofits. Data has been widely touted as the future of the public and private sectors but when it comes to the adoption of big data initiatives, it is difficult to find a sector more ripe for revolution.

The opportunities offered by wider use of data in the third sector are many, from allowing individual organisations to provide donors and funding bodies with more tangible evidence of their successes, to opening up previously inaccessible datasets that could revolutionise their operations.

Peter Wanless, CEO of the Big Lottery Fund (BLF), which distributes lottery money to community-run projects, has already seen how better use of data and increased openness can streamline the funding award process, allowing deserving charities to get the money they need as quickly as possible.

"We are a massively oversubscribed fund, we are operating in a much tighter funding environment, so evidence is absolutely essential if the BLF is to move beyond random acts of kindness and to convince ourselves - and others - that we're learning from what went before", said Wanless.

The BLF has at any time 20,000 awards under consideration, but the traditional funder-fundee model has weaknesses that open data could be used to combat.

"There is a conspiracy of honesty: the organisations which apply to the BLF and get the money are desperate to show they're successful, because they worry that if they're not successful we'll withdraw the grant", said Wanless.

"This has led to a shift towards sharing a sense of success rather than being able to open up, celebrate and understand failure, and acknowledge that as an important part of the learning process."

Solving the fear of failure problem is one of the main arguments for adopting an open data policy in any sector or industry, and the 'fail fast' approach has been used by big data businesses for decades.

The competitive marketplace and bilateral nature of funding awards make this issue perhaps even more significant in the charity sector, and it is in changing attitudes and encouraging this warts-and-all approach that movement leadership bodies such as the Open Data Institute (ODI) will play their biggest role.

"We're in a new era of open data where we need permission to fail, we need to try to open up the whole conversation to allow everybody to see what we're doing, to learn from our mistakes," said Gavin Starks, CEO of the ODI.

An altogether different opportunity offered by open data is the ability for charities to use data directly to improve the service they provide.

"We use data to capture the complexity of how we deliver palliative care and to improve this process," said Ian Carey, CEO of Barnsley Hospice.

"We can demonstrate as an organisation that we're providing an important service to the people of Barnsley. Are we achieving our mission? Are we delivering a good and dignified death? We can use data to answer those questions.

"But we also want to know how we compare to others and this is where we're really keen for other hospices to publish some of their data, so we've got a bit more of a benchmark. For all we know, we could be rubbish in Barnsley, because it's very difficult to compare ourselves with other hospices that don't publish their data", said Carey.

Joining the ODI in driving and overseeing wider adoption of these practices is the Open Knowledge Foundation (OKFN). One of its first projects was a partnership with an organisation called Publish What You Fund, the aim of which was to release data on the breakdown of funding to sectors and departments in Uganda according to source - government or aid.

Without knowing such relatively basic information, charities distributing foreign aid to Uganda could have been blindly providing money for an already well-funded industry, or unknowingly overlooking a sector in desperate need of additional cash.

"We found huge disparities: more than half of spending on government projects in Uganda was coming from non-government sources", said Rufus Pollock, co-founder and director of OKFN.

"Disaster management was [funded] entirely by donors, but health was much more mixed. This was something the government of Uganda didn't even have. Publish What You Fund took this to them and they actually discovered spending they didn't know about.

Open data can often take the form of complex databases that need to be interrogated by a data specialist, and many charities simply do not have these technical resources sitting untapped. OKFN is foremost among a number of organisations looking to bridge this gap by training members of the public in data mining and analysis techniques.

"We're all familiar with the phrase 'knowledge is power', and in this case knowledge means insight gained from this newly available data. But data doesn't turn into insight or knowledge magically. It takes people, it takes skills, it takes tools to become knowledge, data and change.

"We set up the School of Data in partnership with Peer 2 Peer University just over a year and a half ago with the aim of enabling citizens to carry out this process, and what we really want to do is empower charities to use data in the same way", said Pollock.