In December, the Obama administration directed federal agencies and departments to implement "principles of transparency, participation and collaboration," and provided deadlines for making government information available online. At the same time, citizens and journalists are developing new technologies to manage and analyze the exponential increase in data about our civic lives available from governmental and other sources. What new ways of gathering and presenting information are evolving from this nexus of government openness and digital connectedness? Our speakers Linda Fantin, director of public insight journalism at Minnesota Public Radio, and Ellen Miller, executive director of the Washington-based Sunlight Foundation, will explore this and related questions. Chris Csikszentmihalyi, director of the MIT Center for Future Civic Media moderates the discussion.

Linda Fantin from Minnesota Public Radio started the forum by discussing her role in creating the Public Insight Network (view Linda Fantin's presentation slides (PDF)). She explained that while the technology has changed over the years the basic tenets have remained the same: many people have expertise – not just a few; people have knowledge and want to share what matters to them; listening to a broad range of people makes news coverage more relevant and credible; and increasing relevance builds trust and audience.

Fantin continued by describing the type of people that bring value to the Public Insight Network. The mechanic who's worked on a specific aircraft engine for decades, for example, or someone who's been through a foreclosure rather than simply an expert on foreclosures. The Public Insight Network also provides access to many populations often beyond the reach of traditional reporting techniques. She described surveys on inmates (conducted with pen and paper by family members) that led to an expose on conditions that led to riots in California's Chino prison. These people can help make stories more relevant.

According to Fantin, the Network – which now includes more than 80,000 sources – was built through the website, online surveys, on-air encouragement by NPR, over Twitter and Facebook and through games and face-to-face interactions. Questions are posed to the community and those who respond are added to the database.

What's critical about the mission of public insight journalism is what's done with the insights. This means getting questions out to the right sources. Doing this successfully means ongoing engagement with sources. Journalists maintain contact with their sources on a monthly basis – even if they aren't working on a specific story. This leads to greater trust and more willingness to speak and provide insights in the future.

In addition to the sources the Public Insight Network gathers, there is also a growing network of newsrooms across the country that are tapping into the Network to make their stories stronger. Each newsroom has an analyst who helps connect reporters with appropriate citizen sources. This model gives newsrooms access to a wider range of voices, provides individuals with the means to share their stories and improves the overall quality for reportage for everyone.

Ellen Miller of the Sunlight Foundation then spoke about that organization's aims (view Ellen Miller 's presentation slides (PDF)). The goal of this four-year-old organization is to use the power of the Internet to improve access to information. They provide technology to analyze information and tools to help improve our insights and understanding. Their work falls into five categories: digitizing data, advocacy, tools, engagement and media production.

Digitizing data is the core of the Sunlight Foundation's work. They are working to collect and make available as wide a range of government information as possible. Some of the datasets they have made available allow people to view all government grants and projects; another shows where government subsidies are flowing; another tracks the influence of foreign lobbyists.

According to Miller, the Holy Grail is a single data repository. An important current step is the National Data Catalog, a catalog of all the available government data sets. People can provide data and access data to create new tools and applications. The Sunlight Foundation also works with government agencies to identify new data sources (many of which even the agencies themselves are not necessarily aware of) and advocate for their being made publicly available.

The Sunlight Foundation provides a broad range of tools – from widgets which allow journalists and bloggers to add information to their properties to sites like OpenCongress which makes legislative data available, to Real Time Congress which brings Congressional activities to smart phones in real time.

Engagement has long been one of the core missions for the Sunlight Foundation. There are currently more than 2,000 developers helping to create applications for the Foundation. The organization holds Transparency Camps to bring people together to share ideas and experiences on how to make government more transparent. They've also kicked off a new campaign – Public=Online – to provide real-time, online access to government information and they are working with government agencies and officials to move this forward. The goal of all these engagement activities is to help empower communities.

Finally, as publishers, the Sunlight Foundation, maintains a number of blogs, supports a reporting group that aggregates and makes available the information the Foundation collects, and creates data visualization to help people understand the important ideas that can be buried in complex data sets.

As Miller explained, technology is pushing power from the center out to the edges and that's the only thing that will help break Washington's control of information.

Following the presentations, Chris Csikszentmihalyi, director of the Center for Future Civic Media, addressed the speakers.

Csikszentmihalyi: As more and more data becomes available to the public, how do you see the role of journalists changing?

Fantin: There will certainly be a role for journalists because there's always a place for sense-makers. Technology has handcuffed reporters to their desks; it's one of the reasons journalists go to the same sources all the time; because they know and trust them. Will citizens become their own reporters? Most will not – they want to be a part of the process – not a replacement for reporting.

Csikszentmihalyi: We're both funded by the Knight Foundation. Another project they are funding is MediaBugs – which encourages news organizations to take responsibility for misreporting or misinformation. Are you hearing about less reporting errors because of greater openness?

Fantin: It would be great to be able to say that was the case. It helps that in the newsrooms we support there are people devoted to listening and talking to the public. It leads to more and faster corrections. Right now the relationship between reporters and the public is very transactional. Our goal is to encourage more interaction in the journalism process.

Csikszentmihalyi: What are the principals you use to think about design?

Miller: In some cases, information and interconnections are so complex that even the best writer would be at a loss. Visualization and infographics help us reach many more people. There's no single principal. At the Sunlight Foundation we use the “Ellen Test.” If it doesn't work for me it doesn't work. For us, explaining the complicated relationships that are at the heart of Washington is absolutely key, and so we're trying to do more and more in a visual direction.

Csikszentmihalyi: You've worked with LittleSis.org which tracks relationships among politicans, lobbyists and corproations . This site has adopted a more visual and nodal style for presenting information. What kinds of expertise are you looking for to present this data?

Miller: The reason we're using visual models is to help reach a more popular audience. We believe that sites like Transparency.org allow people to become their own investigators. The ability to combine data and social media with the ability to directly contact legislators tells us we should be designing with this type of purpose in mind. This will allow people to do a far better job of monitoring the behavior of their representatives and contacting them when the need arises.

Miller: It's very difficult to say. I predict that corporations will not get into direct spending for fear of alienating customers. This is why real-time information – to track whom is spending, hiring lobbyists, etc, - is so important. The most nefarious impact will be that the lobbyists will have more resources to pressure members.

Audience Discussion

Question: A core issue is redaction. The more data, the greater need to weed through it for the right data. How can we use this information better?

Miller: The success of this will lie in broader citizen engagement. People need to have access, know they have access and know what to do with that access. There need to be core users in each community. What we're seeing in the government is a huge opening of data. We need to understand that the government holds data of interest to us. We don't always know about the data that's out there. The first goal is simply to get the data and then to figure out what to do with it.

Fantin: We can also do a better job of asking people what kind of information they want. We are focused on journalists but there are other audiences out there that might want other kinds of information. The game Budget Hero, for example, made large and complex data sets available in a cool and engaging way.

Question: You said the primary purpose is to get the data out there. What kinds of direct activities are you supporting?

Miller: Our allies come from both the right and the left. To date, we fund people who digitize data and people who have interesting ideas for websites or tools that provide access to that data. We have not funded any kind of data-driven activism. The Public=Online project will be about encouraging more transparency at all levels of government so that will be a fairly direct activity.

Question: From an information theory perspective more data isn't necessarily better. The value is in the narrative that can come from the data. Who should control that narrative?

Fantin: More people should have a hand in controlling the narrative. Journalism is a good place for it start since journalism isn't a conversation but nor is it a lecture. The sense of journalistic curiosity and the means of the media to tell the stories are both critical to the narrative.

Miller: I react to the notion of control. For too long control has been in too few hands. That's changing and it's a good thing. The fact that there are millions of bloggers out there – to me that is small “d” democracy. This is an exciting time in the civic education and participation of people. I like being able to filter the data that pertains to my interests.

Question: With the Public Insight Network – what are the expectations of people who submit themselves as experts? Will they be cited, protected, credited? Are the newsrooms using it? Respecting it?

Fantin: The relationship is based on the assumption of confidentiality. No information is used without a source's permission. They accept that reporters will be able to contact them. There's a process in place to thank and acknowledge the data that is provided. Part of the thrill of journalism is the aphrodisiac of discovery. To the extent that public insight journalism takes some of that away may make the process of journalism less appealing. But many reporters and editors are embracing this. The network is being used in some incredibly creative ways.

Miller: How many people work the Network?

Fantin: Usually there is one analyst per newsroom but this varies. The Corporation for Public Broadcasting is going to fund analysts and reporters in journalism centers around the country so that will expand usage.

Question: One imagines that those in power are reluctant to provide information – how do you encourage transparency?

Miller: It's true; there is a firewall of sorts around Congress. But people – 80 percent regardless of their party – want transparency. There are positive steps being taken. There is also a general cultural change in terms of public expectation of access to information. We can argue with officials that this information is already public and this is just going to make things faster or easier.

Csikszentmihalyi: I know people have been putting public data online since the 1980s.

Miller: We can show the government ways to do this more effectively. We also want legislation to force government to do this. Today, this administration has very good transparency but there's no guarantee that this will continue to be the case in the future.

Fantin: There's plenty of information that isn't hard to get. And sometimes, that easy information can lead to good, big and unexpected stories.

Question: How do you get the public engaged as sources? How do you deal with issues with self-selected responders?

Fantin: All of our responders are self-selected so we need to actively seek out balanced views. When you ask people for information on things they're interested in and care about engagement isn't that much of a challenge.

Question: Can you address the dark areas? Information that you'd like to get that you can't get access to.

Miller: We do mostly work with public information. We only count information as public if the information is online. We're funding a program to identify public information that isn't publicly available. We're also encouraging all agencies to conduct an audit of all the information they produce and how it's being made public and when it will be available online.

Csikszentmihalyi: How do you approach the issue of private data? Should there be a Sunlight Foundation for private data?

Miller: There are clearly privacy issues – even around private data that is provided to the government. To what degree is it redacted? One challenge in the age of electronic documents is we don't know even how many records are redacted since there is no original copy with content obscured.

Question: Who are the people in government who are most responsive to the move toward transparency? How do you handle leaked information?

Miller: There are lots of non-profits that have been advocating for access to government data for years. These groups have been part of the discussion for a long, long time. The Federal CIO, CTO and Cass Sunstein at the OMB have been great. On the Hill the advocates have come from both sides of the aisle. I think the fact that people in government are being allowed to be experimental with information is tremendous.

Fantin: There are many states that are doing transparency to one degree or another without external pressure. Not all governments want to make all government information secret.

Miller: Sunlight doesn't really deal with whistleblower information. There are other organizations that do.