I just read Revolution@State, a pretty amazing report by the Lowy Institute, an independent policy think tank, about the extent and depth of e-diplomacy initiatives at the State Department.

I came away with several impressions:

The Internet and social networking are central to how State does its business

The Net and social networking are transforming how State does its business

The Net is bringing about cultural changes at State

That third point is for me the most striking. The State Department has been hypersensitive about security. While that of course remains part of State’s DNA, the Department is also becoming realistic about the gains that can be made by not reflexively shutting down every proposal. For example, the Lowry report writes:

At a Twitter training course for State Department employees attended by the author, the 50 or so officers present — some of whom admitted to never having used social media — were exhorted to give it a go, you can’t go wrong. Policy guidane was barely mentioned.

Closer edamination reveals why this has not led to disaster. To begin with you are dealing with highly educated employees with a strong desire to keep their jobs…

Likewise, the report cites a new willingness to experiment and fail, which is essential for innovation but anathema to State’s traditional culture. Implicit in many of the initiatives, there is new emphasis on Need to Share rather than Need to Know; the latter policy optimizes for security at the cost of intelligence.

The report goes through the many offices directly involved in e-diplomacy, but singles out the 80-person E-Diplomacy group for special focus and praise, lauding its entrepreneurial spirit. That’s the group I’m proud to have been attached to that group for two years as a State Department Franklin Fellow, and, as they say at Reddit, I can confirm.

If you’ve had any interaction with the State Department — where in my limited experience I have met true patriots — you know that it is one of the least likely institutions to hop on the Internet train. I’d give credit to the transformation to three factors:

First, starting with Colin Powell, continued by Condaleeza Rice, and especially with Secretary Clinton (and her choice of Alec Ross (twitter) and until recently Ben Scott), the leadership has embraced these changes.

Second, groups like E-Diplomacy have served State by building tools that serve State’s needs, and have at the same time modeled the webby way of doing business. One great example is Corridor, State’s new professional networking environment, specially tuned to the needs and norms of State Dept. employees.

Third, the State Department’s 80,000 employees are on the ground around the world. This means that the organization is fundamentally reality-based, even when the leadership gets warped by politics. These Net-based initiatives are being embraced because they work. Likewise for the Net-based culture that is infusing State as more of the world and more State Dept. employees go online. Leaders of the e-initiatives such as E-Diplomacy’s Richard Boly combine a drive to achieve pragmatic results with an entrepreneurial appreciation of failure as a key tool for success.

I acknowledge that my personal experience of the State Department is warped by the amount of time I’ve gotten to spend with its webbiest elements. But I’ve also seen tangible evidence that a belief in openness, innovation, and connection is taking root there. The Lowy report confirms that. Worth reading.

Building a prototype process for federal agencies to source low-cost, high-impact solutions from innovative tech companies and startups.

2. My Gov – twitter: @ProjectMyGov #gov

Building a prototype that streamlines the 1,2000+ government/service websites, with more intuitive interfaces and the ability to accept feedback.

3. Open Data – Twitter: @ProjectOpenData #opengov

Open Data will continue the path set by NOAA’s release of data by further scaling the Health Data Initiative and releasing new databases in the energy, education, public safety, and nonprofit sectors

4. 20% Initiative – twitter: @ProjectTwenty

USAID-led project to transition from cash to electronic payments across public and private sectors. Aims: reducing corruption, improving safety, further opening the door to entrepreneurial innovation. (The name comes from the aim of getting 20% more bang per buck.)

5. Blue Button For America – twitter: @ProjectBlueBtn

Developing tools that enable individuals to utilize their own health records – current medications and drug allergies, claims and treatment data, and lab reports, etc. – to empower them to improve their own health and healthcare.

700 people have applied for the Fellowships. They’ll be announced on Thursday. The fellowships last for six months. The projects will combine the private and public sectors, and will be done in full public, with as much crowd participation as possible. (TechPresident has a good post about it.)

The FCC has launched a site for developers that provides APIsso that anyone can create apps that draw on FCC data. Heres the first one they list: “Over 1 million user speed tests were generated from FCC Consumer Broadband Test. This API delivers data on the number of tests, average user download/upload speeds, and more.”

The White House also launched Challenge.gov, an Innocentive-like site where government agencies can pose challenges, offering prizes for the best solutions. There are almost 50 challenges posted so far.

Beth Noveck is deputy chief technology officer for open government and leads President Obama’s Open Government Initiative. She is giving a talk at Harvard. She begins by pointing to the citizenry’s lack of faith in government. Without participation, citizens become increasingly alienated, she says. For example: the rise of Tea Parties. A new study says that a civic spirit reduces crime. Another article, in Social Science and Medicine, correlates civic structures and health. She wants to create more opportunities for citizens to engage and for government to engage in civic structures — a “DoSomething.gov,” as she lightly calls it. [NOTE: Liveblogging. Getting things wrong. Missing things. Substituting inelegant partial phrases for Beth’s well-formed complete sentences. This is not a reliable report.]

Beth points to the peer to patent project she initiated before she joined the government. It enlists volunteer scientists and engineers to research patent applications, to help a system that is seriously backlogged, and that uses examiners who are not necessarily expert in the areas they’re examining. This crowd-sources patent applications. The Patent Office is studying how to adopt peer to patent. Beth wants to see more of this, to connect scientists and others to the people who make policy decisions. How do we adapt peer to patent more broadly, she asks. How do we do this in a culture that prizes consistency of procedures?

This is not about increasing direct democracy or deliberative democracy, she says. The admin hasn’t used more polls, etc., because the admin is trying to focus on action, not talk. The aim is to figuring out ways to increase collaborative work. Next week there’s a White House on conf on gov’t innovation, focusing on open grant making and prize-based innovation.

The President’s first executive action was to issue a memorandum on transparency and open gov’t. This was very important, Beth says, because it let the open gov folks in the administration say, “The President says…” President Obama is very committed to this agenda, she says; after all, he is a community organizer in his roots. Simple things like setting up a blog with comments were big steps. It’s about changing the culture. Now, there’s a culture of “leaning forward,” i.e., making commitments to being innovative about how they work. In Dec., every agency was told to come up with its own open govt plan. A directive set a road map: How and when you’re going to inventory all the data in your agency and put it online in raw, machine-readable form? How are you going to engage people in meaningful policy work? How are you going to engage in collaboration within govt and with citizens? On Tuesday, the White House collected self-evaluations, which are then evaluated by Beth’s office and by citizen groups.

How to get there. First, through people. Every agency has someone responsible for open govt. The DoT has 200+ on their open govt committee. Second, through platforms (which, as she says, is Tim O’Reilly’s mantra). E.g., data.gov is a platform.

Transparency is going well, she thinks: White House visitor logs, streaming the health care summit, publishing White House employee salaries. More important is data.gov. 64M hits in under a year. Pew says 40% of respondents have been there. 89M hits on the IT dashboard that puts a user-friendlier interface to govt spending. Agencies are required to put up “high value” data that helps them achieve their core mission. E.g., Dept. of Labor has released 15 yrs of data about workplace exposure to toxic chemicals, advancing its goal of saving workers’ lives. Medicare data helps us understand health care. USDA nutrition data + a campaign to create video games to change the eating habits of the young. Agencies are supposed to ask the public which data they want to see first, in part as a way of spurring participation.

To spur participation, the GSA now has been procuring govt-friendly terms of service for social media platforms; they’re available at apps.gov. It’s now trying to acquire innovation prize platforms, etc.

Participation and collaboration are different things, she says. Participation is a known term that has to do with citizens talking with govt. But the exciting new frontier, she says, is about putting problems out to the public for collaborative solving. E.g., Veterans Benefits Admin asked its 19,000 employees how to shorten wait times; within the first week of a brainstorming competition, 7,000 employees signed up and generated 3,000 ideas, the top ten of which are being implemented. E.g., the Army wikified the Army operations manual.

It’s also about connecting the public and private. E.g., the National Archives is making the Federal Registry available for free (instead of for $17K/yr), and the Princeton Internet center has made an annotatable. Carl Malamud also. The private sector has announced National Lab Day, to get scientists out into the schools. Two million people signed up.

She says they know they have a lot to do. E.g., agencies are sitting on exebytes of info, some of which is on paper. Expert networking: We have got to learn how to improve upon the model of federal advisory commissions, the same group of 20 people. It’s not as effective as a peer to patent model, volunteers pooled from millions of people. And we don’t have much experience using collaboration tools in govt. There is a recognition spreading throughout the govt that we are not the only experts, that there are networks of experts across the country and outside of govt. But ultimately, she says, this is about restoring trust in govt.

Q: Any strategies for developing tools for collaborative development of policy?
A: Brainstorming techniques have been taken up quickly. Thirty agencies are involved in thinking about this. It’s not about the tools, but thinking about the practices. On the other hand, we used this tool with the public to develop open govt plans, but it wasn’t promoted enough; it’s not the tools but the processes. Beth’s office acts as an internal consultancy, but people are learning from one another. This started with the President making a statement, modeling it in the White House, making the tools available…It’s a process of creating a culture and then the vehicles for sharing.

Q: Who winnowed the Veterans agency’s 3,000 suggestions?
A: The VA ideas were generated in local offices and got passed up. In more open processes, they require registration. They’ve used public thumbs up and down, with a flag for “off topic” that would shrink the posting just to one link; the White House lawyers decided that that was acceptable so long as the public was doing the rating. So the UFO and “birther” comments got rated down. They used a wiki tool (MixedInk) so the public could write policy drafts; that wiki let users vote on changes. When there are projects with millions of responses, it will be very hard; it makes more sense to proliferate opportunities for smaller levels of participation.

A: We’re crowd-sourcing expertise. In peer to patent, we’re not asking people if they like the patent or think it should be patented; we’re asking if they have info that is relevant. We are looking for factual info, recognizing that even that info is value-laden. We’re not asking about what people feel, at least initially. It’s not about fostering contentious debate, but about informed conversation.

A: What do you learn from countries that are ahead of the curve on e-democ, e.g., Estonia? Estonia learned 8 yrs ago that you have to ask people to register in online conversations…
A: Great point. We’re now getting up from our desks for the first time. We’re meeting with the Dutch, Norway, Estonia, etc. And a lot of what we do is based on Al Gore’s reinventing govt work. There’s a movement spreading particularly on transparency and data.gov.

Q: Is transparency always a good approach? Are there fields where you want to keep the public out so you can talk without being criticized?
A: Yes. We have to be careful of personal privacy and national security. Data sets are reviewed for both before they go up on data.gov. I’d rather err on the side of transparency and openness to get usover the hump of sharing what they should be sharing. There’s value in closed-door brainstorm so you can float dumb ideas. We’re trying to foster a culture of experimentation and fearlessness.

[I think it’s incredible that we have people like Beth in the White House working on open government. Amazing.]

He begins by modestly disclaiming credit. His group didn’t build the 13M-person mailing list. He at first didn’t want to do Twitter. President Obama was a spectacular campaigner. Everyone was clear about what needed to be done. He says he was part of an incredible team that executed well, including, Chris Hughes and Joe Rospars [and one more person whose name I did not get – dw].

Lessons from the campaign: The importance of being genuine. You can’t fake who you are. He points to the fact that old people who try to sound cool always sound ridiculous. “If you fail the genuineness test, you will lose the ability to make a connection with your supporters.” A motto from the campaign: “Respect. Empower. Include.” When they went into Iowa (where they had an astonishing 33 offices), they understood they were visitors. [Unlike the Dean campaign four years earlier? -dw] Also: Be generous with ownership. Empower your supporters to be your best advocate. “People trust other people about you more than they trust you about you.” “Giving away control means you have to teach your values.” That way, instead of telling them what to do, you can give them tools to act and they will do the right things. They’re going to say some things you don’t want them to, but they will pass the “genuity” test more than you will.

This campaign was won in the field, not online. The impressive thing about having 2M Facebook supporters was that half of them went out and did something for the campaign. When you start a campaign, you’re in a relationship with people. When the campaign started, FB didn’t have fan pages, so you’re just another profile, so you have to act like one, respond to people, etc.

Tell good stories, he says. Lots of people can write a good email, but few can structure an email program that will last for two years.

“The Obama campaign was a good object lesson for technology serving purpose.” Tech is merely a means. “We didn’t use stuff in the campaign that didn’t help us win. That was my mantra: ‘How does this help us win?'”

He says that it was amazing that Secretary Clinton gave an Internet Freedom speech, but that much was absent from it. Our goal is promote democracy, but democracy depends on lots of other things in the culture. It’s myopic to think that democracy is banging on the door of totalitarian states. It can be difficult to go from an authoritarian regime to elections; elections may not be the first thing to do. If we want Net Freedom, we should make it more partisan. We are fine with censorship of some things, e.g., child pornography. So, we should be more explicit about what we value, rather than pretending to support Net Freedom overall. We need to articulate what the purpose of Net Freedom is.

Q: What do you mean by stories?
A: The story on the Obama campaign was born out of Pres. Obama’s personal story as a community organizer, who wanted to empower others to enable change. [I’m paraphrasing poorly.] When you run an email campaign, you have a specific goal, but if it’s not part of the wider message you get a clanging cymbal [symbol?] problem. So, if you send an email asking for a large donation, it clangs against the story that this is about empowering a wide range of people, so we wouldn’t send that email. Joe Rospars came up with the idea of flying 4 low-dollar donors to “dinner with Barack”; usually, you have to donate $100K’s to get dinner with the candidate. Pres. Obama loved these opportunities. “In the summer of ’07 when we were this crazy longshot, we were telling Pres. Obama’s personal story.” Even back then Pres. Obama talked about this being “your campaign.” “If that’s going to be who you are, that’s how you have to stay who you are.” E.g., you have to remember what you’ve told your supporters.

Q: How do you get all these people on the same page? And what mistakes did you make that we can learn from?
A: Joe Rospars, the dir of new media, was a peer with the dir of communications. Joe reported to David Plouffe. It was important for us to be able to say no credibly. There was little drama. The top has to lead with values. Not everyone is comfortable talking about values because they think it’s too soft. “It’s hard to distill your passion into something that is easily articulated and easily understood.” “The humility of knowing who you are is really important.” It’s missing from our Net Freedom policy.
In terms of mistakes, it was great to have the freedom to try things and fail. We thought an honest way to do fund-raising: we’d let people buy the things we need by posting a list. You’d rent us a van and then we’d send you a photo of the van. Sounds great, but it didn’t work at all. Too complicated. We tested an email campaign this way, where you could buy a line item or you could get a t-shirt for contributing, and the t-shirt won. (It’s important to understand when you’re in a tech echo chamber, he says.)

Q: [ethan] At a talk one of Howard Dean’s organizers in response to a question said online campaigning means getting a lot of email addresses and raising money from from. You’ve addeed: Make sure your supporters understand your values and let them speak on your behalf. Great for politics and non-profits. But there’s a gap between that and the larger movement-building that some of us hope will come out of the Net. [That Dean person’s characterization is quite unfair to the Dean campaign, which built open source social networking tools, and went out of its way to make sure supporters felt empowered to talk for the campaign. See Joe Trippi’s book. – dw]
A: Emails are great for raising money. But, converting online interest to offline action is the most important metric. Everything we did was based on mobilizing people, including building tools to allow people to volunteer easily online. E.g., signing up for a shift is a gigantic pain; we did it fairly effectively with a lot of different tools. We had MyBarackObama that let people find each other locally and go out into the community. Those are the tools that are most interesting for general advocacy. Cf. Chris Hughes’ Jumo that will create a community and then bring organizations to the table. Chris is a genius, says Michael. For engagement online, you have to go where people are; most advocacy campaigns don’t have the advantage of being on TV every day for two years the way we did.

Q: [wendy] What’s your favorite tool for listening to people, and how many do you listen to as opposed to making people feel like they’re being heard?
A: We handled this by having lots of staff. Don’t go into a social medium that you’re not prepared to support with staff. By the end, we had almost 100 paid staff in HQ on new media, plus people in every state, plus a lot of interns, many of whom were helping us respond to people individually. We did a lot to make people feel heard; we’d ask people for their stories and feature some of them. “The only way to make people feel heard is to hear them. You need a person answering a question.”

Q: [me] The Dean campaign went with the idea of horizontal links rather than staffing up to get people to feel connected. Also, they thought those bonds would later form a movement for governing. Did you think about how the tools you were building to win the election might be used to govern?
A: We knew the campaign was temporary. OfA has continued. But when you only have horizontal connectiions, they’re only teaching each other, and there’s less room for you. That may be one reason they were so powerful after the Dean campaign – they weren’t that connected to the campaign. The whole goal of our campaign was to take on K Street, to change the way influence works.
Q: [me] Do you share the criticism that OfA wasn’t carried forward enough? David Plouffe was brought in to reinvigorate it recently?
A: Everything could be done better, but it’s really hard to carry this forward. It’s far more boring than campaigning. Macon Philips is doing a great job with the new media team in the White house, but it’s hard, in part because of legal issues. And David Plouffe was brought back maybe also because it’s getting to be time for midterm elections.

Q: Involving communities?
A: It’s still a goal. Unrealized goal. Doing local stuff seems easier when you’re campaigning and have lots of local resources. It’s important not to over-promise and under-perform. If you don’t have the people to support local people in a few places, not supporting local people everywhere is bad. It looks like you don’t care. But, all of those goals are still goals.

Q: What else did you do culturally or managerially to keep people focused on the campaign’s values?
A: There was a focus on “What are you doing for Iowa?” Iowa was so key to the campaign. I love Iowans now. Iowa was a big part of who we were for the first year. It’s very hard to be a good boss in a campaign: everything is an emergency, you never have enough resources, no one sleeps. The values stuff is central or else you’ll burn out. People aren’t paid well, overworked, under-resourced, which is a recipe for making people undervalued. You have to find other ways to make people feel taken care of and heard. There was a lot of bottom up, but there’s also top down; you have to cut off conversation and just go. Campaigners are very difficult, so you need to take care of your people.

Q: Your tools matched your values. Do you have problems with people using it for other values?
A: Our tools were election tools. There’s a tension between a national and a local goal. But, you’re right that form follows function. When goals change, sometimes tools adapt, but not always. You need more than one tool, and if your problems change, your tools have to change; it’s hard for big orgs to be that nimble. That’s one reason why open source tools are often useful.

Q: [donnie dong] The Net is so interactive that if someone doesn’t like your values that s/he shuts you out, and s/he takes others with her/him
A: Absolutely. This is how the Net works. You get negative comments. It’s not a problem with the tool; with interactive tools, you now know the symptoms that were there but you didn’t know about.

Q: Criticisms make the praise more meaningful.
A: I totally agree.

[herkko] What legal problems? Any copyright problems?
A: We made sure to educate the people we were empowering. If people were going to fundraise for us, we explained the rules to them. Of course there were problems, but nothing too severe. “There was no point in the campaign when I was worried about what we were doing.” We set a high bar for being a supporter. Hanging out on the email list was not enough. If you’re going to do more, you have to take responsibility for your actions. We tried to educate people properly and give them the right tools.

[donnie] Difference between the content you created and the user-generated stuff in terms of quantity?
A: No idea. Most of the content was coming from the campaign. Personal content tends to be more personal. That’s the reason why it’s good. A person telling a story about their own life is more engaging than me telling a story about that person. Ideally the user-generated content and campaign content works together. UGC is worth the danger.

Q: What did you do in social media that resulted in the most offline activity?
A: Measuring social media is difficult. FB’s calendaring structure is good at supporting events. Text messaging was good for getting volunteers to go to a particular place to do some work. We never got to the Holy Grail of knowing who is where and can do what. The data integration posed by campaigns is overwhelming. You have disparate data sets of disparate quality, and you’re not going to be able to match them all up. Even though it’s hard, you need to try to measure. How do you decide what gets credit for why someone went out to vote — TV ad, FB message, etc.?

[the talk ends, but no one leaves]

Q: Where are you addressing your concerns about Sect’y Clinton’s talk?

A: Privately so far. This is the first I’ve talked about it in public…

After watching President Obama at the Republican Caucus, it’s clearer than ever that press conferences need to go the way of press releases. They are just too constricted for the opportunities and temper of the new connected age. The reporters are too interested in getting headlines, and would rather appear fair and balanced than chase down the truth. We do better, it turns out, when the President is questioned by people who can acknowledge that they really, really disagree with him.

So, what do we replace press conferences with? Or, more realistically, what can we supplement them with?

We know that Question Time in the British Parliament works well in Britain. But, it’d be good for democratic reasons to open it up to The People. Also, why should you have to disagree with the President to press him on an issue?

The problem, of course, is deciding who among us gets to ask a question. So, how about if questions were awarded to people who participate in particularly constructive ways , on any side of an issue, in the comments section of the White House blog (once comments are allowed)? This would be a mighty incentive for engaging civilly in the comments section.

But, then we’d need a way to decide who to pick. If it’s done algorithmically (e.g., have two buttons: I like this comment and I disagree with this comment), it can be gamed. If it’s done by human editors at the White House, it’s subject to charges of favoritism. So, how about if two or three known and respected people in their communities were chosen to select questioners from among the commenters; these people would represent different political views. New selectors would be chosen for each Presidential Q&A session.

Obviously, I don’t know exactly how to do this. But, in the Age of the Web it seems clear to me that we need to supplement press conferences with forums that replace objectivity with transparency, timidity with passion, and professionals with all of us.

The White House is inviting us to talk with the President’s advisers in the runup to the State of the Union. The first is at 3:30pm EST today, on the environment. How open the conversation will be of course remains to be seen, and presumably the speech is already drafted. Still, it could be toward the high end of expectation, and, in any case, it’s interesting to watch the White House try to figure out how to scale conversation.

The report takes “engagement” as the desideratum of e-gov, and was produced in an engaged, public way. It recommends transparency, the use of open standards and Creative Commons licenses, loosening copyright restrictions on “orphan works,” tax benefits for donations to “info-philanthropies,” encouraging agencies and government employees to engage online, and more.

Personally, I think the draft â€” from its principled overview to its broad areas of application â€” is a blueprint for democracies everywhere. (Disclosure: I was on the International Reference Panel, which mainly means I got to comment on some earlier drafts.)

Plenty is being written already trying to parse, understand, and come to terms with Larry Lessig’s article “Against Transparency” in the New Republic. Ethan Zuckerman does his usual outstanding job in clarifying ideas sympathetically. Transparency advocate Carl Malamud responds to Lessig. I presented my own “walkthrough” of the article. The New Republic has run Tim Wu‘s response, which agrees with Lessig in important ways. The New Republic has also run four other responses, including an excellent response from Ellen Miller and Michael Klein, founders of the Sunlight Foundation, the leading advocate for transparency. (My response is included in that set of four.) Aaron Swartz prefigured Larry’s argument in a piece he posted in April: “Transparency is bunk.” Plenty to chew on.

I want to briefly expand on the article’s import.

At the end, Larry expands his own argument to cover “Internet triumphalism.” Over the past couple of years, we’ve been seeing Net triumphalism waning, at least in the circles I travel in. Triumphalism is the notion that the war has been won. It’s over. Net triumphalism thinks that the new tech is in place, cannot be removed, and will change everything. It thus includes Net techno-determinism, i.e., the idea that the mere presence of the Net has predictable, determinate, and inevitable effects. Triumphalism adds: Yay!

Net triumphalism seemed more plausible back in the days when the demographics of the participants were pretty homogeneous, masking the role culture played in the homogeneous effects the Net was having. As regimes have censored the Net in ways the Net has not routed around, as Albert-Laszlo Barabasi and then Clay Shirky showed us that the Net tends towards the old patterns of unequal influence, as the mere networked presence of Howard Dean supporters failed to end GW Bush’s reign of error, naive Internet Triumphalism has become unsupportable. As Joe Trippi said, we need mouse pads and shoe leather. As Aaron Swartz says, we need narrative journalism as well as the Web. As Larry Lessig says, we need political reform as well as the Web. Indeed, as Aaron and Larry point out, the sunlight of transparency casts shadows as well.

I think “Against Transparency” misidentifies the source of the threat and undervalues the benefits of transparency-as-the-default, even as I agree with Larry’s cautions and his policy agenda. I nevertheless think it is one more marker in incremental extirpation of Internet triumphalism. Some of the pain reading his article causes old-time Net enthusiasts like me comes from that. It’s the right pain to feel, even if we disagree with the particularities of Larry’s article.