When somebody asks me for advice on how their government agency should run a hackathon, my response is simple: “please don’t.” That’s because it’s very easy to put on a bad hackathon, and it’s very hard to put on a good one.

Deciding whether to hold a hackathon is about aligning expectations with reality. A hackathon will not result in new software. It will not result in new businesses. It will not be a source of free consulting time for your agency. However, a hackathon may be a way to build a community around your agency’s data, or introduce that data to an existing community. It may be a way to teach people about your data. It may be a way for you to observe how developers attempt to understand and employ your data.

Any hackathon built on the assumption that it will conclude with the creation of success stories is a hackathon that will fail.

However, I do want to describe the unusual hackathon that one government recently held, because I think it presents a compelling new model. In August, Virginia’s Secretary of Technology put on a two-day “Datathon” in Richmond, which included six agencies, with five people representing each agency. They were challenged to build tools using only open data provided by state agencies, using at least two datasets from two different agencies.

Although there was the usual structure of a competition with winners named, the point of this wasn’t really to create software or websites, and it wasn’t to get the participants to quit their jobs and launch new businesses. It was to get state software developers to drink their own champagne (or “eating their own dog food,” if you prefer, but I doubt that you do.) By being immersed in their own data, locked into using only public datasets, they could see where the data lacked key fields, where poor assumptions had been made, where far too much was assumed of third-party developers. They were transported from the government side of the API to the public side of the API, a shift in perspective that is crucial to good API UX.

If you must run a hackathon for your government agency, take a cue from Virginia, and limit participants to employees of that government. And if you want to open it up to a broader audience, set internal expectations and goals based not on a hyped-up, .com-era concept of what success looks like, but instead based on the more modest goals of welcoming people to your agency, sharing knowledge, and building community.