I recently wrote up a ‘body-of-work’ awards application, for which the date restrictions coincided with our team’s first year on the job. It seemed like it would make a good blog post, except for the self-aggrandizing tone. Unfortunately, I’m kinda short on time, so I’m posting it unedited. Please forgive me. A boast post is better than no post at all, right? — Brian

The news applications team has had one hell of a year. Founded last June, we’ve created new ways to tell watchdog stories online, and along the way, dug up and munged, massaged and mapped a mountain of data. And at every step, we’ve given away the data and documented our processes here so that other teams may follow in our footsteps.

On July 29, 2009, the Cook County Sheriff’s office released a database of headstones — they were documenting the graves left undisturbed by a group of crooks worthy of a Coen brothers biopic. To help family members learn if their loved ones remains had not been exhumed and their plots resold, we built this simple searchable database in a day’s work.

To accompany a piece written about excessive and sometimes silly aldermanic spending, we built the city council expenses application. The original story was good, if predictable: Your alderman spends city money on a fancy car. But, since we published all the data online, many of our readers sent in tips of unusual-looking expenses from their ward — and one hit paydirt. It turns out that one alderman was renting office space from himself. The day one story was about unethical behavior, the day two was about potentially illegal acts. A big win for giving away the data.

Nursing facilities in Illinois aren’t just home to the infirm and elderly. They also house angry gang members, convicted sex offenders and volatile psychiatric patients, creating a hidden world of fear and violence. As an accompaniment to more than a dozen hard-hitting investigative reports, our unprecedented Web application answered readers’ questions — “Is my father in danger?” “How can I find a safe home for my mother?” — in the most simple fashion possible: a searchable collection of safety reports.

The response -— from government authorities, industry leaders and citizens —- was swift and powerful. On the third day of the series, Illinois Gov. Pat Quinn appointed a Nursing Home Safety Task Force, and within weeks, sweeping reforms were proposed. And now, a nursing home safety bill is now on the governor’s desk. It will, among many other reforms, require the creation of similar web tools to what we demonstrated with our application. Better software is about to become Illinois law.

Tasked with better informing Illinois voters (they’ve made some questionable choices lately), we built the Election Center: a one-stop-shopping site for Chicagoland and State-wide elections news, polls, Q & A, endorsements and election-day resources. Among the highlights was an online ballot-builder that would guide you from discovering your candidates based on your address, through making a list of your picks to share via Email, Facebook or Twitter. The site changes with the conditions of the day and it’s currently not in full-bloom. Check our our archived primaries site for a glimpse into the past.

Fifty Chicagoans were murdered in June 2010, and a record of each homicide: the name, cause, location, and a story about what happened, can be found on RedEye’s homicide application. (The RedEye is a sister publication to the Tribune.) So far, we’ve collected the data back to 2008, and made all of it available for download — which we have recently learned has been of great service to local anti-killing organizations.

A reader wrote a letter to our editorial board, asking for an easy way to express concern to their elected officials. So, we built one. Take Action! is a context-sensitive elected official lookup tool — when you’re reading an editorial about a city issue, it will direct you to how to connect with the mayor and your alderman. So simple, it’s not really even an app, but a high-powered widget.

We, as a matter of principle, like to show our work. On the news apps team blog, we’ve written tutorials on how to use our data, released our production-grade server stack for free for the world to use, and shared the best practices we’ve discovered. Why? Because we feel lucky to be doing what we love, and want to help other newsrooms who may not be lucky enough to have a team of hackers at their disposal.

Special thanks to Bill for bringing us on, and to Chris, Joe and Ryan, who’ve slayed many dragons on our quest. Kick-ass work, gang. Next year is gonna be even better.

We’ve been thinking about accountability a lot lately. Here’s a quick run down of applications that we find inspiring. okay.

UPDATE: This post was originally titled “Accountability applications we love”, but in retrospect, data without context makes our eyes go cross. There are sparks of storytelling in each, but that’s all they may ever accomplish. (Politifact and Represent are the exceptions, of course, they’re context-rich.) LittleSis is a fun site to browse, but at best it’s a research tool. A place where you’d start, not where you’d finish. I *love* Follow the Money’s scatterplot, but we can aim higher. Our role as a news organization is to seek out and report stories — to interpret the data and infer meaning — not just shine a light. More to come. — Brian

Merging official government data with news and blog coverage, social networking, and public participation tools to give you the real story behind what’s happening in the Congress.

Ryan: The thing I love most about this site is how it fosters community. Users can follow pols and bills, contact their representative, share via other social networks and they can voice their support for legislation. This is how democracy should work in the Internet age, IMHO.

Chris: The ability to track particular bills that you find meaningful takes the overhead out of being engaged at the national level.

Brian: I like that you’re encouraged to send letters to their congressman, and that people post them in the comment feeds.

A free database detailing the connections between powerful people and organizations.
…
Our data derives from government filings, news articles, and other reputable sources. Some data sets are updated automatically; the rest is filled in by our user community.

Ryan: It’s well designed, easy to navigate, and it is wikipedia-like attention hole. You can just keep clicking and clicking on interesting stuff. I like the way it focuses on the relationships.

Brian: LittleSis attempts to connect a *lot* of dots. It calls itself an “involuntary facebook” — showing you more than just names, but profiles and connections of players in the game.

Ryan: It has a great database of money in politics: companies, pols and individuals, all the way down to specific contributions. The database also has a few years of historical data. But it really shines in it’s visualizations.

Brian: More tools for browsing the influence of money, including compelling and statistically interesting (and reader-respectful!) visualization tools, like this one…

Every day, reporters and researchers from the Times examine statements by members of Congress, the president, cabinet secretaries, lobbyists, people who testify before Congress and anyone else who speaks up in Washington. We research their statements and then rate the accuracy on our Truth-O-Meter – True, Mostly True, Half True, Barely True and False. The most ridiculous falsehoods get our lowest rating, Pants on Fire.

Joe: Prototypical example of incremental journalism only possible on the web. Small reports gather weight over time and provide context.

Ryan: The wonderful example of how reporting can be presented online outside of a traditional story. We can augment simple databases of numbers and facts with reporting and add important context.

Chris: Leverages the power of aggregation to provide an insight into political truths and half-truths that would be almost impossible without the web.

Brian: I love their use of ‘link journalism’ — write what’s news, and link to the background, like in their recent piece on health care facts.

The backgrounds and records of thousands of political candidates and elected officials… their voting records, campaign contributions, public statements, biographical data (including their work history) and evaluations of them generated by over 100 competing special interest groups. Every election… volunteers test each candidate’s willingness to provide citizens with their positions on the issues they will most likely face if elected through the Political Courage Test.

Brian: Like many of the sites above, it’s a bit ugly and hard to use, but I especially like Vote Smart’s aggregation of issue positions and ratings by interest groups.

We’re seeking an intern to spend the summer hacking in sunny Chicago, Illinois. As a member of the news applications team, you’ll work with the reporters and editors of the Chicago Tribune to gather and visualize data and build web applications for the lovely people of Chicago.

It’ll be fun. Also, educational.

Compensation (the rub)

The Tribune’s 12-week internship program is part time, 20-24 hours a week, and we pay $7.75 hourly. That ain’t much, so this gig is likely best suited to Chicago-based students looking for something more fun than classes.

(If you’re not already local, we can try to give a hand finding a roomie, but you might be better off arranging a sofa on which to crash. Perhaps this will present you with the opportunity to pursue that freelance project. Or maybe learn to sail. There’s a lot of fun to be had in Chicago in the summertime.)

Applying

To apply, email newsapps@tribune.com, and tell us why you’d like to waste a perfectly good summer working at a newspaper. Include your preferred start date and some type of resume-like rundown of your abilities and experience. Bonus points will be given to applicants with neat skills and/or a project or two in mind.

So, I just rebuilt my machine from scratch and installed Snow Leopard. Miraculously, this is all I needed to do to get my development environment (read: full GIS-enabled Python stack) up and running again.

At this point, I had to edit the brew’s postgis formula to drop reference to a non-existant file. This may be fixed in the future, but for now, edit line 17 of /usr/local/Library/Formula/postgis.rb and remove postgis/postgis_upgrade.sql. After you’ve done this, proceed to install the GIS stack (and git, cuz you need git)…

The postgis brew formula will install PostgreSQL, proj, geos, and more. During that process, it’ll present you with a nice little explainer about how to start your PostgreSQL server. Follow those directions if you like. Mine automatically starts on login.

For extra fun, install QGIS using KyngChaos’ *standalone* installer. (I tried to build QGIS based on my own and failed. It’s not worth the trouble. Kyng’s standalone version is tidy and completely self-contained.)

Lots of folks have asked us about the tools we use to build and deploy applications at the Trib. Well, now that we’ve got little time to breathe between the primaries and the build up to the general election in November, the team has been working on ways to share what we’ve learned.

So far, we’ve written about a handful of solutions to specific programming and deployment problems:

Our production environment runs on two Amazon EC2 small instances running Ubuntu Linux. One’s our web server and runs Apache + mod_wsgi, Memcached and Django. The other handles data and runs PostgreSQL, with pgpool (indispensable) to manage connections. We can spin up additional, near-identical (minus Memcached) web servers at-will when the load gets too great, but so far, it’s not been necessary.

Our staging environment is *exactly* the same. Running a mirror-image staging environment enables us to do things like test heavy load in a controlled situation. The idea is that we never want to be surprised when it’s time to roll to production.

And that’s pretty much it. (Forgive me the lack of a diagram. I always get hung up choosing the perfectly puffy cloud clipart, so I decided to skip it altogether.)

At $0.085/hour for a small instance (~$14/week), plus nominal bandwidth and S3 costs, we burn around $400/month in hosting expenses. If you’re not running an extremely demanding site like the Election Center, you could easily cut that in half by merging your web and data servers. (And if you’re really feeling cheap, you could shut down your staging server when not in use… but please, don’t forgo it altogether. You need a staging environment. Trust me.)

That’s it for now, but please, stay tuned! There’s some great stuff coming soon:

Using homebrew to set up your GIS stack on OSX.

Best practices in web development with Python and Django

And our server stack, complete with disk images and a recipe to get started!

Our team is in need of someone who will lead the design conversation. Someone who will interview stakeholders, develop personas, intuit features, arrange information, draw mockups, and everything else necessary to design a web site. You will work fast and agile, in tight iterations, and in close contact with our stakeholders — the editors and reporters of the Chicago Tribune.

You should also be ready to close the loop and put our work in front of users, take their feedback, and redesign it all — cuz that’s what you gotta do when you’re agile.

You must care deeply about usability and grok the web.

Extra points if you love to sketch, didn’t have to google ‘grok’, and don’t need an education on agile development practices.

Position two: Web designer / developer

We are also in need of a creative web designer. Someone who cuts tight, valid and semantic HTML/CSS and makes it look *hot*. Graphic design skills are a must, but we also require the ability to implement those designs. We need more than a photoshop jock. You will work fast and agile, in tight iterations, and in close contact with our stakeholders — the editors and reporters of the Chicago Tribune.

(If you’re a print designer, you’re probably not who we’re looking for, but we’ll do our best to not be prejudiced. Show us you’ve got serious web chops, and we’ll talk.)

Extra points if you have worked with Django (we’ll welcome Rails skillz too, they translate) and have built many beautiful websites.

Even more points (for both positions) if you know a thing or two about:

Data science (statistics, exploratory data analysis, R)

Information design (beautiful charts, graphs and other Tufte-geekery)

Building and gardening social media or crowdsourcing applications

Some days we’ll huddle and sketch with reporters, imagining ways to present information and tell their story on the web — and we might turn that story around in a day, a week or a month. Other days, we’ll develop news products that’ll take months to realize.

Either way, we work fast and lean, relying heavily on frameworks, and following agile best practices. It’s fun.

…all at a desk somewhere in the Tribune newsroom, where you’ll be surrounded by reporters arguing with the cops, yelling about the ball game, telling crazy stories, and otherwise practicing their trade.

There is no free pop, pinball or posh cafeteria.

But, you’ll like what you do. You’ll come to work energized, and leave satisfied that you’ve done something that will make your mom proud. You’ll have held our government accountable, spoken truth to power, given voice to the voiceless, and contributed to the public good.