Search

A Case for Open Data in Transit

Ever find yourself waiting for the next bus, not knowing when it will arrive? Think it would be great if you could check a subway countdown clock from the sidewalk? Or get arrival times on your phone? Giving transit riders better information can make riding the bus or the train more convenient and appealing. And transit agencies are finding that the easiest and least expensive way to do it is by opening data about routes, schedules, and real-time locations to software developers, instead of guarding it like a proprietary secret.

I recently got the chance to dive into the topic of open data in transit with my colleagues at OpenPlans. We went up to Boston to see what transit riders got out of the transportation department's decision to open up its data. We also talked to New York MTA Chair Jay Walder, City Council Member Gale Brewer, Zipcar co-founder Robin Chase, and Transportation Alternatives director Paul Steely White to paint a full picture of what it would mean if cities shared their transit and transportation data. The information is there, waiting to be put to use to help people plan transit trips, waste less gas driving, or make their streets safer.

<br>
<p><font size="3" face="Times New Roman">[intro music]</font> <br>
</p>
<p><font size="3" face="Times New Roman"><i>Nick Grossman:</i>
[00:03] Transportation’s a natural for open data efforts to take route
in cities because transportation touches everybody’s lives everyday.
It’s how we get around, it’s how we get from home to work.</font></p>
<p> <br></p>
<p><font size="3" face="Times New Roman"></font></p>
<p><font size="3" face="Times New Roman">[music]</font> <br></p>
<p><font size="3" face="Times New Roman"><i>Chris Dempsey:</i>
[00:18] So think about all the different ways that you can find out
whether it’s going to rain that day, or what the temperature’s going
to be the next day. You can get it on TV, on the radio, on the
internet, you can get it on your smartphone and you never have to pay
for it. And the reason that that’s true is because the National
Weather Service is open. Transit agencies have been very hesitant
to share that information with third parties. They’ve always
felt like they have to be the primary source of information for riders.
But if you take the model of the National Weather Service and you apply
it to transit agencies, you realise that you can have just as many options
to get transit information as you do to get weather information.
And the beauty of it is that it’s no cost for the transit agencies. </font></p>
<p> <br></p>
<p><font size="3" face="Times New Roman"></font></p>
<p><font size="3" face="Times New Roman">[music]</font> <br></p>
<p><font size="3" face="Times New Roman"><i>Chris Dempsey:</i>
[00:57] So we opened up our data and we’ve started sharing it with
developers, and within hours of sharing that data, developers had taken
it and built iPhone applications, websites, phone numbers, text messaging
services, that allow our riders to know where their bus is and when
it’s going to pick them up. </font></p>
<p> <br></p>
<p><font size="3" face="Times New Roman"></font></p>
<p><font size="3" face="Times New Roman">[music]</font> <br></p>
<p><font size="3" face="Times New Roman"><i>Nick Grossman:</i>
[01:20] All of the events that we’ve seen in transportation open data
recently plug into a larger conversation that’s happening in the country
about Government 2.0. One of the most vocal proponents is Tim
O’Reilly who talks about government as a platform. </font></p>
<p> <br></p>
<p><font size="3" face="Times New Roman"></font></p>
<p><font size="3" face="Times New Roman"><i>Tim O’Reilly:</i>
[01:32] Gov. 2 0 is the use of web 2 0 technologies, like cloud computing,
social media, open data, by government. Government should think
of itself as the platform that society builds on, rather than government
as a vending machine of actual service delivery. The idea of being
a platform provider is you do the least possible, not the most possible,
to enable others to build on what you do. In this transportation
space, it has to do with thinking about what steps can be taken that
will then let private individuals, private companies, actually deliver
additional services to the public. </font></p>
<p> <br></p>
<p><font size="3" face="Times New Roman"></font></p>
<p><font size="3" face="Times New Roman">[music]</font> <br></p>
<p><font size="3" face="Times New Roman"><i>Chris Dempsey:</i>
[02:19] We’re at J.P. Licks which is a great ice-cream store and coffee
shop in the heart of Jamaica Plain. Right out front the 39 bus
goes by, inbound and outbound. And what we have here today is
we have the LED sign that a third party developer installed at J.P.
Licks that tells you when the next bus is going to go by in each direction.
We now know that we have, you know, two minutes until the next bus is
going to come.</font></p>
<p> <br></p>
<p><font size="3" face="Times New Roman"></font></p>
<p><font size="3" face="Times New Roman"><i>Joshua Robin:</i> [02:39]
One of the most popular apps for bus riders here in Boston is an app
called Catch The Bus. You click on the app and it loads for you
a list of all the routes. We’re taking the 39 outbound, and
there it is, it tells you the next bus is going to come in eight minutes.
Buses are always more confusing than trains because there’s not a
track there, it’s not as obvious. When you see it on an app
like this and you can interact with it, it makes the bus something real
and something really easy to understand. </font></p>
<p> <br></p>
<p><font size="3" face="Times New Roman"></font></p>
<p><font size="3" face="Times New Roman"><i>Chris Dempsey:</i>
[03:02] Transit agencies traditionally think that it’s always their
responsibility to get this information out, and it can be a big hurdle
because of the organisational challenges and the financial challenges
that we face. It’s nice to have a team of literally hundreds
of developers out there who are thinking about these questions.</font></p>
<p> <br></p>
<p><font size="3" face="Times New Roman"></font></p>
<p><font size="3" face="Times New Roman">[music]</font> <br></p>
<p><font size="3" face="Times New Roman"><i>Nick Grossman:</i>
[03:18] Up until recently the MTA in New York was the largest transit
agency that did not publish open data. </font></p>
<p> <br></p>
<p><font size="3" face="Times New Roman"></font></p>
<p><font size="3" face="Times New Roman"><i>Nicholas Bergson-Shilcock:</i>
[03:24] Last year, the way you got data from the MTA was you either
went onto their website and through a process which is called Scraping,
tried to take it off and put it into a useful format, or you submitted
a foil request and you got a CD, and you don’t know if that data was
up-to-date, and the format of the data on that CD was hard to use and
undocumented. </font></p>
<p> <br></p>
<p><font size="3" face="Times New Roman"></font></p>
<p><font size="3" face="Times New Roman"><i>Nick Grossman:</i>
[03:44] And over the past few months that’s changed. In January
they launched a developer centre and an open data program and recently
they had their first developer’s conference. </font></p>
<p> <br></p>
<p><font size="3" face="Times New Roman"></font></p>
<p><font size="3" face="Times New Roman"><i>Jay Walder:</i> [03:54]
The Unconference is really about harnessing the power of applications
developers. The MTA has for too long really pushed the development
community away, and this conference today is really about embracing
the development community, about finding ways to be able to work with
them, and about allowing them to bring the type of creativity and innovation
that they’ve brought into so many other areas into our transit system
as well. </font></p>
<p> <br></p>
<p><font size="3" face="Times New Roman"></font></p>
<p><font size="3" face="Times New Roman"><i>Nicholas Bergson-Shilcock:</i>
[04:14] It’s a night and day difference from last year when developers
were being sued for using this data, to this year where the MTA is encouraging
developers to use this data.</font></p>
<p> <br></p>
<p><font size="3" face="Times New Roman"></font></p>
<p><font size="3" face="Times New Roman"><i>Nick Grossman:</i>
[04:24] Public transit is just a piece of the urban transportation network.
There’s incredibly interesting data being produced by all sorts of
other modes, including private cars. </font></p>
<p> <br></p>
<p><font size="3" face="Times New Roman"></font></p>
<p><font size="3" face="Times New Roman"><i>Robin Chase:</i> [04:31]
In cars there’s a whole bunch of data that’s being generated that
is now going into the wind. It’s uncollected, and today we could
be providing that information. Imagine how much innovation we
would get if there were an open black box in cars. So car sharing
would be one application. Better routing would be another.
Tell me how good a driver I am, what my fuel efficiency is would be
another one. And there’s a whole number of applications that
I can think of some, but more to the point, there are thousands of people
out there who could think up tens of thousands more.</font></p>
<p> <br></p>
<p><font size="3" face="Times New Roman"></font></p>
<p><font size="3" face="Times New Roman"><i>Nick Grossman:</i>
[05:03] Open data can even be used to help us make our streets safer. </font></p>
<p> <br></p>
<p><font size="3" face="Times New Roman"></font></p>
<p><font size="3" face="Times New Roman"><i>Paul Steely White: </i>
[05:07] It is imperative that moving forward we have more information
about the extent of the traffic safety problem in New York City.
Right now the public thinks that injuries and fatalities on New York
City streets are isolated accidents. With more open and timely
information about these crashes, it will be much easier to show everyone
that these are preventable tragedies that are occurring, and that there
are clusters of these tragedies that are occurring precisely on streets
that need better enforcement and better street design. </font></p>
<p> <br></p>
<p><font size="3" face="Times New Roman"></font></p>
<p><font size="3" face="Times New Roman"><i>Gale A. Brewer:</i>
[05:35] The fact of the matter is when the data is public, really smart
people who want to take it and make an application can do so.
And then I think by definition force government to be more efficient. </font></p>
<p> <br></p>
<p><font size="3" face="Times New Roman"></font></p>
<p><font size="3" face="Times New Roman"><i>Tim O’Reilly:</i>
[05:49] And when you think of government as a platform rather than as
a service provider, then what you are doing is you’re creating the
capabilities for the people to say “we did it ourselves”. </font> <br>
</p>
<p><font size="3" face="Times New Roman">[music]</font></p>
Transcription Sponsored by: <a href="http://transcriptdivas.com/">Transcript Divas Transcription Services </a>

some history here. Over a decade ago, Rescue Muni began asking Muni/SFMTA to put the Daily Service stats http://www.sfmta.com/cms/rdlymuni/dlymuniindx.php on their website. They stonewalled for several years, prompting us to ask for the reports so Rescue Muni could post them. That happened with some outages after Nat Ford succeeded Michael Burns as Exec Dir. Finally a year or so back they began posting, but several months ago, the most important data was scrubbed from the reports. We used to be able to pinpoint both which routes were being denied service and which barns(operating divisions) had the worst absenteeism. The next step in my view would be for public pressure on the agencies to clean up the "not outs" and make better efforts on maintenance so that no routes are unserved for lack of roadworthy equipment. As in all other aspects of government, total transparency is the goal.

JK

This is really well done, and this area of civic life is about to explode. One thing to consider. The example of the weather data and public transit data are apples and oranges --- which I think shows the limits in O'Reilly's "government as platform" philosophy. The weather service controls weather data, not the weather. The transit agencies control transit data and service. Likewise traffic safety. There is no app that will make streets safer --- organized people will. The traffic safety data is an accountability tool that can be used to politically mobilize communities to improve public service, in this case police enforcement and DOT street design.

http://developer.wmata.com/ Alex Gaber

Washington DC Metro (WMATA) is launching real-time APIs August 11th!

momos

Excellent piece, Elizabeth. This really shows the potential and importance of sharing public data.

http://www.sandora.mobi Ramesha

The need for improvement in public transport system is long over due. By just making available the requisite data to better inform the commuters enables them to better plan their journey. This is a case of opportunity to improve productivity by just better use of already available data.

Read some where that 7 % of our time we waste as waiting time. Image the savings if we can reduce this.

sh

How does this benefit people who cannot afford or do not want to pay for mobile devices? Quite often, people who ride buses do not have a great deal of disposable income. What good does an "app" do them? Or don't they matter?

As for the electronic signs that PDX Trimet provides at some stops? They're not accurate. Unless they've been improved within the last year. Last time I called to find out when a bus was going to arrive, the information I received was definitely incorrect. Repeatedly--so however Trimet had its information system set up, it wasn't working well--although Trimet at the time claimed that it was (there were multiple complaints about whatever system they were using--same as mine--inaccurate information provided).

http://www.oreilly.com Laurel Ruma (@laurelatoreilly)

Thank you for producing this video! It is a fantastic and comprehensive view of the importance of open transit data.

As more data comes out, we'll find more and more experiments of bridging the digital divide. In Boston, one developer created an IVR system, which is just a way to call an automated number to find out when the bus is coming. We're in the early stages of this open data revolution, but the benefits are obvious.

http://true-cost.re-configure.org JZ

http://smart-city.re-configure.org from 2007 which was published in a book called "Collective Intelligence" in 2008 touches upon this and it's good to see these developments accelerating.

An important issue not mentioned is "open process" in regards to the *methods* of collecting data. Think open sources and open methods.

As far as any weather service controlling data, using them as an analogy is interesting when you think that they do not have a good reason or incentive to manipulate data they are going to present to the public when telling people a storm is coming. Knowledge about the weather is useful for everyone's plans. The more examples we find where an open process is good for everyone, the easier it should become to get more people to pressure whomever to open the gates and prevent future gates from starting out closed.

As far as Tim O'Reilly's concept of government, I do not like the idea that the analogy illustrates government as a *foundation* that all else is built on. Later on O'Reilly says of gov doing the least and that enabling citizens to develop, etc. I prefer a clear mention that citizens be developers of the platform. Government workers are citizens are they not? They know people who know people and it's all connected, not a separation of person and platforms/system. If we allow too much drift between person and state, we get closed data, closed process, less accountability, more corruption, more waste, less quality of life, etc.

http://www.stevevance.net/planning Steven Vance

To sh:

There are many other applications for open transit data, at least two of which were featured in this video.

1. A call in service that gave real-time information by calling a phone number.
2. An LED message board inside a bakery (J.P. Licks) in Boston gave predicted arrival times for bus route 39.

Neither of these require a mobile device. There are also applications for desktops, too, like widgets that display predicted arrival times.

JD

Open Plans / StreetBlog should disclose their contract with the MTA in all their coverage on the tpoic. Is this super-supportive piece produced as a part of the contract?

http://www.livablestreets.com/people/nickyg Nick Grossman

JD: thanks. We do, in fact have a pending project with MTA, which is not connected to this video. As a nonprofit organization, we do mission-driven work (such as Streetfilms and nytransitdata.org, which lead the campaign to convince MTA to open their data) in support of our goal of more open, interoperable, livable cities. We also do commercial software development, often with government agencies as clients, to supplement the charitable support we receive for our nonprofit work. Thank you for prompting this clarification.

http://www.livablestreets.com/people/stevevance Steven Vance

To JD, Nick Grossman: The Active Transportation Alliance in Chicago, Illinois, is a non-profit organization with both advocacy (mission-driven work, like OpenPlans) and consulting arms (mostly bike/ped plan development for Cities and Counties).

Lisa Sladkus

My favorite part was the cafe with the bus times displayed. I love having my coffee and knowing when I need to bolt!

http://true-cost.re-configure.org JZ

I was told about the NEXT Bus System in Baltimore and this led me to find this large list

This well-prepared and densely informative video really led me to quickly grasp the importance and functionality of data. Kudos on this. I'd like to see much more like this that go into depth on critical subjects.

Tim Bischoff

I think it would be good to know more about transit schedules, I work in NYC but would be concerned that not all people would use the info for good. NYC receives soo many bomb threats, and they are real threats, Times Square a few months ago. If it is shared it should be by subscription and those who subscribe should have their info cross referenced by homeland security. Then it's a good idea.

http://blog.russnelson.com/ Russ Nelson

Would it surprise any of you to find out that most forms of vehicle tracking are protected by software patents? For example, the ineffably obvious one of attaching a GPS to a radio, and "phoning home" with the location ... patented. Or so I am assured by deeplocal.com, which is heavily invested in this space.

It's clear to me that the first thing we need to do is get the government out of our way, before we can use it as a platform. You know Mordac, Dilbert's character who is the "Preventer of IT Services"? We have government, preventer of citizen services.

I don't want a free market. I want instead to replace government regulation with citizen regulation. Rather than keep bad products off the market, I want citizens to not buy things that don't work. Rather than create idea monopolies, I want businesses to have to provide service with their ideas. If they're good ideas, they'll get rewarded by customers. If it's a good idea, but they can't execute, why should they be able to block others' execution with a patent?

David Turner

Even before the data was public, it was not a secret where the trains and buses ran. It's also not a secret where large groups of people congregate. Even if it were the case that MTA vehicles always ran on time, someone bent on wrongdoing would not benefit at all from this data, because it doesn't reveal anything otherwise unknown.

It's worth considering security before releasing public data -- but a security threat has to be real; merely saying the word "bomb" does not justify keeping anything secret.

(full disclosure: I wrote some of the software seen in this film)

http://www.clevercommute.com Josh Crandall

Great video. I actually spoke at the Google/MTA unconf in NYC on 5/5/2010...and I was blown away by the "bear hug" that Jay Walder (and MTA) gave to the developer community. My system (clever commute) does NOT use much MTA data...but nonetheless it was great to see how open they are to democratizing data

http://www.livablestreets.com/people/Jessica Jessica

Go JP licks!

Great video. how do we make this happen in the Los Angeles region? with over a million daily bus riders is would be a great asset to say the least.

http://nyc.gov Andrew Nicklin

Out of curiosity has anyone started looking at Transcom's data feeds (in beta testing), which is supposed to include realtime MTA data amongst others from the NYC Metro area?

Transcom could/should do two things differently: 1) allow for unregistered users to access downloadable data. The prevailing wisdom is that registration should only be required when using APIs that could be processer intensive and might need throttling. And 2) they should open up a public discussion list for communication with data users -- the transit agencies that are doing this well (MTA and MBTA are two good examples) use an open discussion group to create a relationship with developers and also build a public archive of questions & answers.

A while back, we wrote up a list of recommendations for agencies who are starting up data programs and want to build developer communities: http://openplans.org/civichacker/2010/02/12/bootstrapping-a-developer-outreach-program/

Quikboy

Open Data in Transit is nice, but one has to wonder the security issues with releasing on-the-spot data about approaching trains or buses. Not that buses/trains don't already have set schedules, but it's not like you want to make it any easier for the bad guys.

I think a Bing Maps API would be better, because not only is it cheaper, but also because Bing has better urban maps.

http://www.livablestreets.com/people/stevevance Steven Vance

To all the people alluding to the possibility of terrorism in relation to "up to the minute" predictions on bus and train arrivals:

1. Has any transit terrorism occurred (or failed to occur) because of the presence (or lack) of a sign that said the next train arrived in 2 minutes?
2. Should FlightStats.com stop publishing departure and arrival times for 1,000 domestic and international flights a day? And stop publishing alerts about delays?
3. Will "the bad guys" be more inclined to attack, cause violence, or "be terroristic" if they know the Route 8 bus arrives in 4 minutes versus "every 3-10 minutes"? If so, how?

Let's take this a step further:
4. Let's order Google Maps and NAVTEQ to stop providing live traffic information.
5. That's not enough. Let's stop posting information in the newspaper about upcoming construction projects and detours in your town. Not knowing about a detour might throw off the bad guys' plans. Woo!

Give me a break.

anon

Does not work for the poor, because you need a computer and phone to use the apps. The government should put information displays in and around transit points for everyone. Boston and New York fall way behind in this aspect.

@anon - it's true that a device of some kind is needed to make use of the data. But as they've shown in Boston and elsewhere, waiting for agencies to build 100% of the hard infrastructure just locks up the info and makes it so no one can use it. Better to have step 1 being open the data, which is incredibly cheap relative to building signs, and then have step 2 be to start building hard infrastructure as needed.

This way, the info is out there as early as possible with the lowest possible government cost.

Then, enterprising citizens can build displays, both in the form of websites and mobile apps, as well as public displays like the one shown at JP licks in the video. Chicago even provides a template for DIY info displays: http://www.transitchicago.com/developers/diydisplay.aspx

Jym

@anon - In Boston, at least, there's a solution for people who have phones with texting capability: send "mbta" to the number 41411. Info for other cities here:

It really is a no-brainer to open up the bus information. What really needs to happen is a more flexible busing system that will reroute according to demand. I regret to say that most jobs in Cleveland are not directly downtown. It would be better to reroute buses to directly respond to traffic patterns to get the most people where they need to go.

Katie

Great piece! I recently visited Seattle and discovered, pretty much accidentally, that I was able to get complete directions, including times, for public transit by using google maps on a mobile device. Had I not been able to use the bus system so easily, I probably would not have gotten to as many places around the city as I did. I suspect it was tapping into schedules, and wasn't real-time, but the buses seemed to all be running on schedule.

http://www.haramtube.tk porno

Links between tech, transparency and community-based democracy in India.

Pragmatist999

It is very disappointing that transit officials in my hometown Phoenix Arizona has decided to retreat to the caves for fear of "terrorists" who might use real-time transit information to blow up people.

Pragmatist999

It is very disappointing that transit officials in my hometown Phoenix Arizona has decided to retreat to the caves for fear of "terrorists" who might use real-time transit information to blow up people.

Pragmatist999

It is very disappointing that transit officials in my hometown Phoenix Arizona has decided to retreat to the caves for fear of "terrorists" who might use real-time transit information to blow up people.

Chris

Unfortunately you are referring to a very conservative state! It is common for conservatives, as we know the term today, to be fearful.

Chris

San Francisco is improving on that score..about 60% of their bus and train stops have real-time displays of arrival times connected to bus gps systems. They still have a ways to go in improving schedule reliability however.

http://www.outsourcingtypingservices.com/ Snehal Joshi

Open data in transit is awesome idea, but one has to wonder
the security issues regarding on-the-spot data about trains or buses. Not that
buses/trains don’t already have set plans, but it is not like you want to make
it any easier for the bad people.

Joseph

Assist NY offers various types of transcription services in New York that are needed in various sectors. We offer Audio Transcription, Legal Transcription, Interview Transcription, and Video Transcription in the New York State area.

Click to Watch Some Great Collections

Copyright Notice

No Streetfilms may be re-purposed or used in other works without prior permission. However, you are free to embed, share, make copies and distribute full length Streetfilms without contacting us. This includes broadcasting to the public or at in presentations without any alterations.

You may download FREE medium-resolution copies directly off of Vimeo. Just click thru to Vimeo and you will find a DOWNLOAD button. These files are high-quality enough to use in group settings, classrooms or for presentations.