Posted
by
timothyon Thursday March 08, 2012 @11:08AM
from the wonder-about-the-gracenote-problem dept.

beelsebob writes "In the recent release of iPhoto for iOS it appears that Apple has started using OpenStreetMap's data. Unfortunately, there are still some problems. Apple is currently not applying the necessary attribution to OSM; they are using an old (from April 2010) dump of the data; and they are not using the data in the U.S. Fingers crossed that Apple works through these issues quickly! Apple is now one of a growing list (including geocaching, and foursquare) to Switch2OSM."

"Appears" eh? I'll wait till we know for certain what maps they are using now before I worry about attribution. Apple have bought some mapping companies in recent years, on the face of it it seems more likely these maps are Apple's own.

Certainly in the comparative viewers that OSM are linking to, it doesn't look to me any more like OSM than it looks like Google Maps. All three look different in presentation, but similar in content.. given that they are all modelling the same reality.

Take a look at the comparison linked in the article. It's very clear that the data is identical to that found on OpenStreetMap in a lot of areas. The data is literally identical to the april 2010 planet dump.

My guess for what has happened here is that Apple thought that they bought the data when they bought some mapping company; but that it turns out the company they bought just ripped the data out of OSM.

The map tiles are certainly Apple's own - they have defined their own stylesheet, with their own look.

However the map data those tiles were rendered from appears to be a mix of TIGER in the US and OSM elsewhere. TIGER is a public domain dataset from the US Census Bureau, and OSM is CC-BY-SA.

Looking at the shape of the data is often enough to tell you where it came from. One one level it's modelling the same reality, but in practice mappers tend to make slightly different versions of "the same" object (a road might be smoothly curved, or quite angular, depending on how much effort they went to). As such you can quite easily see when data comes from the same source, even if it's rendered in a different style.

It's pretty conclusively OSM if you look at which small features [wordpress.com] (footpaths, lanes within a car park, etc) are rendered. This data isn't present in the commercial datasets you can licence from people like TomTom, however it is in OSM (neither Navteq nor TeleAtlas have footpaths, or this kind of micro-mapping of lanes within parking areas).

Based on things like this, typos which appear on both maps, and roads that are in OSM now but aren't in Apple's tiles - it looks pretty clear that they used a snapshot of OSM, specifically one from early April 2010.

"It's pretty conclusively OSM if you look at which small features (footpaths, lanes within a car park, etc) are rendered. This data isn't present in the commercial datasets you can licence from people like TomTom, however it is in OSM (neither Navteq nor TeleAtlas have footpaths, or this kind of micro-mapping of lanes within parking areas)."

However, that level of data is evident in some cases from commercial datasets. For example, the street directory in my car (which pre-dates OSM - I should really buy a n

The OSM data that Apple is using is rather old (start of April 2010)... It’s also missing the necessary credit to OpenStreetMap’s contributors; we look forward to working with Apple to get that on there.

One burning question: will we be able to get audible turn-by-turn directions if Apple moves to OSM?

You can bet that if they do offer turn-by-turn directions then it'll only be available in the iPhone 5 when it gets released. Just to force everyone to upgrade again - the same way that Siri is technically capable of working on the iPhone 3/4 but is only available on the 4s.

this is supposedly needed to make the Siri UX good enough for Apple's standards.

As long as Apple have an excuse they'll use it to try and persuade people to upgrade. Siri would have worked on the iPhone 4. Yes, it would have worked better on the 4S but I'd be astonished if the reason it was not on the 3G/3GS/4S was technical and not marketing.

this is supposedly needed to make the Siri UX good enough for Apple's standards.

As long as Apple have an excuse they'll use it to try and persuade people to upgrade. Siri would have worked on the iPhone 4. Yes, it would have worked better on the 4S but I'd be astonished if the reason it was not on the 3G/3GS/4S was technical and not marketing.

Really? It would honestly astonish you that Siri would require any development, testing, QA, integration, sales, administrative, or other costs? It would be completely free? Or do you not consider paying for engineering talent a "technical" cost? Because otherwise, it makes a lot of sense for Apple to invest money on their profitable products instead of their old ones. Apple already does so much better than Android, Windows Mobile, and others at supporting old hardware with the latest releases that I see little room for complaint. The iPhone 3GS is many years old and yet got iOS 5.1 the day it was released!

You seem to have proved his point for him there - that the newer and better features are only on the newer models because it's a business decision (and quite justifiable to boot) to not spend the money supporting the older models rather than a technical limitation of the old hardware.

Really? It would honestly astonish you that Siri would require any development, testing, QA, integration, sales, administrative, or other costs? It would be completely free? Or do you not consider paying for engineering talent a "technical" cost?

You're absolutely right. It's a shame no-one at Apple has been able to come up with a way of letting users of their mobile operating system purchase individual programs to provide new functionality. Someone should come up with some sort of online shop where these pr

I won't speak of the "audible" part, since that's just a small matter of programming.

Turn by turn is... complicated. Of course, you can upload OSM maps on your Garmin right now and get turn by turn instructions. However, accuracy is a factor.

The amount of information needed to drive through a city is absolutely astounding, as is the frequency with which it changes. For example, a street near my house is closed mornings and evenings to vehicle traffic, except weekends and holidays and June through August. That data has to be in there to accurately route. "No left turn, 4-6 PM Monday through Friday." "No northbound traffic except bicycles." "Carpools only 7 AM to 10 AM"--God help us.

Not to mention just plain errors in the data. Near my house, an overpass was accidentally connected to the freeway. My Garmin with OSM data wanted to route me off the freeway directly onto the overpass. (I fixed the error.)

Realignments don't happen that often in cities any more in the US, but they happen on country roads and interstates *all the time*. I didn't realize until I started contributing to OSM exactly how much construction was always happening.

Highway 36 west of Red Bluff, CA, was recently realigned. Google even has it wrong for now: http://g.co/maps/mhdkm [g.co] . And check this out: Google wants me to drive on a hiking trail: http://g.co/maps/jpxr8 [g.co] I'm not saying they suck--Google's map quality is *exceptional*, and yet it errs. But I'd say that for turn-by-turn, it has OSM currently beat.

OpenStreetMaps has generally good map data at this point, but their reverse geocoding (i.e. place data) is still very sparse compared to Google or Factual, etc. Would love to see a free, open database of comparable quality to the paid ones.

Not even remotely close. They seem to do quite well with things like schools, churches, public buildings, but not at all so well with restaurants, shops, and commercial places. In the immediate vicinity of my office just adjacent to New York City, geonames has perhaps 1/10th of the number of places that Factual has.

Whilst OSM is very good for free data, there are still pockets of areas where the coverage is very poor indeed. I had to map out half of my uncle's town as it just wasn't there on OSM (about 9 months ago). At every stage, it's getting better, but the more 'big players' that start to switch to it, the more momentum it will get and the better the coverage will be as more contributors flow in.

This is especially the case as parts of the OSM dataset are about to be wiped out due to the forthcoming remapping [openstreetmap.org].

OSM doesn't exclude elevation data. You can tag any node you like as "ele="... But, 1) most nodes in OSM are not created directly from GPS traces, but instead by inputting a cleaned up version, because the GPS data is a bit noisy 2) GPS is very bad at gathering height data –it's roughly 20 times less accurate for height data than for horizontal location.

Elevation data is best modeled as raster data, and doesn't really fit well with the OSM vector datamodel. That doesn't mean that you can't combine OSM-data with elevation data from different sources, and a lot of projects really do that.

In practice, OpenStreetMap is more up to date than Google for areas where locals know how to update it. That includes quite a lot of Europe (particularly Germany and England) and most metropolitan areas in the US. As more people learn about OpenStreetMap and begin using products that include OpenStreetMap data, that pool of up-to-date areas will grow. Basically, right now, there are areas where Google is better and areas where OpenStreetMap is better. (But where OSM is good, it's generally *very* good.)

Google is up-to-date, but in some cases it's completely wrong. They've recently changed from buying in map data from one source to amalgamating it from many many sources. This provides a headache for google as they can't manually fix things that are wrong as the fixes will be overwritten by the automated amalgamation in a week's time or so.

Take for example Normansland [g.co]. There is no place in the New Forest called Normansland. There is one up the road called Nomansland (without the 'r') but for some reason Goo

How is OpenStreetMap determining that Apple's using their data versus a similar data set from a different source? I haven't seen anything about their methodology for coming to the conclusion that it's OpenStreetMap data. How easy is it to pin down map data to a specific provider?

There's a lot of data that's only in OpenStreetMap, as compared to the other big map data providers like Navteq. In addition to roads, OpenStreetMap has bicycle paths, pedestrian paths, hiking trails, and a host of other things that are not generally collected in other general-purpose road databases. At least one person on the OSM mailing lists has pointed to an area where he added some but not all of the hiking trails in an area and Apple is showing only the trails he added to OpenStreetMap. Even more conclusive, though, is that when you overlay the two on each other, such as at http://ivan.sanchezortega.es/leaflet-apple.php [sanchezortega.es] , there are quite a lot of places where the data matches exactly--not just "both have a road here", but "every point making up Apple's road lies exactly on top of a point making up OpenStreetMap's road".

Every time I see a map rendered with different colors and a different style as Google Maps I immediately feel how much I prefer the Google Maps style.
Is it only me or is the rendering really that refined that it's just so much easier to spot things and therefore feels better?

Apple contribute code back. A fair bit of code has been accepted into FreeBSD from Apple. Do they contribute back as much as they should, ethically? That's debatable, personally I'd say no. Recently they're reducing their open-source efforts too (CUPS).

Why can't people just discuss things normally rather than having all this pro-Apple / anti-Apple garbage?

WebKit, LLVM/clang, and yes, they even did a lot for CUPS, and zillions of bugfixes across many products. And if you're an OS wonk, you can even look at the entire MacOS X kernel source code and borrow if you'd like, as well as many of the low-level processes that make MacOS interesting. It's true that many of these were not taken up by other products, but that's hardly Apple's fault.

I don't understand why this nonsense gets modded up. Apple is under no ethical obligation to contribute anything to open source. They are under a legal obligation to contribute to open source software in specific circumstances, such as in the case of distributing of modified GPL software. Its complete bullshit to expect any company or person to go above and beyond the legal obligations of using open source software, even if that software is helping them bring in heaps of cash.

Yes, you make it quite clear that you don't understand the concept of ethics or morals.

Apple contributed their changes to gcc to support obj-c and a modern runtime back (along with a bunch to improve c/c++ support). The gcc team refused a good number of them. The gcc team then moved to a more restrictive license with the explicit goal of cock blocking companies like apple (it actually explicitly banned the kind of integration into IDE that apple need to do).

Apple then set up a new C, Objective-C and C++ compiler front end (clang). Apple then released

Strangely, that list doesn't have Webkit or LLVM on it. That's by far their two greatest contributions. Webkit, one might note was based off of exsiting project KHTML. The history there wasn't good. They essentially ignored the existing community around KHTML, took the code modified it with all sorts of OSX only hooks, released that to comply with the license (GPL or LGPL). Then a couple years later they did it the correct way and created the webkit open source project. Konqueror now defaults to the Webkit implimentation and all is well. Although, google isn't that much better at working with existing open source communities, if you look at chromium browser.

So, yeah, there is still an uneasy feeling when it comes to Apple and open source projects.

No, its Apple's problem that they don't play well with Open Source Communities that they don't control. This story is about an Open Project which Apple has decided to utilize. Their past history with simular communities is very relavant.

Hmm... It appears that I did. Apparently, I care. I care that Apple doesn't play well with Open Communities. It will influence how well I play with Apple. If there are more that feel the same way as I do, then it becomes a problem for Apple. It would behoove them to work better with communities they don't control. Same goes for Google, Microsoft or any other company. Follow the license and follow the spirit of open collaberation.

The past story with khtml webkit, and the recent story about apple-only planned features in CUPS, and the general attitude of big and small commercial entities towards free software, should make people just a little wary.

The past story with khtml webkit, and the recent story about apple-only planned features in CUPS, and the general attitude of big and small commercial entities towards free software, should make people just a little wary.

Simple roadmap for you: If it is within Apple's Patents they are stingy. When it is outside of Apple's Patents they can be very generous.

The "apple-only CUPS" thing was about removing Linux-specific features out of the main distribution on OS X. Those features are still there and you can roll them back into CUPS if you need them, for example, on a Linux machine.

You mean the one where the KHTML devs complained that Apple was doing big changeset dumps making them hard to merge, so Apple switched to using a public svn repository? Or the one where committed sandboxing to WebKit in a way that (unlike Chrome's) is browser-agnostic and so can be used by other WebKit users?

As it happens, a fairly important part of Apple's Acid3 push actually involved merging in improvements from KHTML that they'd missed out on due to the fork. KHTML is mostly dead now but that's basically a result of all the developers subsequently abandoning it for Webkit in the expectation that their changes would be more widely used, and that process took years...

Even if they don't, having them bet on an easily user-updateable mapping system is pretty cool.

That said, OSM here in Germany is a bit disappointing. I've been adding a few locations around my neighborhood, but OSM based navigation apps are still pretty bad in terms of accuracy and actually finding house numbers:(

If I lived in Berlin, I might have noticed that earlier. What's missing for me is consistency - if I need to go somewhere that's nearly off the grid (but still easily findable with Google Maps, i.e. there's cell reception and everything:p), there's a pretty decent chance that I won't be able to get there with an OSM based app. A lot of the problems are from the front end, though, so not entirely OSM's fault...

You misunderstood his comment. From his perspective as a German, the attention to detail in OSM is lacking. I mean, in that example, there is no mention of where the nearest trash can is or where the stop signs are located.

I wish I was kidding. Germans are a unique breed when it comes to detail and rules.

You misunderstood his comment. From his perspective as a German, the attention to detail in OSM is lacking. I mean, in that example, there is no mention of where the nearest trash can is or where the stop signs are located.

I'd like to point out that the Berlin map beelsebob picked as an example does include locations of trash cans [openstreetmap.org]. It's just that the default map style on openstreetmap.org omits trash cans to avoid cluttering the map. They are, however, available in the database for anyone who needs them.

There are also several groups of recycling containers such as this one [openstreetmap.org] nearby, which indeed appear in the default map style.

So even as a German, I don't see a reason to complain about the level of attention to detail displaye

No, it looks like they took an April 2010 planet dump a while ago, and haven't updated since, which isn't great. This rather suggests that they don't know that they're using OSM data. Hence my bet being that one of the companies they bought used OSM data as a starting point, and then claimed to apple that it was theirs.

I suppose they will, simply because it would be a waste of their effort to maintain a fork, thus missing out on map updates from other parties. This is a naturally collaborative task. It's not like source code where patches from different parties are likely to have conflicts.

The question is what has Google given back. Webkit has given some code back to KHTML. WebKit2 is the open source alternative to chrome. While parts of chrome may be OSS, it is not like WebKit2, which is OSS. Certainly there would be no chrome without the development efforts of Apple.

This is not to say that Apple has not closed some projects. They have. But Google business model depends on acquiring public property, repackaging it, closing it, and suppling it for ad revenue and other compensation.

Anyone want to bet me a dollar they won't give data back? I'll take the first comer.

Think you might be off base here. Apple wants the user experience to be the best, not giving back would make theirs Second Best user experience. I could be wrong, but I think this is only a stepping stone and they'll be behind OSM.

OSM on Geocaching rocks. I just wish the MapquestArial wasn't such a steaming pile.

Even if they have the major roads, does it have the arterial streets like Google does? I use that information extensively on my commute.

In general yes but YMMV. It depends on the region: main cities in Europe and US are pretty well mapped whereas little village may have only their main road drawn. It really depends on where the users/editors of OSM live or what their interest is.

BTW if your commute is not mapped, you can just map it yourself, it's like wikipedia, but for maps and without the deletionist plague.

Given the little fuss over Apple's quiet-but-massive location tracking of cellular iDevices some time back, it would not be a total surprise to learn that they are planning on using their comparatively pervasive handsets in order to produce their own traffic data... If you have a street map, and you know how fast and in what direction the iDeviced commuters are moving, you may be able to draw useful inferences from there.

But that "tracking data" turned out to be just a local cache of cell tower locations that the phone had been near. It was not a record of the precise locations your phone had been and if someone wanted to track you it would only give accuracy to the roughly the nearest city.

Given the little fuss over Apple's quiet-but-massive location tracking of cellular iDevices some time back,

It's not a tracking database, it's just a database of locations of WiFi accesspoints and towers. It LOOKS like a tracking database because it grabs a few MAC addresses and tower IDs and sends them to Apple, who then sends you back a list of lat/long coordinates of those MACs and towers and a bunch more of nearby ones so you can triangulate your location.

Just to be a little more precise, the wifi/cellular tower locations in Apple's location data appear to be the averages of the GPS coordinates of the phones that received ID from that transmitter. They're not the exact coordinates of the antennas.

Plotting all the locations in my database after a day trip in a car, (I queried the data from the tables in KML format and displayed them in Google Earth), I did get a very good picture of where I went, and saw "larger blobs" around the places on the trip where I s

Slashdotted. They're working on getting the site back up.
The gist of it is that OpenStreetMap only really provides raw map data and building a useful product on top of that data, whether it's map tiles like Apple's photo app is using or turn-by-turn navigation like Skobbler, takes a fair bit of work. switch2osm.org takes you through the basics of putting the OSM data to work for you.