iCloud gets kicked in the Core Data sync -- totally had it coming

iCloud -- specifically the iCloud frameworks for syncing Core Data databases -- has been getting kicked around lately, and by almost all accounts, deservedly so. Back in November developers like Instacast's Vemedio and Steve Streza of Informal Protocol posted about their problems with it and its opacity, and Paul Haddad expressed similar concerns during the second episode of Debug. More so even than Siri and Game Center server issues, it felt like proof positive that Apple faced significant challenges in a future where online services were as important as native software.

Since then, more developers have come forward to share their frustrations with Core Data sync. In a post intended to reassure users of NetNewsWire about the app's future in a post-Google Reader world, Daniel Pasco of BlackPixel wrote:

As far as sync is concerned, we knew we would likely need an alternative to Google Reader as early as last year. At the time, the option that seemed to make the most sense was to embrace iCloud and Core Data as the new sync solution of choice. We spent a considerable amount of time on this effort, but iCloud and Core Data syncing had issues that we simply could not resolve.

What seems to make the ongoing issues so vexing for developers is that iCloud was introduced with iOS 5 back in 2011, and while iOS 6 in 2012 was an improvement, it wasn't anywhere nearly improvement enough.

Ellis Hamburger of The Verge did a brilliant job summing up much of the reaction and reasoning, calling iCloud Core Data sync a broken promise:

Many veteran developers have learned their lesson and given up on iCloud’s Core Data syncing entirely. "Ultimately, when we looked at iCloud + Core Data for [our app], it was a total no-go as nothing would have worked," said one best-selling iPhone and Mac developer. "Some issues with iCloud Core Data are theoretically unsolvable (stemming from the fact that you’ve put an object model on top of a distributed data store) and others are just plain bugs in the implementation," he said.

One of the reasons for this is, just like with Game Center APIs, Apple has very little skin in the Core Data sync game. They're not making massive use of it, so they're not the first ones hitting pain points and problems. Their developers are, and that's a terrible, terrible thing for everyone.

Matthew Panzarino of The Next Web also pointed out that Apple conflating several distinct services all under the iCloud banner further compounds the problem developers face:

Recent criticism of Apple’s iCloud has exposed just how fractured the brand actually is behind the scenes. Developers are having problems with some of the technologies bundled together under the name and it’s causing some confusion. The truth of the matter is that there are really two iClouds, which couldn’t be more different.

Users who get their mail, contacts, or calendars synched without issue just don't understand what developers are complaining about because, for them, iCloud works, it just doesn't work in that developer's app. That leaves some users thinking developers are actually incompetent or lying.

Glassboard developer Brent Simmons, on Inessential, added that that's the risk of depending on systems you can't control:

How comfortable are you with outsourcing half your app to another company? The answer should be: not at all comfortable.

Just like services are the future for Apple, they're the future for a lot of developers. More important than hardware, arguably more important than software when that is already a core competency, iCloud is what Apple has to nail. Rather than getting kicked around, iCloud has to kick ass.

Update: Rich Siegel of Bare Bones software has weighed in with some interesting technical details on the different ways iCloud works... and doesn't work. It's not possible to pull a single quote out of his piece, so go read the whole thing on his Mistitled blog.

Reader comments

iCloud gets kicked in the Core Data sync -- totally had it coming

Exactly, as a non developer the little I use of iCloud works for me. But people need to understand the developers frustration. Apple needs to fix iCloud. And if they want to blow people out of the water with iOS7, it needs to have working iCloud as well things the general populace and average consumer can see as improvement.

I understand iCloud has had it's problems with cooperating with developers. But isn't cooperation their new mantra?
And maybe this is why I see more and more Dropbox sync options through various apps that I use. Maybe that's just because its ability to be cross-platform though.

All this syncing isn't as easy as it sounds. It's one thing to show the user two versions of a file and let them choose which one is "right." Remember the Sync app on OS X? You can't do that with a Core Data database (or at least I don't think there's a user-friendly way doing it.)

There's the complexity of maintaining database integrity. Do you copy over just the SQL statements and let each device perform its own transactions against its local copy of the database? Then trust that they all got the same results? This minimizes the amount of data being synced: you just pass the SQL around, and it's relatively safe from a database integrity standpoint.

Or do you try to copy over a chunk of actual binary data, then patch each device's local instantiation? This might require less computation for each device, but it just seems "brittle" to me. You're Frankensteining your database with pieces from other devices.

No idea which of the above, if any, Apple is using to sync Core Data through iCloud. But trust me, if your Core Data database is corrupted, it instantly crashes your app. I've seen that happen during development.

Another great post SockRolid. People think robust sync is easy when in fact it's really really hard. I've seen many a developer do a faceplant trying to roll their own sync. As much as people laud Dropbox it to doesn't offer support for reliably syncing persistent stores like Core Data. This is one of those watershed moments where Apple has to deliver the "right" solution for developers and move forward. iCloud key value sync is working well enough but we need Documents in the Cloud to catch up.

Maybe the best solution (for developers) is to sync NSDictionaries between devices through iCloud. Each device can then parse the dictionaries and save the data to its own Core Data file. I'm doing that because my app's data model naturally works with lots of small dictionaries that rarely ever need to be updated.

If the NSDictionaries are small, the iCloud latency will be low. And if there is ever some kind of Core Data corruption issue, one device won't screw up the data on any other device when iCloud propagates it. The user can "rebuild" the database from the dictionary files.

Talked to some developers (the Day One guys) at WWDC and they said that Core Data through iCloud was a nightmare. So I never even bothered attempting it. I'm sure Apple will fix it eventually though. They're pretty good at moving Objective-C and Cocoa forward every year.

Great article Rene, I still wish you'd hire a full-time editor and use the persons skills, but this article was so much more insightful than the Verge article. Your connection to developers being dependent on Apple was very poignant.

More and more I see Apple devolving into Microsoft and Nokia. THey didn't get it in the beginning that third party apps were what people really wanted. Now that Android is up to par, people are realizing just how well Android is working with developers because they get it; Keep the developers happy, and people will go where the best apps are. Articles like this are the reason I'll probably switch back to Android when Google releases it's X-Phone (fingers crossed).

I'm not saying MS isn't still wildly profitable, or creative. They are the ones who made the first tablet and smartphones. I just mean that they didn't jump on mobile the way Apple did and they got left behind FAST, just like Nokia with thinking Symbian could compete. I think their smartphone market share speaks for itself.

If Apple continues to drag it's feet, I see Android continuing to take market share from them and developers will eventually start seeing how much easier things work for them on the Android platform. Or an entirely new developer friendly platform will come along that realizes just how integral developers are for making a platform successful. It's just paying attention to how the past has ALWAYS worked.

Cloud cloud cloud... ah I so miss the times where nothing was "connected". It's such a mess nourdays: you have Google Drive, Dropbox, iCloud, Mega, Skydrive, and countless more online storage solutions. Add to that the "services" they claim to offer and it all looks marvelous. The only problem here is all these apps barely talk to each other and you effectively multiply the number of apps needed to retrieve data, whether it's files or database contents.

For the end user I don't see it as a benefit versus the "old way": a file system the serves as repository where every app can go and put or retrieve stuff.

I don't know if this cloudy trend will lead us anywhere near a better solution than what we had prior to that.

The ideal would be a kind of mix between the old way and the new one: export the whole filesystem to the cloud and use our devices just as terminals BUT keeping the option to be able to work offline with a copy of the filesystem (to avoid the Chrome OS issue). Something like that.

I'm a developer myself and though I don't develop on the iOS platform (yet), but I've had to work with my share of systems that my application had to depend on without giving me any control. One such instance was a Point of Sale solution for a leading enterprise solution who's API had so many bugs and performance issues that even though the simplest logic in the POS app would take a second or two to execute, the API of the backend software took its sweet time in completing its tasks that could have been completed in under a second as well. As a result, customers who used the Point of Sale didn't get the proper performance from the system and ended up blaming us without realizing that all of our tests yielded top end results but failed when it came to the 3rd party API. Admittedly, the 3rd party/backend company agreed to the issues in their API and notified to be working on updates to fix/improve them... but after over two years of back-and-forth, though the API improved in some placed, it worsened in some others.

Right now iCloud serves my purposes perfectly - syncing the data that I want over the cloud. However I did review some of its documentation and had the pleasure of discussing it in detail with a friend who's been working on the Apple Platform and developing iOS apps, and his frustrations with iCloud colored me disappointed.

Hopefully iOS7 will bring with it a much needed upgrade to iCloud so that, like you said, it can stop getting kicked around and start kicking some ass! ;)

Apple has a real opportunity with iCloud. It would be really great if we had some sort of Office App that worked on the iPad (or iPhone) and we could synch that easily with iCloud and distribute among work groups. There's an existing installed base of iPad users who'd like to get some work done in this way. Just a thought.

Apple learned from the MobileMe fiasco that they need to be very careful rolling out their services. They used up any goodwill way back in summer of 2008. People won't cut them any slack, especially not the tech press, so they need to 1. roll out fully debugged services, and 2. get the design right the first time so they don't have to change or remove awkward features.

The problem I have with all this negativity about iCloud is that everyone admits that "database sync" (Core Data Sync), is almost impossible to implement on *anyone*'s platform and has yet to be successfully implemented anywhere, yet it's somehow Apple's fault simply because they are the latest to try and make it work? This makes no sense at all.

Secondly, as all the articles also seem to agree, most of the time, the sync that *works* (Document sync), which is also the syncing method that Apple predominantly uses, is actually all the users are asking for.

I can think of several very high profile apps that users have been asking to have iCloud sync with for ages, where the so-called document syncing model would work perfectly fine, but still the developers point to this problem with Core Data syncing, even though no one is actually asking them to use that.

I know these sites are biased in favour of developers and that they will disagree and call me al kinds of names, but it still seems to me to be entirely the fault of the developers here. Just use regular document based syncing (which obviously works because dozens of other apps already use it without any problems at all), and stop trying to defend your decision with this nonsense about Apple being at fault for being unable to implement the "miracle" protocol.

What puts the lie to the whole argument is that the developers are pushing everyone to use DropBox or something similar instead, but DropBox also doesn't do the kind of syncing we are talking about and also uses a simple document syncing method as most of iCloud does.

Yea..it's a sort of developer facing issues, but non-dev people and even geeks could take presumption that iCloud as a product doesn't work, which is NOT the case! As an iOS developer, I want to Thank Rene for putting up correct headline. Even the Verge headline almost reads like iCloud doesn't work. Apple is trying to do something that no one even planning. There is no CoreData like object-lifecycle framework in competing platform and syncing that using iCloud across the devices is hard job. Let see what iOS 7 has to offer...

All I use it for, really, is backing up my contacts, and the Find my iPhone stuff. I have a calendar set up that way since my parents use it, but I don't actively use that feature. But I don't really get into the serious involvement with the "wrongs and rights" of the service. I'd like to see it expanded but I'd also like to see it not require me to update my OS software on my iMac and tower all the time to get it.

I would like to have iCloud more in depth with Applications. But I'm really hoping Apps get better communication between other Apps. While I'm not a developer, I've been told this is a big problem. Also that Apps don't have access to all the APIs thereby being hindered.