There is no simple solution for local storage

TL;DR: we have to stop advocating localStorage as a great opportunity for storing data as it performs badly. Sadly enough the alternatives are not nearly as supported or simple to implement.

When it comes to web development you will always encounter things that sound too good to be true. Sometimes they are good, and all that stops us from using them is our notion of being conspicuous about *everything* as developers. In a lot of cases, however, they really are not as good as they seem but we only find out after using them for a while that we are actually “doing it wrong”.

One such case is local storage. There is a storage specification (falsely attributed to HTML5 in a lot of examples) with an incredibly simple API that was heralded as the cookie killer when it came out. All you have to do to store content on the user’s machine is to access the navigator.localStorage (or sessionStorage if you don’t need the data to be stored longer than the current browser session):

The main issue: terrible performance

LocalStorage also has a lot of drawbacks that aren’t quite documented and certainly not covered as much in “HTML5 tutorials”. Especially performance oriented developers are very much against its use.

When we covered localStorage a few weeks ago using it to store images and files in localStorage it kicked off a massive thread of comments and an even longer internal mailing list thread about the evils of localStorage. The main issues are:

localStorage is synchronous in nature, meaning when it loads it can block the main document from rendering

localStorage does file I/O meaning it writes to your hard drive, which can take long depending on what your system does (indexing, virus scanning…)

On a developer machine these issues can look deceptively minor as the operating system cached these requests – for an end user on the web they could mean a few seconds of waiting during which the web site stalls

In order to appear snappy, web browsers load the data into memory on the first request – which could mean a lot of memory use if lots of tabs do it

localStorage is persistent. If you don’t use a service or never visit a web site again, the data is still loaded when you start the browser

This is covered in detail in a follow-up blog post by Taras Glek of the Mozilla performance team and also by Andrea Giammarchi of Nokia.

In essence this means that a lot of articles saying you can use localStorage for better performance are just wrong.

Alternatives

Of course, browsers always offered ways to store local data, some you probably never heard of as shown by evercookie (I think my fave when it comes to the “evil genius with no real-world use” factor is the force-cached PNG image to be read out in canvas). In the internal discussions there was a massive thrust towards advocating IndexedDB for your solutions instead of localStorage. We then published an article how to store images and files in IndexedDB and found a few issues – most actually related to ease-of-use and user interaction:

IndexedDB is a full-fledged DB that requires all the steps a SQL DB needs to read and write data – there is no simple key/value layer like localStorage available

IndexedDB asks the user for permission to store data which can spook them

The browser support is not at all the same as localStorage, right now IndexedDB is supported in IE10, Firefox and Chrome and there are differences in their implementations

As always when there are differences in implementation someone will come up with an abstraction layer to work around that. Parashuram Narasimhan does a great job with that – even providing a jQuery plugin. It feels wrong though that we as implementers have to use these. It is the HTML5 video debate of WebM vs. H264 all over again.

Now what?

There is no doubt that the real database solutions and their asynchronous nature are the better option in terms of performance. They are also more matured and don’t have the “shortcut hack” feeling of localStorage. On the other hand they are hard to use in comparison, we already have a lot of solutions out there using localStorage and asking the user to give us access to storing local files is unacceptable for some implementations in terms of UX.

The answer is that there is no simple solution for storing data on the end users’ machines and we should stop advocating localStorage as a performance boost. What we have to find is a solution that makes everybody happy and doesn’t break the current implementations. This might prove hard to work around. Here are some ideas:

Build a polyfill library that overrides the localStorage API and stores the content in IndexedDB/WebSQL instead? This is dirty and doesn’t work around the issue of the user being asked for permission

Implement localStorage in an asynchronous fashion in browsers – actively disregarding the spec? (this could set a dangerous precedent though)

Change the localStorage spec to store asynchronously instead of synchronously? We could also extend it to have a proper getStorageSpace interface and allow for native JSON support

Define a new standard that allows browser vendors to map the new API to the existing supported API that matches the best for the use case?

We need to fix this as it doesn’t make sense to store things locally and sacrifice performance at the same time. This is a great example of how new web standards give us much more power but also make us face issues we didn’t have to deal with before. With more access to the OS, we also have to tread more carefully.

124 comments

>What about Internet Explorer 10 ? Chrome ? Even the Blackberry 10 browser seems to support it. Really, the only platforms that does not improve are Safari and Opera here.

Which is the entire significant mobile market. The one place where network connections are so intermittent, slow, and expensive that having offline storage really pays off and offline operation is routine. Way to prioritize implementation.

We’ve decided that we are going to support offline operation only on mobile devices with websql and require network otherwise. Making our app with all its different views performant without SQL is just not practical.