Extras

Category: Scrivener for iPad and iPhone

We have a lot of people asking after the iOS version given that I haven’t posted much about it on here recently (although I have been giving incremental updates over on the forums). The lack of updates isn’t intended as a slight against our very valued users; it’s just because we’ve had our heads buried in code and haven’t been coming up much for air.

The good news is that our iOS version is (at last!) feature-complete and is currently in internal beta-testing. We’re in the process of smoothing out the rough edges we find through use and fixing bugs before we make it available to our wider beta-testing group. Having taken so long to get to this stage (we know), we don’t want to fall at the last hurdle and rush beta-testing. After all, the sync code is complex (it was a three-month job in itself) and we want to ensure it’s never going to cause any data-loss in the real world (it is holding up well in testing so far, though). Once it’s out with our wider beta-testing group, I’ll start posting some screenshots and showing off the features. Given past debacles, I’m hesitant to talk about release dates, but we expect a summer release, though whether mid or late summer will depend on what is thrown up during beta-testing, of course.

To answer some common questions we’ve had about the iOS version:

Version 1.0 will sync with the desktop version using Dropbox. The sync code has two components, the Dropbox-specific code, and the code that handles merging all the changed files between platforms and dealing with conflicts. This second part of the code has been designed for reuse with other sync clients, so we plan on adding support for sync solutions other than Dropbox post-1.0 (Cubby is one that has been requested several times, for instance).

There is no iCloud syncing in 1.0 because iCloud does not work well with package-based file formats such as Scrivener’s. We were hoping that iCloud Drive would provide a solution, and are still looking into it, but the APIs don’t seem to provide the flexibility we need on the iOS side. (The reason syncing is more complex with Scrivener than for most apps is because .scriv files are really folders, which is how we can allow you to import any sort of research file and only load files into memory as they are viewed in the app.)

You can also transfer projects simply using iTunes, of course.

When the iOS version is released, there will be simultaneous updates for both Scrivener for Mac and Windows that provide sync features. So don’t worry, Windows users, we have no plans of leaving you unable to sync!

Version 1.0 will not have any dedicated screenwriting features, but scriptwriters will be able to sync their scripts using Fountain syntax for paragraphs.

Scrivener for iOS is a full-featured writing environment – the binder, corkboard, outliner, full rich text editing (with comments, internal links and so on), import, export, reference to other documents (including images, PDFs, media files) – all the core features of the desktop version are there in the iOS version, but with a UI designed from the ground up for mobile devices.

Scrivener will require iOS 7.0 or above.

It’s been a long journey, but at last the light at the end of that proverbial tunnel is in sight. I can’t wait to start showing you screenshots and talking about it properly.

It’s not all about the iOS version, however: our heads have been buried deep in code working on serious and extensive (and exciting) updates for our existing platforms, too. But we’re not yet ready to talk about those, and we want to get the iOS version out first.

EDIT: We’ve had a lot of users ask about how to get involved in beta-testing the iOS version (thank you!). In the past, the way we have always selected beta-testers has been to look out for users on the forums and social media pages who know their way around Scrivener, are good at reporting bugs or giving constructive feedback, or who post useful information on using Scrivener for fellow users. We continue to look out for new beta testers that way, and from that, we already have quite a large pool of testers for the Mac and Windows versions to whom we’ll be sending out iOS beta invites to begin with. Once we’ve got a better idea of what the numbers are like, and when the time comes, I’ll post more information here on how to apply to become a beta-tester for anyone else interested. Please note that it’s not a good idea to apply to beta test if you just want early access, though – the beta-testing stage is where we find the most bugs, so you have to be prepared for crashes and even possible data-loss (obviously we hope to have ironed out data-loss bugs by the beta, but beta-testing is where users find the things we’ve missed and we cannot guarantee that there won’t be hideous bugs lurking – beta-testers are guinea pigs). Beta-testers also have to be prepared to spend time trouble-shooting and providing us with detailed reports on every issue they find. We’ll provide more details once we’re nearing the end of internal testing.

We’re over halfway through 2014, so I wanted to give our users a quick update on the progress of the iOS version of Scrivener. We receive emails and forum posts daily asking how it is coming along, and because of how long it seems to have been in development, we’ve even had users assume that it has been abandoned (we’ve even been accused of lying about it being developed at all, but I promise that’s not the case, even if we have continually underestimated the time involved).

The short version – the “tl;dr” version as the cool kids say – is that we still expect it to be finished this year, but now very much doubt it will be released until next year (2015) because of the amount of testing we need to put it through before letting it loose on the world. Don’t misunderstand – we are closing in on the final stages of development right now. But these final stages can still take a while, and I’d rather tell users now, in the middle of August, that we might be pushing into 2015, rather than keep my fingers crossed that we can squeeze it into 2014 and then only announce a delay at the last minute when we realise we aren’t going to make our target. Thorough testing of the iOS version is crucial, because it is designed to sync with desktop projects, and syncing can be a dangerous process – we want to ensure that any bugs that could cause data loss or corruption are truly ironed out before putting it into users’ hands.

I’m aware – if only because we’ve had it pointed out a lot! – that we have been saying “later this year”, then “next year” for two years now. Naturally, users wonder how so many delays can be possible. For a start, there’s a fairly obvious rule of software development: software development takes time (and developing good software takes a good amount of time). And then there’s the corollary to this rule: software programmers always underestimate how much time it will take (myself included).

Take the sync code, for example. Most iOS apps can take quite a simple approach to sync, using the built-in features that Apple provides for iCloud. Or they can use the Dropbox API to manage uploads and downloads. Nothing is so simple for Scrivener. One of Scrivener’s main features is that it allows you to bring in research files – photos, PDF files and so on – so that you can view them next to your text and keep everything together in a single project. In order to achieve this, Scrivener requires a file format quite different from most writing software. Scrivener, in fact, is something of a hybrid between a document-based program and a shoebox app.

Most writing software packages are document-based programs. A document-based program allows you to create, open and save documents, showing different documents in different windows. They use single, flat files that contain all the text and any graphical data for each document. These files are loaded entirely into memory when opened, and written out in their entirety as completely new files when saved.

A “shoebox app”, in contrast, usually has a single window that allows you to interact with many different files, opening them inside the one window. There will be a single folder on disk storing all of these files, but you usually won’t touch that folder at all. iTunes and iPhoto are examples of shoebox apps.

Scrivener is a bit of both. As a user, you interact with your Scrivener projects as though they were documents – each project is opened in a different window, and you have a .scriv file on disk for each project. But those .scriv files are really folders containing many different files – they are not flat files at all – and you navigate between them in the project window via the binder. This means that Scrivener doesn’t have to read these files entirely into memory on opening them, or save the whole file when saving. This is very important for projects that have a lot of research in them – you don’t want Scrivener storing a gigabyte of PDF files in memory while you are typing, or having to write all of this data to disk every time auto-save kicks in. This file format allows Scrivener to load into memory only the data that the user needs for a particular session, and it only ever needs to save what has changed (which also reduces the chances of catastrophic data loss caused by file system save errors).

The only downside to this file format is that whenever Apple introduces new technologies – Versions, iCloud, Handoff – it only provides out-of-the-box support for simple document-based apps. All of which to say: we have had to put together, from scratch, our own syncing solution so that our iOS version syncs with our desktop version. (We will use the Dropbox API, and hopefully other syncing APIs as time goes on, but all the conflict resolution handling has to be done by us.) Over the past couple of years, we’ve spent a great deal of time discussing how this could work, hashing out various ideas and solutions that could deal with the conflicts that will inevitably arise if someone works on both the iOS and desktop versions while offline so that the changes become out-of-sync. (Everything has to be considered: what happens if the user edits scene A on the mobile version but then also edits it on the desktop before there has been a chance for them to sync? What if the user’s kid opens up the app on the iPad and hammers away at their precious novel? How can we tell the difference between a scene that’s been deleted from one device and one that’s been added on another? And so on.)

Once most of this had finally been hashed out, it then took me a month to write out the design document that covers how the code will work (a design document is necessary for our Windows team to follow as well as for me and our iOS developer). And then yet another month to write the code itself, cross-platform for Mac and iOS (our Windows team will need to write this code separately). This has been what I’ve been doing for the past couple of months, and there is still work to do on getting all the sync components that have been written into place on the Mac and on iOS and then thoroughly tested (then testing on Windows).

So, that’s between three and four months on a single (but integral) feature. (And while I’ve been doing this, Tammy, our iOS developer, has been adding various “smaller” features that had to wait until late in development – printing, basic compiling, import and export, statistics, split and merge, find and replace… The sort of features that it’s easy for users to forget when suggesting that Scrivener for iOS should just be simple and only have the basics, but without which, Scrivener for iOS would feel crippled. And even some of these smaller features can take a couple of weeks each to implement, of course.)

The trouble is that it’s not only non-programmers who (understandably) don’t appreciate how long seemingly simple things can take to code – we do it ourselves. We’ll often think that a feature will only take us a day or two to complete, then find ourselves growling at the screen, desperate to have done with it, two weeks later. (This is no doubt why software and games are so notorious for being delayed throughout the industry.)

In a program as complex as Scrivener (even on iOS it requires a degree of internal complexity if it is to work with the desktop version), where various features can take a month or more to implement, the time creeps up on you and suddenly two years have passed… There are certainly programs out there that can be written in three or four months, but sadly, Scrivener isn’t one of them.

Not that “development takes time” is the whole picture. Something like Scrivener can easily take a couple of years to develop properly, but it has now been two and a half years since we started work on it; we had hoped it would be finished by now as much as anyone else – we don’t enjoy frustrating those of our users who write on mobile devices. The other big factor for us has been scaling up. Scrivener started out as a hobby – the software I wanted for my own writing. I wanted it so much that I taught myself how to code in order to create it. For the first couple of years, L&L was a one-man operation, and we are still a very small team. I work alone on the Mac version, two programmers work as a discrete unit on the Windows version, we have a couple of other full-time team members who write and maintain the manual and tutorials and handle all QC testing and some support, and we have a couple of part-time support staff. We have no central offices but are scattered across the world (the UK, US, Australia and Bulgaria). None of which is unusual for shareware, “artisan” software such as Scrivener.

The wonders of the information age make it easy for geographically-scattered teams to stay in constant touch and work together, but as a small company with such a setup, it meant that it was not possible for us just to hire some programmers and stand over them until the iOS version was finished. I was always adamant that I would not write the iOS version myself, because I’ve seen too much Mac software suffer as its developer has spent all his or her time working on iOS, and that would have been unfair to our existing user base. (Moreover, the Mac version is where I live for my own writing, and Scrivener is always going to be predominantly a desktop program.) So it was key to find someone self-motivated, with a good understanding of Scrivener, who could develop Scrivener for iOS and keep on developing it into the future, beyond version 1.0.

As I announced last year, unfortunately our first developer had to step away from the project because of family health issues, which had also stalled development progress throughout the eighteen months she had been working on it. We were very fortunate to find Tammy Coron, an experienced iOS developer and Scrivener user, to take over development. When she took over, she made the painful decision to rewrite much of the existing code, in part because of new technologies that had been introduced with iOS 7.0 (a rich text system at last!). Which means that while our iOS version has been in various stages of development since the end of 2011, the form it is in now has only been in development for just over a year.

We are really looking forward to releasing the iOS version – in many ways it’s become the proverbial albatross around our necks.* But it is getting there, and we hope the subsection of Scrivener users who are eager for an iOS version will be patient for a little while longer. We are genuinely grateful for your enthusiasm. I’m not known for great reserves of patience myself, so I do understand the frustration of some users who are more vocal in their eagerness, but to those I would say, please do try to remember that we want to get our iOS version into your hands as much as you want it. We’re a tiny company but we have put a lot of energy and resources into the iOS version (and for all we know it could be a loss-making venture), and we think Scrivener users will be happy with the final product.

I’ll post later in the year more on what sort of features to expect, but suffice to say that it has full rich text editing, a corkboard, an outliner, an inspector and, on the iPad version, the ability to view research alongside your text as you type. It is, essentially, Scrivener on iOS.

P.S. For anyone interested, I’ve uploaded a sort of developer diary for the iOS version here:

Given the number of tweets and messages we get enquiring after the progress of our iOS version (thank you everyone for your enthusiasm!), I wanted to give everyone a quick update, since there have been some important developments recently.

As many of you know, development started on Scrivener for iPad and iPhone early in 2012, and we had hoped that we would have it ready for the end of that year; we later revised that estimate to the first quarter of 2013. And, as many of you have pointed out to us, here we are in April – so where is it?

I’ll provide the information you are most likely after first: it will still be a little while–certainly much longer than we had hoped. We can give no firm release date yet, and given that past estimates have been wildly off, I’m not even going to make a guess at this stage. We very much hope it will be out this year, in time for National Novel Writing Month, but we’re not going to make any promises at all until we know for absolutely sure that we can keep them.

The frustrating part for us is that, for the past four or five months, we have had a version of Scrivener for iPad that is in many ways so nearly there and yet still not ready for beta-testing. We hit snags with the rich text system (or iOS’s lack of one) and building the synchronisation code is incredibly complicated because of Scrivener’s package-based file format, but we had most of the other basics in place and felt we were really making good progress.

Unfortunately, however, owing to unforeseen and serious health problems in our iOS developer’s immediate family, over the past few months our original developer has been unable to spend the time on the project that is required to get past the final roadblocks and finish it. It’s been a difficult time for everyone as we tried to work out the best way to proceed, all the while hoping things would get back to normal even as time slipped by, especially since she has done such an amazing job so far (the iOS versions of the binder and corkboard are a joy to work with). It gradually became clear, sadly, that our iOS developer had no choice but to officially reduce the time she could dedicate to the project so that she could concentrate on her family, and that we would need to find someone else to step in. But because we’re a small company with limited resources and no headquarters or offices, it’s not as though we could just throw money at the problem by hiring a bunch of developers and supervising them until it is done; we’re not Microsoft, Apple, or even Omni. We needed to find another iOS developer not just passionate about coding, but passionate about creating an iOS version of Scrivener in particular.

It is with great pleasure and some degree of relief, then, that I can now announce that we have found a new developer to focus on getting Scrivener for iOS completed. (The original developer will continue to work on it too, as much as her current circumstances allow.) The new developer is Tammy Coron, an experienced iOS coder who is also involved with Nickelfish and developed the iMore app. She’s a dedicated Scrivener user and, like many of our users, is desperate to have the iOS version for herself. (It’s not just our users eager for this–I want to be using the iPhone version as soon as possible too.) There’s an interview with Tammy available online for anyone interested here:

Tammy can also be found on Twitter @Paradox927 (which is the same user name she uses on our own forums).

Tammy has been getting up to speed with the project for the past fortnight and has now started diving into the code proper. We hope to provide more news over the coming months, but we’re all excited to have her on board and to once again be forging ahead with getting our iOS app completed.

To those desperate to have an iOS version as soon as possible, we are very sorry for the delay, but remember that in the meantime Scrivener for Mac already has some great ways of working with mobile devices (e.g. via external folder sync). We know we’ll get a bunch of angry emails and tweets over this, but I hope that most of our users continue to find Scrivener one of the best desktop writing packages around, and that the iOS version will be worth the wait when it is eventually in everyone’s hands. We also hope that everyone will understand why we haven’t wanted to say too much about what has been holding up development until we really had to and had already got a solution in place.

We’ve been dropping hints for a while, but can finally make an official announcement: Scrivener for iOS is now in development.

I’ve said all along that I wouldn’t develop an iOS version myself, any more than I would have tried to code the Windows version myself – with such a small team, I think it’s in the customers’ interests to have a dedicated developer for each platform to ensure that each version is always kept up-to-date, and my hands are full with the Mac version. To that end, I’m very pleased to announce that we have just signed contracts with a developer, Jen Yates, to develop the iOS versions for us. Jen has been beavering away in secret for two or three months now, putting together some proof-of-concepts:

I have to say that moving index cards around on a touch screen is a lot of fun, and the corkboard implementation she has come up with is, I think, one of the nicest I’ve seen on a touch screen device in terms of selection and dragging.

It’s still early days, though – we are about to embark on the design process proper, and all we can say in terms of a release date is that our iPad and iPhone versions will be out some time in 2012. If you would like to share your own ideas about what you see as essential in an iPad or iPhone version, please feel free to drop by our special sub-forum for iOS suggestions here:

Thanks to everyone who has contacted us to show their enthusiasm for an iOS version of some kind – we hope you’ll like what we come up with – and welcome to Team L&L, Jen!

EDIT 17/12/11: Thanks for all the enthusiastic responses to this post, much appreciated. To those asking about an Android version, this is on our radar too, don’t worry. We have to take it one step at a time, though. Our design process for iOS will take Android into consideration (although our iOS version will be Cocoa and native), and we hope to investigate Android in more detail later in 2012.

It’s a beautiful-looking text editor, but the really cool thing is that it syncs with Dropbox. This means that it can be used with the upcoming Scrivener 2.0’s “Sync with External Folder” feature, which is an alternative to the Simplenote syncing. So you will be able to sync documents from a Scrivener project with your iPad or iPhone, edit them (or create new ones) in PlainText, and then bring your changes back into Scrivener when you return to your computer. (There are other text editors coming out for the iPad that use Dropbox sync, too, many of which will also work with this Scrivener 2.0 feature.)

Anyway, congratulations to Jesse on getting PlainText out – it looks and feels great, and I think a good number of Scrivener users with an iPad will find it an invaluable way of editing documents on the go.