'We were taking a step into the unknown': why The Ghostkeeper's Journal is Carlton's most ambitious AR book yet

Today marks the launch of The Ghostkeeper’s Journal, "the world’s first augmented reality novel for children 11+" according to publisher Carlton Books. At heart, the...

Today marks the launch of The Ghostkeeper’s Journal, "the world’s first augmented reality novel for children 11+" according to publisher Carlton Books. At heart, the book is a classic ghost story, but using the accompanying AR app, readers also scan the book’s pages to reveal clues, capture ghosts and unlock the secrets of the story.

Carlton has form with AR, having created a whole series of Digital Magic titles as well as the iExplore range of interactive natural history books. But Japhet Asher, the company's digital director and author of the book, is clear that The Ghostkeeper’s Journal marks a leap forward for both Carlton and the whole concept of AR fiction. So we asked him to give us the inside story on the research, technology and design decisions behind the project - and the other AR applications he thinks publishers should know about.

How does AR make The Ghostkeeper's Journal a better book than a non-AR equivalent?

First and foremost, it’s because The Ghostkeeper’s Journal isn’t just a book — it’s a haunted book. From the very beginning, I wanted the reader to feel that the book in their hands was more than simply a device for delivering words and pictures. The book is an artefact from the world of the story. It belonged to a Ghostkeeper and contains ghosts. The Ghost-o-Matic app allows the reader to uncover missing pages, reveal paranormal phenomena, find hidden clues and capture ghosts themselves, making the reader a much more active participant in the narrative. So I’ve been able to tell a remarkably immersive story, with elements that simply wouldn’t be available without the power of augmented reality.

But the advantages to making The Ghostkeeper’s Journal an AR powered novel don’t just accrue in my approach to story-telling. For parents and educators, an AR app that connects to a book creates a journey between screen and page that appeals to the current “screen generation” who’ve grown up with ubiquitous smart devices, is a massive positive. Data shows that many tweens and teens have left books behind thanks to the allure of digital content. I hope that The Ghostkeeper’s Journal can remind these readers of the pleasure of the book as a physical object, the power of words to generate characters and mystery, while the AR elements from the app can fulfil this generation’s love of immersion, interaction, personalisation and control. Most of all, I believe that the combination of book and AR app delivers an deeply immersive storyworld that is tangible and present in your own world, raising the emotional stakes of the narrative.

(How) did you scope out demand (from the kids themselves or parents, for example)?

Carlton Books has created and published 30-AR powered books over the last eight years, so we know from the volume of books we’ve sold on our previous titles – over 4.5 million books in 30 countries – that there is an appetite for the digital/print hybrid products. But up until now, our titles have been primarily based around non-fiction subjects or licensed entertainment franchises where we know that the book alone, even without the AR, would be in demand with parents and kids. In fact, we always focus on making sure that the books we deliver can stand on their own as great experiences around compelling subjects, as well as delivering something even more extraordinary and informative once combined with our augmented reality apps.

For The Ghostkeeper’s Journal, however, we were taking a step into the unknown. From the beginning we wanted the book to work in manuscript form, through its characters and story without reference to the AR, and indeed our co-edition partners would not have proceeded with us had they not believed in the text as a story. We also knew that many parents and educators see screens as competition for books and reading, rather than allies in discovering books and encouraging literacy.So we focused on making sure that our concept would work by creating a virtuous cycle between app and book. We also worked with consultants and kids along the way to make sure that they understood how to use the book and app together, and were able to understand the narrative and solve the puzzles that unlock new elements of the story world. We made changes to our approach based on the feedback from that testing – I was determined that every reader could find their way through the experience and enjoy the story world, regardless of their strength as a reader, puzzle solver or ghostkeeper!

Why did you choose a ghost story?

As the creator, I wanted to tell a ghost story because I lived for a time as a child in a house that was said to be haunted by a ghost known as ‘The Friendly Colonel’. Perhaps as a result, I never thought of ghosts as particularly scary, and recent fiction from Harry Potter to Coco has encouraged a more benign view of spirits. But even friendly ghosts can have enemies, and the secret at the heart of The Ghostkeeper’s Journal lies in one of the hidden AR messages near the start of the book: “Beware the Ghoul!”. It’s a world where I hope our readers will enjoy spending time.

But one of the things that augmented reality technology is best at is giving the user ‘special powers’, which in mundane terms might be the ability to see the schematics of an engine they are looking at, or in more exciting fictional terms, the ability to see paranormal phenomena in the world around us. The Ghostkeeper’s Journal effectively allows the reader to tap into a superpower they already have – their imagination – and enhance it with another that the book describes as ‘Ecto-Sense’: the ability to perceive ghosts. We are able to tell a story about a secret society that seeks out and helps troubled spirits known as ghosts, and makes you a member of that society because of the powers you have, which are enhanced through AR to reveal amazing phenomena on the page and in the world around you.

Technically, how did building this ‘from the ground up’ involve a different process from adding AR to an existing book?

It’s not so much about adding AR to an existing book, as at Carlton we’ve always created book and AR app together. But in the past we’ve worked with existing characters from licenses of animals/phenomena from the natural world to make our books and apps.With The Ghostkeeper’s Journal, we could design a story and visual treatment that took best advantage of what we could deliver through AR and the page, so we’ve made something incredibly rich with the resources at our disposal, and indeed something that couldn’t be experienced any other way.

What useful failures or pivots did you make along the way?

Actually, when I recently went back and read my original outline for the book and app experience, it was remarkably similar to what we ultimately made, which is pleasantly surprising. However, I know that at various points we took detours that were ultimately abandoned. We considered a much more elaborate physical package – a box containing, book, documents, cards and more – to justify a higher price point for the product, given how much we were investing in the augmented reality experience. Ultimately, though, we chose to place our confidence in the power of our content, and to keep the price point down, so we could sell more books (the app is free to download) and reach more readers.

On the technical side, when we first started on the page design, I had in my mind the hope that the differences in leading and kerning generated by the type would create sufficiently distinctive patterns for us to trigger unique AR from pages with print only. This ultimately didn’t work for two reasons – the variations in the patterns were too small to create anchor points and, more obviously in retrospect, the print patterns would be different for every language. Instead, we developed a visual style we called ecto-printing, with different ghost types generating patterns on the edge of pages for their case file. These became beautiful decorative elements as well critically functional trigger points for AR.

What other applications do you see AR having for the book trade?

The most obvious opportunities for publishers with AR lie in the education market. Our iExplore series of natural history books is already being used by teachers in the classroom. If they have a tablet they can link to a monitor, then the whole class can participate in the learning memory that bringing a wild animal or a giant arachnid or a cross section of a brain into the classroom can create. Any educational publisher who’d like to partner with us on expanding this activity we’d love to hear from. Many corporations are already adopting AR for training, and this is a clear growth area.

AR does not need to involve animation or gameplay. A hybrid physical book with audio could be a great experience, for example. There are infinite ways for the physical and digital to support each other, and over the next few years consumers will come to expect it.

What other AR projects or companies do you admire/ draw inspiration from?

I’m really excited about the future of AR. There’s been an extraordinary explosion of creativity in apps enabled by ARKit and ARCore – I love Zach Lieberman’s Weird Type app, for example. Everyone knows the location based AR of Niantic’s games, but there are interesting AR apps now powered by Google Maps that show you directions on the street, and others that deliver rich experiences at specific locations like the Gruffalo Spotter app from the UK Forestry Commission. But I have a very broad view of augmenting reality – experiential theatre like Punch Drunk is AR, creating a soundtrack for your daily walk or run is AR. And I’m also a big fan of Magic Leap – not just the potential of their device and their technology, but their ‘creativity first’ attitude to delivering content for their platform. It’s an extremely exciting time to be a storyteller, and an equally exciting time to be a consumer of stories.

Molly Flatt

Molly Flatt writes how tech changes the way we work, think and live. She is associate editor of FutureBook. Her debut novel, The Charmed Life of Alex Moore, is out now. You can find her on Twitter @mollyflatt.