State Library Stories

For the last year, we have been working with the State Library of New South Wales to help create a digital storytelling project as part of their new website. The sub-site, Stories, would offer visitors an engaging way of understanding the Library’s image and object collections.

We created these stories as scroll-down responsive web apps. Our in-house team handled the copy, curated the content and coded each story to create a self-contained storytelling experience. The digital stories are interspersed with large-tiled images, animations, videos, maps, interactive nodes and hand-drawn illustrations.

Here are our first three digital stories. Check in regularly to the Library’s site as new stories get released throughout the year.

Holtermann Collection

In 1951, a hoard of 3,500 glass plate negatives from the nineteenth century was discovered in a garden shed in Chatswood. The collection included images of the goldfields, portraits, shopfronts, and the largest glass plate negatives in the world showing a panorama of Sydney Harbour.

Made using a wet plate photographic process, these negatives have no grain and are capable of extreme enlargement. Visitors can now zoom into these photographs and see the details of faces, hand-written posters and even the labels on pharmacy bottles.

This extraordinary collection of natural history illustrations is the Library’s most significant addition of early colonial material since the 1930s. Containing 745 watercolours in six volumes, the collection conveys Europe’s naïve yet genuine sense of wonder at Australia’s unique natural history.

Eora: 1770-1850 is created through a close and innovative interrogation of the European records of early colonisation. Compiled from letters, maps, prints, books and drawings, this digital story pieces together a surprisingly rich account of Aboriginal lives and families after contact. Running contrary to the notion that colonisation completely displaced Aboriginal people, this account gives testimony to a continuing Indigenous presence in Sydney.

Light Reading Issue 3 is all about two new art installations: a stereographic video projection and a hand-drawn virtual reality experience that is traveling around Australia. As always, you can find documentation of all our new projects here on our website.

--

The Inland Sea

Many of us at Lightwell have been to art school and much of our creative thrust and artistic energy comes from the things we learnt while skipping classes there. We recently returned to the hallowed halls of Sydney College of the Arts to exhibit an interactive work comprising three large scale projections and a touch interface.

Using an iPad, visitors could bend and warp spherical projections of three videos shot above the surface of the ocean. The videos were mapped to a virtual sphere and, using the controls on the iPad, visitors could change their point of view to look from inside or outside the sphere. Some users beheld a gaping rift in the time-space continuum while others felt merely a bit queasy.

We had been working for some time with 360-degree virtual spaces for Google Cardboard and Oculus Rift, but hadn’t shot any 360-degree video before. 360-degree videos are fairly common now - YouTube shows a number of these. They are wonderful for capturing spaces and events and users can interact with these video streams and effectively guide the camera after the fact.

We shot our video using 6 GoPro cameras mounted on a 3D printed camera mount. We dangled our ball of GoPros from a bridge over the waves at La Perouse, and wound up with 6 different video segments to be stitched together into a single hi-res piece.

Here is what the stitched footage looked like (flipped 180 degrees) before being warped into a stereographic projection.

For the geometry geeks, the process of flattening a sphere onto a rectangle was known to the ancients as an equirectangular projection and they used this technique to produce their maps of the world. Using VideoStitch (a video stitching software) and PTGUI (a panorama stitching software) we created some wonderful seamless videos after only 23 attempts and days of manual adjustments.

To create a stereographic projection that works as a virtual sphere we take the two ends of a rectangle, and twist them round so that they meet each other. The effect of doing this with a panoramic video that has been mapped onto a plane is a remarkable video of a 'little planet'.

There are a variety of variables that can be tweaked with a stereographic projection, and we wanted users who were interacting with the installation to see how futzing around with x and y coordinates would influence the video in each of its states.

Click here to explore a WebGL version of the software (you'll need WebGL ready hardware).

You can also check out the documentation video of the three linked installation projections in action here.

The exhibition Lines of Force was curated by Nicholas Tsoutas and featured work from artists such as Nikki Savvas, David Haines, Anne Zahalka, Mark Titmarsh, Eugenia Raskopoulos and many others.

--

Virtual Voyage

We like a mix of low and hi tech. A lot of our projects combine old media and new technology to bring some humility to the new and bit of life to the old. We are currently working on projects that combine watercolours with LED screens, wet-plate negatives with HTML5, hand-drawn collages with touchscreens and smartphones with cardboard boxes.

One of these projects, Virtual Voyage, is a VR experience for the Google Cardboard and Samsung Gear VR headset that gives users a glimpse into two vibrant, hand-drawn environments. Visitors can discover the thriving Great Barrier Reef inhabited by shoals of fish and scurrying crabs, or the serene Blue Mountains where lorikeets and cockatoos spiralled through the trees or danced in the undergrowth.

We used a combination of libraries from Unity 3D and the gyroscopes embedded in Android devices to create two fully immersive 360 degree views from a single stand-point. The watercolour artwork was all drawn and hand-painted in our studio, then digitally animated in Unity.

One of the main challenges we faced in creating the Virtual Voyage was how to bring to life various 2D elements in a 3D world. We ended up creating a complex diorama of animated and static objects to give the visitor the feeling that each object was hand-drawn directly into its environment.

Simply mapping scenes to a flat surface wouldn't have given users the sense of depth that true stereoscopic imagery provides. Line weights and objects had to be scaled and changed according to their distance from the virtual stereoscopic camera.

Virtual Voyage was created for AMP Capital's 2015 Amplify Festival and retail centres around Australia. If you're interested in the project, you might be able to catch it at a centre near you. There will temporary installations in various locations, including Malvern Central, Marrickville Metro and Ocean Keys.

Hello there. This is the second in the series of our thoughtful and life-affirming newsletters. This one is all about interactive touchtables and a virtual dive under Sydney Harbour built for the Oculus Rift headset, which is a simple way of delivering virtual reality experiences. And a virtual dive is different to a real dive, which is the place we go for after work drinks.

You can get these insights delivered straight to your inbox by clicking here.

Meet The SIMS

Screen designs for the Phytoplankton and Coastal Erosion workstations.

Like a lot of Australians over the summer, our thoughts naturally turned to the sun, sea, sand, coastal erosion and the growing threat of ocean acidification. Since mid last year we have been working with the Sydney Institute of Marine Science on their new exhibition, learning about what confronts us and future generations beyond the first line of breakers.

SIMS is a research and training institute based on the shores of Sydney Harbour at picturesque Chowder Bay. SIMS brings together scientists from six universities, as well as a number of state and federal marine agencies, to conduct research on climate change and urbanisation, biological diversity, fisheries, tourism, marine disease and coastal development, which you might call land disease. This year they will be opening their new interpretive centre as an accompaniment to their existing educational programs.

A juvenile box fish. Image courtesy of Emma Birdsey.

We have developed the interactive and video programs for the exhibition in conjunction with exhibition designers Five Spaces Design and Peter Campbell Design to be housed in the old Submarine Mines Store at the water’s edge. The site itself is reason enough to Uber it to Chowder Bay, but then you add in the café and the kookaburras and you have a lovely day out, complete with some fun and a bucket of marine science.

Look out for the interactive workstations and touchtables on Coastal Erosion, Exploring the Sea Floor, Phytoplankton and the East Australian Current. Interspersed with animations, illustrations, videos and high-resolution imagery of the sea floor, each installation is unique and an important showcase for the research that is being developed at SIMS.

One of the best parts of this project has been the opportunity to work directly with the marine scientists. Our teams have worked together to curate the Institute’s extensive content and choose media that complements each story. We hope this new exhibition, in conjunction with the educational programs that run at SIMS, can contribute to more informed debates at a time when the need for greater public understanding of what’s happening in our oceans could not be more acute.

Dive Another Day

A view of a rock platform through the Oculus Rift.

You know those days when you can’t lie in bed any longer and you want to get up and go to work and dive straight into it? And you excitedly rush to your workstation and strap on your Virtual Reality headset and block out the whole world and your meaningless so-called real life? Well, that’s every day round here.

We’ve been developing an Oculus Rift project for the Sydney Institute of Marine Science that is a virtual underwater dive of Sydney Harbour. You put on a headset with lenses and a screen that gives you sense of a 360 degree view as you move your head in any direction. Fascinating and wonderful and it actually does give a different experience than any other screen-based technology. Something happens to your brain where it tells you: you are in a different space. Peter Vergo said that it was a basic human need to immerse yourself in alternate worlds every now and then. It used to be done with stories round a campfire, meditation or alcohol, and now there’s another way.

The Oculus Rift differs from lighter and more portable tech like Google Cardboard in that it needs to be connected by cables to a PC. But having that extra computing power means that you can produce intricate 3D environments and let the user move around in them and interact with objects. Our dive has been developed in Unity 3D, scripted in Javascript/UnityScript, and deployed on Windows 8.1 machines, using the Oculus Rift DK2.

Kelp, a weedy sea dragon, and a blue groper in the distance, viewed through the Oculus Rift.

When we first discussed the project with SIMS, the options were to either create an immersive 360-degree video or make our own 3D environment. The 3D approach was chosen because it meant that we could give visitors control over their movements through the space and we could create a contained environment that showed all the animals we wanted, and frankly it would have been hard to get live bull sharks to perform. SIMS provided a list of creatures that should appear near a rock platform and kelp garden, and we sketched out a plan of the environment and scripted each creature’s behaviour.

Visitors can explore the sea floor, and discover animals such as a blue groper, weedy sea dragon, moray eel, and giant cuttlefish. Corals, sponges and sea urchins encrust a central rock platform, accompanied by a slowly waving kelp forest. Looking up, and they can see their air bubbles float to the surface towards a circling bull shark contemplating its next move.

The Oculus DK2 and the rough joystick/button setup which was made for us by Interactive Controls.

This new iteration of virtual reality technology is only in its initial stages and as a result there are still several issues that need to be addressed. Some users report feeling simulator sickness – a feeling akin to sea sickness. This is mainly caused by the feeling of moving through a virtual world without your physical body actually moving, and can also be triggered by poor frame rates. Interaction methods are also unsettled and the use of these headsets within public exhibition is still being tested.

Over the past few months, the scale and look of the environment has evolved. We’ve constantly tested the application along the way, working on improving the textures, lighting and various set pieces, and adding visual effects such as particle systems, breathing bubbles and caustic patterns. We’ve also worked with 3D modellers and animators to create the various creatures (and other organisms such as sea tulips and sponges) that now populate the ocean floor. Throughout the entire project, SIMS provided detailed feedback about the colours, textures, and precise movements of each creature. It’s been a challenging and important process finding a balance between the visual impact and performance of the application, as it needs to run extremely smoothly in order to provide the best experience.

Progress shots of the dive in various stages of development in Unity 3D.

This has been a great first Oculus project to work on and we see this sort of program and technology working brilliantly well for all sorts of applications where visitors might explore an environment like a convict ship or the surface of Mars or inside a volcano or be projected forward or back in time to have a deeper experience of a different world.