Recently, members of the Rutgers University Libraries at both integrated Information Systems and the Scholarly Communication Center began an auspicious collaboration with the Asia Society in New York City, in our first project to digitally preserve, to standards, their digital video archive for The China Boom Project. It is the first time that RUcore has ingested a fully born-digital video archive, using the original source content and project files, and creating presentation video from those source files.

The China Boom Project’s goal is to seek an answer to the question, “Why did China Boom?” The site comprises taped interviews with individuals and experts with insights into China’s rapid economic expansion in recent decades. It offers to site visitors packaged video content from these interviews arranged by subject matter and relevant time periods in China’s history, in a very effective and attractive format that is described as a “mosaic explanation.”

But while the China Boom site itself provides snippets and prepackaged commentary, an ancillary goal of the project has been to partner with educational institutions to make the full-length content available to researchers, and to have the video archived and preserved. This is where Rutgers University Libraries, and RUcore, come into the picture.

1/2 inch, consumer, open reel video format used in the 1960s-70s. This format suffers from Sticky Shed Syndrome, making playback difficult. Working machines for this long-obsolete format are scarce. Source: IUB Media Preservation Initiative, used with permission. Note: IUB has asked me to stress that the above photo is not representative of all media collections at the university.

Home to at least 3 million media objects, including sound and moving image recordings, photos, documents, and artifacts, IUB has come to grips with the issue at hand: a great deal of their heritage is locked within obsolete electronic and analog playback formats for sound and moving images. As an old format becomes obsolete, it gets harder by the day to find working equipment to play back these objects. And that assumes that the objects can be played back, not having succumbed to age, wear and physical decay. Lacquer from old aluminum audio discs can delaminate, making them unplayable. Video tapes from the 1960s, 70s and 80s can suffer from a condition where the binding agent that holds the recording material to the plastic base sheds, allowing audio and video recordings to literally flake into nothingness. Film, too, has its own serious decay problems.

With its vast collection, faculty and staff at IUB knew the situation could become serious if nothing was done. Their first step was to take stock of the situation, and consult outside experts (myself included) to get input how how best to address the problem.

Their efforts began nearly 18 months ago when a group of IUB faculty and staff, concerned about the potential fate of important special collections on campus, approached their Office of the Vice Provost for Research about the critical issues of media, and to impress upon them that time was of the essence to address these issues.

“Even though [IU Bloomington’s] needs are now documented, and it is far better equipped than most universities in the country to meet them, there is no guarantee that IU can adequately preserve its collections in the near future.”

The culmination of their efforts to date have been documented on the IUB Media Preservation Website, where they document their comprehensive effort to preserve IUB’s vast audio, video, and film holdings. Some important documents from their study and efforts including IUB’s Director of Media Preservation Services Mike Casey‘s Media Preservation Survey (PDF), outlining the collection holders, preservation stakeholders, the risks involved, and potential preservation strategies. A follow-on public report (12MB PDF) also lays out the situation and what steps are being taken to save their special collections and historic content. Continued engagement, updates, and discussion on decisions made and procedures undertaken are regularly made available on their Media Preservation Blog.

The IUB Media Preservation Blog

IUB has many years of work ahead of it, not only to transfer older content into more modern digital formats, but also to continue to maintain those archives, preserve new content, and keep pace with new technologies and formats to ensure that their collections are accessible. It’s encouraging to see them in action, and their efforts stand as a potential framework for other organizations in a similar bind to model their initiatives after.

On July 7, Sony announced that production of MiniDisc playback equipment would cease in September of 2011. According to Sony, the format’s creator, the blank MiniDisc recording media will continue to be manufactured for up a year beyond the players’ discontinuation.

MiniDisc never made as big a splash as Sony had hoped, at least in markets outside of Asia. Introduced in 1992, Sony had envisioned that the format would be just as ubiquitous in the 1990s as the audio cassette – and another Sony invention, the Walkman – was in the 1980s. Unlike Audio CDs, MiniDiscs offered a more compact design to increase portability, greater durability and anti-skip capabilities, and all MiniDisc playback equipment was capable of writing to recordable and re-writeable media from the outset. By contrast, the first sub-$10,000 CD writers wouldn’t become available until 13 years after Compact Disc’s 1982 introduction to the market, and almost 3 years after MiniDisc was widely available.

Unfortunately, MiniDisc had barriers to adoption from the outset, most of which were placed – deliberately or otherwise – by the company who introduced the format in the first place.

One of the topics I like to bring up in the discussion of preserving digital data is the idea of a Digital Dark Age… the notion of a period in our historic knowledge that ends up getting lost due to a failure to plan and preserve our early digital content.

The New Scientist, however, recently published an article (Feb 2, 2010) on something a bit more cataclismic: the concept of Digital Doomsday. From the article:

Suppose, for instance, that the global financial system collapses, or a new virus kills most of the world’s population, or a solar storm destroys the power grid in North America. Or suppose there is a slow decline as soaring energy costs and worsening environmental disasters take their toll. The increasing complexity and interdependency of society is making civilisation ever morevulnerable to such events (New Scientist, 5 April 2008, p 28 and p 32).

Whatever the cause, if the power was cut off to the banks of computers that now store much of humanity’s knowledge, and people stopped looking after them and the buildings housing them, and factories ceased to churn out new chips and drives, how long would all our knowledge survive? How much would the survivors of such a disaster be able to retrieve decades or centuries hence?

The article is a compelling read, and offers an intellectual exercise on how much of our “stuff” will survive such a castastrophe. Ironically, the logic is that the digital content with the most copies oin existence may win out. So, while scholarly works, theses, research and other important scientific data would be at risk, pop music may surive just fine.

The facility I work in at Rutgers, known as the Scholarly Communication Center (SCC), has a fairly short history in the grand scheme of academia, and yet a fairly long one when it comes to the rapid changes in technology it has seen in its lifetime. It was originally started in the 1996, and meant to be a location for university students and faculty to access a growing body of the then-nascent collection of digital content.

Back then, the internet still wasn’t very fast and wasn’t nearly as media-rich as it it seems today. And so, most of the data-heavy reference materials arriving in digital form came to the SCC as CD-ROMs (and later, DVD format). To accommodate this, the SCC had a lab of ten desktop computers (known as the Datacenter), dedicated solely to accessing this type of material.

But the times changed, and so did the way people accessed digital material. As the ‘net grew in size and capacity, it no longer made sense to ship reference material on disc, and so the access moved online. Students migrated from visiting computer labs to bringing their own laptops (and later, netbooks and handheld mobile devices). Traffic at the datacenter dropped to virtually nothing. The space had to be re-tooled to continue to be relevant and useful.

And so, with my taking on the newly-minted role of Digital Data Curator, and in collaboration with my colleagues, a new plan for the former datacenter was developed. Instead of being a place to merely access content, we would be a place to create it. Analog items that needed to be digitized would be assessed and handled here. New born-digital content would be edited, packaged, and prepared for permanent digital archiving in our repository. We would be a laboratory where students getting into the field – and even faculty and staff who have been here a good while – would learn, hands-on, how to triage and care for items of historical significance, both digital and analog, and prepare them for online access.

The concept for a new facility was born. And we call it the Digital Curation Research Center.

The center is still in “beta,” as we plug along with some internal projects for testing purposes along with a couple of willing test subjects within the university and surrounding community. This is so we can test out the workflow of the space and make tweaks and optimizations as needed. Our plan is to officially launch the space in the Spring of 2010, with a series of workshops and how-to sessions for the various things that make digital curation vital (e.g. digital photography, video editing, audio and podcasting, and scanning).

The plan is that this will be a continual, evolving learning experience for all involved. People who have never really used cameras and recording equipment in a historical context will learn just how increasingly valuable the content they create, and the stories it will tell, can become over time. And those of us in the DCRC day in and day out will encounter things that we’ve never run into before, and will have to wrap our heads around the issue of preserving it effectively.

Below are related documents that provide additional information about the DCRC. More information will be coming up as we get closer to the official launch: