Certification Report on the Canadiana.org Digital Repository

The Center for Research Libraries (CRL) conducted a preservation audit of Canadiana.org (hereafter referred to as “Canadiana”) between January and December 2014, and on the basis of that audit certifies it as a trustworthy digital repository. Specifically, this certification applies to Canadiana’s ability to preserve static digital files in common file formats produced through the digitization of physical materials by Canadiana. Those materials include, specifically, content accessible as of December 2014 through one or more of the following web portals: Early Canadiana Online[2], Héritage[3], Historical Debates of the Parliament of Canada[4] and DFATD Digital Library[5]. It does not apply to born-digital materials and other digital files archived by Canadiana but produced by others.

Moreover, the CRL Certification Advisory Panel expects that in the future, Canadiana will continue to be able to deliver content that is understandable and usable by its designated community.

It is important to understand that, unlike other digital preservation services such as CLOCKSS and Portico, Canadiana is not a digital repository service solely focused on preservation. Instead, Canadiana’s preservation functions are integrated within its platform for managing digital content for a set of publicly accessible digital libraries. The resultant exposure of the repository’s content to users adds a measure of transparency to the Canadiana operations that can be considered an asset.

The present report is based upon review by CRL and the members of its Certification Advisory Panel of extensive documentation gathered by CRL independently from open sources and third parties, as well as data and documentation provided by Canadiana. Information was also obtained through two site visits by CRL audit personnel to the Canadiana offices in Ottawa, Ontario. CRL’s evaluation of Canadiana and the information provided in this report reflect the policies, systems and procedures that were in place at Canadiana to manage deposited content as of December 2014.

The overall conclusion of the Panel was that Canadiana can be recognized by its designated community as a repository capable of managing and preserving the digital content produced through its own digitization program, and content created by others for which Canadiana itself creates the submission information package (SIP). This content is described in Canadiana’s online document, TDR Repository Content, to be found at the time of this writing on the Canadiana website.

About the Audit Participants

Canadiana.org

Canadiana.org is a non-profit, registered charity established in 1978. Canadiana works with other public and private organizations (notably Library and Archives Canada) to present Canada's documentary heritage online. Canadiana supports, facilitates and coordinates initiatives to digitize Canadian historical materials, and maintains and provides electronic access to those materials through a number of online portals.

Canadiana is a membership alliance governed by a volunteer Board of Directors consisting of scholars and representatives of major memory institutions from across Canada. The organization is “pan-Canadian in outlook and governance” and aims to represent the interests of many stakeholder constituencies. Those constituencies include organizations that rely upon Canadiana to maintain and make available digital copies of heritage materials that said organizations hold in physical form; researchers and others who access those digital copies through open access and subscription databases; and the academic and research libraries, and public and government libraries, archives, museums, and other organizations that subscribe to those databases on behalf of users.

The Canadiana “TDR” is one service offered by the Canadiana organization. In addition, Canadiana acts as a publisher for reformatted analog-to-digital content of interest to its subscribers. Digital content is produced and made accessible through their digitization services, and discovery and access platforms. Not all of the content published by Canadiana is included in the Canadiana “TDR,” and such content is not within the scope of this audit.

The purpose of the Canadiana “TDR” is to serve the digital preservation and access needs of its stakeholders. It does this by providing the following services:

1. Preserving digitized content produced, owned by or of interest to its stakeholders; and

2. Providing long-term access to the preserved content to user communities represented by its stakeholders, based on terms of access and funding models agreed to by the stakeholders.

Some TDR content is produced, stored and disseminated by Canadiana on behalf of other organizations, such as materials from Library and Archives Canada digitized by Canadiana and distributed through Early Canadiana Online. Other content is contributed by client organizations like the Short Grass Public Library System (SGPLC), whose The Brooks Bulletin is stored on their behalf.

Center for Research Libraries

The Center for Research Libraries (CRL)[8] is an international consortium of university, college, and independent research libraries. CRL supports advanced research and learning in the humanities, sciences, and social sciences by ensuring the survival and accessibility of source materials vital to those disciplines. To enable its community to accelerate the shift to electronic-only resources in a careful and responsible manner, CRL both maintains shared physical collections of materials and evaluates digital repositories of interest to its community.

CRL analysis of Canadiana documentation and operations was undertaken by Marie Waltz and other CRL staff. Additional technical support for the site visit and the assessment of Canadiana repository systems and architecture was provided by James A. Jacobs, Librarian Emeritus, University of California San Diego. Mr. Jacobs has more than 20 years’ experience working with digital information, digital services, and digital library collections.

To guide this audit CRL enlisted a panel of advisors representing the various sectors of Canadiana’s designated community. The Certification Advisory Panel includes leaders in digital resources, preservation, and digital information technology, and is so constituted as to ensure that the certification process addresses the interests of the Canadiana designated community.

The members of the CRL Canadiana Certification Advisory Panel were:

Perry Willett (Chair)

Digital Preservation Services
Manager

California Digital Library

Mark Jordan

Head of Library Systems

W.A.C. Bennett Library, Simon
Fraser University

Pascal Calarco

Associate University Librarian,
Digital and Discovery Services

University of Waterloo

Tim Au Yeung

Consultant, Digital
Object Repository

University of Calgary

Methodology and Criteria

This assessment was undertaken to determine whether or not Canadiana meets the commitments it has made with regard to the long-term preservation of digital data for its designated community and whether the repository complies with established criteria for trusted digital repositories. The assessment included two site visits, a review of the information independently gathered by CRL from published and unpublished sources, and examination of documents and documentation provided by Canadiana.org.

CRL conducted its audit with reference to:

generally accepted best practices in the management of digital systems

the interests of Canadiana’s designated community

the practices and needs of scholarly researchers in the humanities, sciences, and social sciences in the United States and Canada

the criteria included in Trustworthy Repositories Audit & Certification: Criteria and Checklist[1]

the criteria included in Audit and Certification of Trustworthy Digital Repositories (TDR) checklist (ISO standard 16363)[2]

the Open Archive Information System reference model[3] (OAIS)

other metrics developed by CRL through its analyses of digital repositories.

The primary metrics used by CRL are those specified in the Trustworthy Repositories Audit and Certification (TRAC) checklist. TRAC was developed by a joint task force created by the Research Libraries Group (RLG) and the National Archives and Records Administration (NARA) in 2003 to develop criteria for use in identifying digital repositories capable of reliably storing, migrating, and providing long-term access to digital collections. TRAC represents best current practice and thinking about the organizational and technical infrastructure required for a digital repository to be considered trustworthy and thus worthy of investment by the research and research library communities.

CRL assessed Canadiana on each of the three categories of criteria specified in TRAC and has assigned a level of certification for each. The numeric rating (below) is based on a scale of 1 to 5, with 5 being the highest level, and 1 being the minimum certifiable level. The minimal certification rating of 1 is assigned in instances where a repository has inconsistencies or deficiencies in areas that might lead to minor defects of a systemic or pervasive nature, but where no major flaws are evident.

TRAC Category

Canadiana.org rating

Optimum rating

Organizational Infrastructure

4

5

Digital Object Management

3

5

Technologies, Technical
Infrastructure, Security

4

5

TOTAL

11

15

The basis for assignment of these ratings is provided in Section B, Detailed Audit Findings, below.

Again, it should be noted that CRL certification of Canadiana applies specifically to the repository’s ability to preserve and manage digital content created by Canadiana itself and content created by others for whom submission information packages (SIPs) are created by Canadiana.org. CRL did not assess Canadiana procedures and processes for migrating or otherwise altering and preparing digital content for distribution via future platforms or devices.

Detailed Audit Findings

The Panel believed that Canadiana should increase its support for professional development of critical repository staff in an effort to improve digital preservation decision-making (TRAC A2.3). In the course of the audit, in discussions with staff regarding practices related to ingest of content, it was evident that some key members of Canadiana staff were not always acquainted with certain digital preservation technologies and practices. To ensure that Canadiana implements up to date digital preservation strategies CRL encourages Canadiana to provide staff more opportunities to interact with others in the digital preservation field. Such activities as working with other digital repositories, attending conferences focused on digital preservation and investigating new preservation approaches and technologies will enhance the organization’s ability to fulfill its commitment to digital preservation.

CRL’s audit noted that in the future Canadiana will, no doubt, be expected to begin to accept and archive born-digital content and digitized content prepared for archiving by other organizations. This service will be a natural expectation of constituencies in the world of academic research that Canadiana serves. At the time of the CRL audit, most of the digital content archived by Canadiana was material created by Canadiana itself, through its own digitization of works on paper and print materials of cultural and historical importance. The processes for ingesting and maintaining that content were determined by CRL to be basically sound. Canadiana has more recently begun to put in place the technologies and processes necessary to manage and preserve born-digital content, but its ingest of digital files produced by other organizations involved processes for verification of file types and format validation that were at the time of the audit still untested and unproven. CRL encourages Canadiana to continue to develop and document this capability.

Ongoing Requirements

The TRAC document notes that “. . . attaining trusted status is not a one-time accomplishment—achieved and forgotten. To retain trusted status, a repository will need to undertake a regular cycle of audit and/or certification.” To that end CRL expects that Canadiana will also make certain disclosures on a regular basis. CRL and Canadiana have agreed that ongoing certification is contingent upon Canadiana making the following disclosures every two years:

A detailed listing of new content and its producers added to the repository since certification.

Disclosure of results of quality control testing on ingested content.

Description of any significant changes in repository system architecture or configuration, operating systems and/or critical software.

New agreements and contracts with key depositors of content, content users, major funders or sources of revenue, and providers of critical repository services.

New key policies regarding acquisition, management, and disposition of archived content and related files and metadata.

Records of significant events (such as content migrations, system failures, loss or corruption of digital content) and significant changes in the characteristics of digital content ingested since the most recent audit; and of significant events and changes in the operations of the repository.

The most recent three years of financial statements for the repository organization. The financial statements should indicate the categories and, where appropriate, sources of revenue and the level of same; the functional allocation of expenses; and changes in the financial position of the organization.

Revenue and expense projections by function, for the repository organization for the next three years.

Certification is also contingent upon Canadiana agreement to a periodic, systematic sampling and/or inspection of the repository’s archived content by CRL, or by a third party designated by CRL and Canadiana jointly, using either a manual or an automated process, as determined by mutual agreement between CRL and Canadiana.org.