Feeling the weight of the Obama administration's directive to deploy cloud computing where possible, federal agencies have begun migrating their email systems to commercial cloud providers. But agencies haven't always given adequate thought to electronic recordkeeping requirements. That may lead to legal problems down the road, say former and current officials at the National Archives and Records Administration, who maintain agencies need to integrate e-recordkeeping requirements into a cloud migration from the start.

"It doesn't surprise me that the issue of recordkeeping doesn't come up much in discussions about going to the cloud," said Jason R. Baron, a lawyer at the Washington law firm Drinker Biddle and former director of litigation at NARA. "When people think about the cloud, the first issues that come to mind are security and privacy. Of course, those are extremely important, but from an information governance perspective, one needs a more holistic picture."

The administration's "cloud-first" policy intensifies the challenges for agencies because managers are also facing a series of deadlines to manage electronic email records more thoroughly.

An August 2012 memorandum issued by former Office of Management and Budget director Jeffrey Zients, along with US Archivist David Ferriero, requires agencies to manage both permanent and temporary email records in an electronic format by the end of 2016. And by December 31, 2019, agencies must manage all permanent records in an e-format for eventual transfer to NARA, which collects and preserves US government records.

Agencies that aren't spending enough time on electronic records requirements at the beginning of the process of migrating to the cloud for email may have difficulty meeting these deadlines, said Baron.

"Otherwise you're building what amounts to a slow-motion train wreck, where you've got this cloud and you've got a million emails somewhere in there and that's all very good. But at the end of the day, when the agency wants [to forward email records] to NARA, it may not have deleted any email or differentiated between what's permanent and what's temporary. You need to think about these issues on the front end."

A graduate student reviews microfilm records at the National Archives and Records Administration (from NARA video).

In order to ease the burden agencies face managing email records, especially as more of those records move to the cloud, NARA last August issued guidance to the heads of all federal agencies about "Capstone," a simplified approach to managing email records. Under Capstone, agencies can capture and preserve as "permanent" entire email accounts -- generally those of senior officials -- for eventual transfer to NARA, instead of requiring staff to file email records individually. The remaining email accounts in the agency can be scheduled as "temporary" and need to be preserved only for a set period of time.

In effect, the Capstone approach makes it less onerous for agencies to meet the requirements of the August 2012 mandate. Moreover, reducing disposition to a few categories makes it easier to deploy automated methods of managing email accounts, NARA said.

The transformation NARA hopes to achieve calls for taking "human intervention out of these processes (as much as possible), because as a practical matter, having people look at one email and deciding which folder to put it in is not going to work," said Paul Wester, chief records officer for the US government at NARA.

Agencies seeking a best-practices model for integrating electronic records requirements into a cloud-based email migration need look no further than NARA itself. Last year, NARA successfully moved more than 3,000 of its users from GroupWise to Google Apps for Government cloud-based email and collaborate suite. While the migration took only six months, officials spent several years carefully planning it.

Between 2009 and 2011 NARA officials conducted market research on how to accomplish the transition, developed a business case, and established requirements, said Susan Sullivan, corporate records manager in the office of NARA's chief operating officer. The agency awarded a contract in September 2012 to Unisys to lead the project.

"What NARA did, which is different from what anyone else has done as successfully, was to bake in records management as part of the cloud procurement," Baron said.

Early in the contract, NARA officials formed a records management integrated project team (IPT) comprising Sullivan, Wester's group, the CIO's office, agency attorneys, and the contractors. The records management IPT "became the driver of all the work behind moving forward with this," Sullivan said. "We met weekly. Once we understood how the records management application [ZL Technologies' e-records management software] works with Google Apps, we configured the application to capture email automatically."

She added that NARA's automated file plan, based on Capstone, specified only three folders -- permanent, temporary, and nonrecord -- as opposed to the "typical hierarchical file plan" with an abundance of dispositions.

NARA officials are keen to share lessons-learned with other agencies looking to move to the cloud for email, according to Wester. "We've been letting agencies look under the hood at what we're doing so they can understand our thinking," he explained, "not only about the requirements and mechanics of it all but also about how they can [acquire] enterprise-wide solutions that meet their business needs."

Moving email to the cloud has lowered IT costs and improved efficiency. Find out what federal agencies can learn from early adopters. Also in the The Great Email Migration issue of InformationWeek Government: Lessons from a successful government data site. (Free registration required.)

Richard W. Walker is a freelance writer based in the Washington, D.C., area who has been covering issues and trends in government technology for more than 15 years. View Full Bio

Thanks for noting, Charlie. NARA's IT leaders insist the contextual scanning tools available today, that help attorneys scan and catalog 10s and 100s of 1000s of documents have gotten good enough lately to tackle the job. But setting the rules up correctly remains a fine art. If there's anyone to watch, its NARA.

It tooks an integrated project team three years to come up with a way to designate Google email as tempory, permanent or nonrecord and build an automated way of acting on those designations? Well, it hasn't been done before, I'll give them that. And if it's automated, that finally removes the human trying to classify and capture keywords from millions of emails being moved into archival folders. Many agencies and businesses should be following the NARA example, except maybe for the three years.

Most major migrations, whether from legacy to cloud or just from one toolset to another may face issues about what to do with historical data that can't be tossed but would be prohibitively costly to migrate/translate into the new venue. Companies need to think about this when undertaking a major project like a move to cloud-based email or even from tape backup to cloud backup. Sometimes it means keeping the legacy tool around for a while until those old records expire.

In this report, we'll examine the use of cloud services by government IT, including the requirements, executive initiatives and service qualifications, and auditing and procurement programs that make government cloud adoption unlike that in the private sector.