EMC Corp. today announced delivery of its fully automated storage tiering (FAST) technology across its Symmetrix, Clariion and Celerra line of storage arrays. The technology will allow data volumes to be dynamically moved between tiers of storage, depending on business performance requirements.

The first phase of EMC FAST technology is available immediately for new and existing EMC Symmetrix V-Max and EMC Clariion CX4 networked storage systems, as well as EMC Celerra NS unified storage systems. The technology will identify data sets at the volume level only and, on average, will allow sets down to a gigabyte to be automatically moved between storage tiers.

The company also laid out its strategic direction for other advances in automated tiering technology, which will eventually allow its software to identify data sets smaller than a megabyte in size and move them to the most appropriate level of data storage - be it solid state drives (SSD), Fibre Channel or SATA.

The software allows users to combine the benefits of enterprise flash drives, which can increase application performance by as much as eight times for active data, with the cost benefits of high-capacity SATA disk drives, which can lower the cost-per-megabyte by as much as 80% for inactive data, EMC said.

Brian Gallagher, senior vice president and general manager of EMC's Symmetrix division, said the introduction of FAST technology is an "inflection point" for the company and its customers. "If we don't do anything about storage growth, the future looks pretty bad," he said. "Data capacity growth is doubling every 18 months" and growing 10-fold every five years.

EMC has been espousing the virtues of SSD for nearly two years, vowing to do all it could to help drive the price of flash technology down because it has the potential to revolutionize the way drive technology is deployed. The automated movement of data volumes between tiers will spur the adoption of SSD, Gallagher said, because enterprises will be able to take full advantage of NAND flash performance without having to manually allocate data to it.

Today, many enterprises requiring high I/O performance for applications, such as relational databases for online transaction processing, deploy dozens of high-speed Fibre Channel drives using a technique called "short stroking." That's where only the outer sectors of a disk's platter -- those closest to the read-write head -- are accessed in order to accelerate performance. The method is both costly and wastes vast amounts of drive capacity. Even using short stroking, hard drives only achieve a fraction of the I/O performance of SSDs.

"You can drive 5,000 I/Os per second with an SSD. On a good day, a hard drive will give you maybe 200 I/Os per second," Gallagher said. Having SSDs in the mix will eliminate the need for short stroking, he noted.

Brian Bosserman, network and systems operations manager at Foster Pepper, a large law firm in the pacific northwest, said he wants to deploy FAST technology on his Celerra systems to maximize capacity use by placing infrequently accessed large document files on SATA drives. Exchange and SQL data would go on enterprise flash drives and Fibre Channel drives, all within the same system.

"This will also enable us to have a single business continuity strategy with its own levels of automation and ability to meet strict recovery point objectives," he said in a statement.

Bob Laliberte, a senior analyst with Enterprise Strategy Group Inc. in Milford, Mass., said FAST technology will be "very significant" for EMC customers, who make up a large portion of the enterprise storage market. "FAST is focused on delivering optimized application performance while enabling greater cost and energy efficiencies," he said.

According to Laliberte, typically 80% of an application's data is dormant after 90 days, so being able to automatically have it moved from high-end Fibre Channel disk to more cost-effective and energy-efficient SATA drives reduces costs. "Of the other 20% [of frequently accessed data], take 25% of that and put it on flash, to guarantee performance and better energy efficiency and leave the rest on Fibre Channel drives," he said. "While Flash is still expensive now, the cost should be offset by the SATA, and pricing should decline as volume increases."

EMC's Gallagher said FAST is the embodiment of information lifecycle management (ILM), or how to classify data in order to move it to the most appropriate tier of storage through policies - a vision that never came to fruition. According to Gallagher, EMC customers just didn't have the time or staff to dedicate to classifying data types.

"We built this for where the virtual data center is going," he said. "When we virtualize servers and applications, the changes in a virtual environment are too dynamic. A server could be 30% utilized and the next minute, 70% [utilized]," Gallagher said. "The human reaction time is not enough to keep up with that load."

Gallagher compared FAST technology to automation in jet fighter planes, where an aircraft's speed dictates the need for computer control over the aircraft to avoid human error.

"The automation we've developed in FAST gives them what they wanted with ILM, but we took the pain away," he said. "They don't need to classify data or set policy types within data structures."

For example, an administrator can set up a production policy that uses SSD and Fibre Channel, and then Oracle and Exchange can be mapped to those policies. Or an administrator can set up a development policy that dictates only SATA drives be used for storage capacity.

Although FAST technology will initially be aimed at volume level data movement, there are more efficiencies to be gained, according to Laliberte. For example, FAST can create multiple tiers of storage for a single application.

"I think the key is to just get started, regardless of what level of detail, as it often takes time for large organizations to adopt new technology - especially automation technology," Laliberte said.

While it can be completely automated, the FAST software also allows storage administrators to work in a "user approval mode," where an admin is notified through a wizard before data is actually migrated to other drives. That way, users can get accustomed to using the FAST technology before implementing it in full auto mode, "which I believe will be important to driving adoption of this technology at the enterprise level," Laliberte said.

Another tool FAST offers administrators is when they want it to look at data activity for tiering purposes. For example, an administrator could tell his Symmetrix array to look at all data activity for tiering purposes only on Monday through Friday, 8 a.m. to 5 p.m., on weekends, or 24 hours a day.

EMC said its FAST roadmap involves combining capabilities such as sub-LUN tiering; capacity allocation on demand; block and file level deduplication; data compression; disk drive spin down; built-in archiving; and private and public cloud federation. The goal is to provide greater levels of automation, management, and cost savings. The first leg of that roadmap will be sub-LUN tiering, which is due in the second half of 2010. It will enable FAST to pick out data hot spots within a volume for movement to flash drives.

Other key capabilities of FAST include:

Built-in software to monitor, analyze and respond to changes in the value and access patterns of the data to self-optimize storage resources. That allows applications to leverage the ultra-high performance of enterprise flash drives with the cost, energy and footprint efficiencies of high-capacity SATA drives.

Flexible policies that enable users to define tiering rules to support service-level requirements automatically, or with administrator approval.

File system capabilities that permit integration with internal private clouds and external cloud service providers to seamlessly federate placement and movement of file-based and unstructured data.

Management capabilities that automate the end-to-end discovery and reporting of applications to storage tiers to easily view storage resource topologies and attributes; capacity allocation and free space; and real-time and historical performance metrics that enable "pay for use" and "charge back" models.