When your core business is providing technology solutions, making an appropriate hardware and software buying decision to enable that takes on additional meaning. Xand Corp., an application infrastructure and managed service provider, responded to growing business needs by reevaluating its storage needs.

As business grew, it quickly became apparent to Xand that managing controllers in all its distributed storage was becoming increasingly problematic. The company's RAID system used PCI controllers in the individual subsystems. That's fine in systems with only a few machines, but with many hundreds of servers, as Xand has, it can become a management nightmare. The company also realized there was a lot of wasted storage space in all these systems.

Joe Fuccillo, senior vice president for Xand, says the firm began looking at SAN technology a few years ago. Because Xand wanted to offer a higher service level to clients while easing its own storage management challenges, it watched as the market and technology matured. In addition to proving to be a good solution for the company, Xand's use of SANs, Fuccillo says, "was also a win-win situation for our customers, who could connect their systems to storage they wouldn't have been able to individually afford because of the high cost of SAN technology. It becomes quite economical, though, when you divide it up on a per-gigabyte basis."

The Xand SAN Wish ListInteroperability, standards and native support were some of the signs of maturity Xand was looking for in a SAN solution. In a business where the buck stops at your site, you want the server's operating system vendor to have blessed the technology as a certified solution that works. "If there's an uptime issue, the first thing the client points to is the SAN, so we wanted vendor certification," Fuccillo says, "and we wanted interoperability with the Fibre Channel HBA [Host Bus Adapter], the switch fabric vendors."

Vendor certifications for clustering were also a requirement because customers who opt for this caliber of solution typically require such support. Xand needed multi-platform support as well to support and manage its endorsement of Windows NT/2000, Sun Solaris, Linux, the Intel hardware platform, and IBM AIX for the RS/6000 platforms.

Another Xand requirement was the ability to boot off the SAN—a more recent development in SAN technology. Unlike having a boot drive on a RAID, which is a point of failure in the system, if a drive fails on a storage SAN, you can have complete redundancy throughout the system. "The only thing we might use a local disk for is swapping," Fuccillo says. "That, and we can always do a redundant swap on the SAN, which we typically configure. That way, in case the local swap fails, we don't have a system failure."

From a software perspective, Xand wanted to be able to do point-in-time copies of data, and remote peer-to-peer data replication over IP as a transport protocol. The company didn't want a solution that required ESCON, which would necessitate its own dedicated link in order to expedite facility-to-facility replication. (ESCON stands for Enterprise Systems CONnection—an IBM S/390 fiber optic channel.) This was in part a financial decision. Xand also has space that it operates at other facilities that are transporting IP, and knew, based on customer feedback, that it would need to have another SAN in such a facility to replicate at some point in the near future.

Xand already offers customers the capability of replicating data across an IP network and is looking into making storage available through some of the new iSCSI standards. "Customers could take data on their facility, replicate it to ours and have it here for disaster recovery," Fuccillo says. "Loaner servers could be set up to map to the data that's stored on the SAN to [make] them functional. Everything we do is based around IP as the transport—that's something we really know and that fits well in our network."

Ingredients for a Xand SAN

Joe Fuccillo, senior vice president for Xand, had these comments about the SAN system from IBM that Xand implemented:

"We're running the IBM ESS [Enterprise Storage Server] with full redundancy and 11 terabytes of storage. For the SAN itself, we actually use a mix of Brocade and McData switches that make up the fabric. They're very similar products, and we've been happy with both, so we haven't yet made a decision as to which one we're going to standardize on."

"On the systems' servers, for the Fibre Channel HBA adapters, we've been using Emulex Corp.'s—mainly because it was one of the few certified vendors that allow you to run the same adapter on PCI for Solaris, Windows 2000 and Linux. From a spare-end perspective, the fewer SKUs I need to spare and stock, the better off we are. We've been using the Emulex cards since October, and they've been working well.

"System-wise, we have Windows 2000, NT and Linux boxes, and a couple of Solaris boxes on the SAN as well. So, we have a good open systems environment. We're also using StorageTek tape libraries."

—S.J.B.

The Search Is OnIn June 2000, Xand began a thorough, nine-month evaluation of vendors and technologies, looking at feature sets and running though exhaustive trials. Originally, the company considered a list that included Hitachi Ltd. and Compaq Corp., but it quickly narrowed the field to EMC Corp. and IBM Corp. From a feature standpoint, both companies had the checkmarks, including vendor certifications. They also offered everything Xand wanted in terms of software and performed well in tests. Both technologies had good caching algorithms and were reliable in terms of how the data integrity was maintained in power outages.

However, when Xand got to the planning phases with EMC tests, it discovered that there were certain changes Xand wouldn't be able to make internally. For example, once an EMC BIN configuration file was set up, reconfiguration had to be done by the vendor, preventing internal configuration and management. The IBM ESS, in contrast, didn't require BIN file changes, according to Fuccillo. The inability to change the file internally "was unacceptable for us as a service provider," Fuccillo says, "[Since we're] 'the one neck to choke,' we have to be in control of our own destiny. Luckily, we have some really technically savvy people, sharp Unix system administrators, who were able to catch that."

When the dust settled, there were three winning factors for IBM's Enterprise Storage Server (ESS, code-named "Shark" by IBM):

Management flexibility—the ability to make any change and reconfigure storage on the fly without the need for vendor intervention.

Scalability and upgrade flexibility—upgrades can be done seamlessly without having to remove and replace entire systems or equipment. Because ESS more easily adapts to the service provider model, Xand can better respond to customers' changing needs.

A cost-effective solution at a lower price per gigabyte that would enable broader customer adoption.

Things to Consider in Selecting a SAN

Joe Fuccillo suggests these factors to consider in selecting a SAN:

"You really have to sift through all the FUD [fear, uncertainty and doubt] that the storage vendors throw out about each other, to see what's real and what's not. For this, you need technically savvy people to ferret out the truth. And, of course, you need to thoroughly understand your environment and what you want to do.

"Definitely look at a vendor that's into open standards. You want to ensure that, as new technologies evolve, you'll be able to use what you already have. For example, with IBM being part of the iSCSI Foundation, we know that we'll be able to put an iSCSI gateway on our SAN and have our SAN storage available over iSCSI in the future.

"Be sure to consider the future as well in making a purchasing decision. Every one of our clients on the SAN has requested additional space since being on it. And it's definitely easier to add [space] in the Unix world than in Windows, which reflects a limitation on how Windows does volume management."

—S.J.B.

SAN LimitationsYou may never reach the storage limits of your SAN, but what about the limitations of the SAN unit itself? There's a "SAN-out" ratio that limits how many servers you can put onto a unit before the unit itself becomes a bottleneck. That is, you may reach a processing limit, at which point the unit simply can't deliver enough gigabytes per second to the servers talking to it. As a result, you must constantly watch your performance levels and may need to add another unit long before the storage reaches its upper limit.

One of the things Xand liked about IBM ESS was the fact that the components inside are RS/6000-based. This means Xand will be able to upgrade to 2 GHz processors, while keeping the same infrastructure, when higher speed processors come out for the RS/6000 next year. In contrast, vendors with proprietary line cards need an upgrade to the entire unit.

Xand has taken the features of the SAN, using IBM's Enterprise Storage Server, to drive revenue and show value to its customers from a service perspective.

As Fuccillo notes, "The common thing with our customers, whether small- to mid-sized or Fortune 500, is that they don't want to focus on technology. They're trusting us to run a portion of their infrastructure for them—and they only want to deal with one vendor."