Data in transit. Those three words are at the heart of business in the 21st century and the rise the of the secure managed file transfer (MFT) industry. Companies function by sending, receiving and sharing information, often in very large files, and often in huge numbers of files in batch transactions. Files have to move quickly, reliably and securely.

Enterprises have to deal with file-transfer methods that don't scale to hundreds or thousands of partners, customers and suppliers doing business across the Internet. Knowledge workers and managers can't get their work done because their email systems kick back files over 5MB. Companies can't ramp up new initiatives quickly because of the time and complexity required to provision employees and partners with a file-transfer system everyone can use.

Data security, compliance and data governance suffer because file transfer is fragmented among one-off deployments using FTP and custom scripts; individuals send sensitive corporate information via email, if their system supports it, or find their own solutions, such as cheap cloud services.

Managed file-transfer has become big business—half a billion dollars and growing, according to Gartner, which tracks more than 40 MFT companies. Those tracked include MFT pureplays, specialists in ad hoc person-to-person transfers, and broad-based vendors that include MFT as a component of full B2B integration services—middleware, infrastructure, application and system integration.

In this Toolbox, we'll explain the issues that affect file transfer, how MFT addresses these issues and what companies need to know as they select, implement and manage the technology.

The Stuff of Migraines

MFT is one of those technologies that gets sold as a business enabler and data-protection and compliance tool. You can automate and centralize both massive and ad hoc data-movement processes, give employees an easy way to exchange files, improve audit oversight and embed security controls such as policy-driven encryption and access control.

In the absence of MFT tools, large-scale file-transfer operations are generally based on FTP servers and some automated scripts. They are highly distributed, with each transfer system having grown organically with every new set of partners and business initiatives. These approaches often scale poorly, as each FTP server supports perhaps 20-50 connections when hundreds or even thousands may be required. That also means there is no way to off-load transfers when demand exceeds capacity. As a result, they are very expensive in terms of infrastructure and support, and highly labor-intensive to manage, as they lack any sort of central management capabilities.

"One of key objectives from senior management was that we need to automate a lot of processes," says Mike Shrader, network security specialist for Glatfelter Insurance Group, a Cyber-Ark customer. "We can leverage automation processes, which increases productivity. We can schedule overnight processes."

Without APIs or native support for file-transfer programs, it takes substantial time, resources and expertise to integrate them with back-end systems and applications, which are often mainframe-based in financial services and other mature verticals.

In the absence of central control and any kind of automated workflow, it is difficult to administer uniform policies across all these systems. Monitoring, reporting, audit and policy enforcement is, at best, problematic and in many cases, impossible. From an operational and business perspective, it is difficult to verify that transfers were completed on time or at all, which creates problems with meeting service-level agreements (SLAs) and proving that the operation went smoothly, at least on your end. MFT products aim to make that a lot easier.

"If you have SLAs, you can track and monitor and send business partners reports to show that you are within your contract," says Thomas Skybakmoen, Gartner senior research analyst. "With scripts and FTP, all reports have to be done manually."

Much of the data should be encrypted both while it is at rest, using strong algorithms, and when in transit, using SSL or SSH. The decision to encrypt is not trivial. Without a commercial MFT product, managing SSL certification and encryption keys can be a major burden.

"Managed file-transfer takes it to another level for encryption for data both in transit and at rest," says Skybakmoen. "Some offer PKI, key management, tokenization; some offer encryption up to 512 or 1024 bits. Managed file-transfer has tools, [which are] lacking in scripts and FTP, that make integration easier to manage."

Key management alone was sufficient in the past to derail encryption products because they were difficult and resource-intensive to implement, and because they raised concerns over whether data could be handled securely and accessed quickly. Again, lack of central management means that encryption has to be administered separately for each transfer system.

"Even if we encrypted, we had to deal with the complications of passing keys back and forth," says Chuck DePalma, CISO for Volkswagen Group of America, an Axway customer. "Making sure things worked was very much a mess. We wanted something automated, secure and clean."

Access controls and data-governance policies based on data and transaction sensitivity are also difficult to manage and enforce on individual systems because of poor administrative tools, little or no audit and reporting capabilities, and no facility for monitoring or oversight.

Ad Hoc Transfers: Users Gone Wild

Individual, human-to-human ad hoc file transfer in particular spawns its own set of security and compliance issues. In the absence of managed file transfer, end users commonly rely on email as their primary means of file exchange. Companies have no native way of enforcing data policies: what information can be sent via email, who can receive it (e.g., only authorized internal personnel, select partners, lawyers), and whether it should be encrypted. There is typically no good audit trail to track what was sent, who sent it and to whom. Organizations have to implement point solutions, including data loss prevention and email encryption to enable and enforce policy-driven email security.

Moreover, email doesn't address the need for a centrally managed file-transfer system that provides these security and compliance capabilities and enables employees to do their jobs efficiently. Employees need an easy way to send and receive large presentations, video and audio files, CAD files, and so on, without shipping CDs, DVDs and USB drives. If they have to move large files regularly, they may go outside the organization, using webmail or some inexpensive and unprotected software-as-a-service application that requires only a credit card number. The employees are just trying to do their jobs, but in doing so, they move out of reach of corporate oversight.

The aim of ad hoc MFT is to provide an easy, transparent transfer mechanism for end users, and to bring individual file transfer under corporate control.

Choosing MFT: 10 criteria

Some MFT products are more focused on large-enterprise file transfer, emphasizing centralizing and automating heretofore clumsy and costly operations first, with security an important second. Others focus on human-to-human file transfers, and some offer suites that attempt to serve all use cases. Companies should select products and vendors that not only solve their immediate problems, but can also serve their long-term file-transfer requirements.

Consider these criteria when shopping for managed file-transfer software:

Automation. Can the product transact large and small scheduled transfers based on workflow, operational rules and corporate policy? Can it execute authorized on-demand transfers automatically, without additional human intervention?

Scalability. The product should be able to handle enormous workloads consistently across a distributed, extended enterprise, its partners and customers. This includes enterprise network features such as high availability and load balancing. It should be able to accommodate spikes in load (without increasing capacity) and an effectively unlimited number of users. Determine if the price includes per-user costs.

Easy provisioning. You should be able to easily enable new business initiatives by quickly bringing on back-end systems, applications and data sources to the MFT product and enrolling new outside transfer entities without one-off customizations or out-of-band activity. On an individual level, it should be easy to enroll new users with appropriate authorization and controls and automatically de-provision them when they leave the company.

Access control and authentication. Look for strong authentication and granular, role-based access-control capabilities to assure that only people with appropriate authorization can send and receive sensitive files. The product should enable clear separation of duties so that operational personnel can manage transfers but not see sensitive data, and partners and their personnel only have access to files that are relevant to them. Support for third-party authentication, such as one-time passwords, may be important as well.

Ease of use. From an end user perspective, the MFT tool should be at least as easy or easier than what he's doing now. Look for tools that make managed email transfers transparent, as simple as adding a file attachment; Web-based user interfaces should be as simple as clicking on a desktop shortcut. The idea is to steer users away from finding their own solutions so they're more productive.

Central management. An MFT solution should eliminate the redundancy of separately managing homegrown FTP and scripted-transfer processes. In large, distributed environments, a group responsible for managed file-transfer can, in effect, act as an internal service provider for organizations within the enterprise. For example, in California, one office, using Axway products, provides transfer services for other state agencies. This eliminates the redundancy of each business unit purchasing or building its own transfer capability.

Monitoring, auditing and reporting capabilities. You should be able to use an MFT product to track many aspects of transfers, including the success of a transfer, performance, workflow's effect on the business, adherence to policy, compliance and security practices.

"Some MFT tools have audits that can run on continuous basis," says Skybakmoen. "You have visibility, a consolidated view across multiple servers and clients. You can see who is logged in, what files have been sent. You see all accounts and see the progression of onboarding partners."

Encryption and key management. Look for encryption that can be applied automatically, according to corporate policy, based on data sensitivity and who is sending and receiving. If you need it, there should be encryption strength options to support different data-security scenarios. Key management should be highly automated and transparent, providing secure handling based on user role while also ensuring data availability.

Workflow support. This can be relatively simple or extremely complex, but in any case, the product should be able to support your business workflows for handling sensitive files. Managed file-transfer is especially critical to assuring the smooth flow of information in large, complex B2B integration initiatives, which involve hundreds of partners who all have their own requirements and workflows.

Integration and application support. In addition to complex B2B workflow integration, MFT tools should natively support major enterprise-level applications, such as databases, ERP and CRM, ordering and transaction systems, and popular mainframe systems, plus all OSes. The product should provide robust APIs to integrate less-common, specialized or proprietary in-house systems. Also, look for strong Lightweight Directory Access Protocol and Active Directory integration.

"MFT can pull information in different formats from disparate data systems, such as CRM, supply chain management, order management," says Andre Bakken, Ipswitch File Transfer director of product management. "Data translation and transformation must be correct for B2B integration."