As IT outsourcing has evolved so, too, have IT outsourcing customers' benchmarking needs. As a result, a new approach to benchmarking is necessary, a new method for establishing new and better ways of operating. Kathy Rudy, partner with outsourcing consultancy Information Services Group (ISG), discusses pros and cons of benchmarking for transformation and how to do it right.

Traditionally, IT services customers have approached periodic benchmarking of their outsourcing deals as a hammer to tamp down on vendor pricing. But as outsourcing itself has evolved so, too, have outsourcing customers' benchmarking needs.

"[Customers] are upping the ante on the providers and pushing the envelope on benchmarking suppliers," says Kathy Rudy, partner with outsourcing consultancy Information Services Group (ISG). IT leaders don't just want to know that their costs are in line with the market. Increasingly, they may want to see where they stand in areas of innovation, agility, standardization and quality.

That requires a new approach to benchmarking, says Rudy, one that's approached not as a way to tighten the screws on the way things have always been done, but as a method for establishing new and better ways of operating.

CIO.com talked to Rudy about the drivers behind benchmarking for transformation, its benefits and drawbacks for customers and providers, and how to do it right.

How is benchmarking of an outsourcing deal typically approached?

Kathy Rudy, Partner, ISG: Clauses have historically often been used to focus on discrete service towers. While this is useful in terms of making adjustments to pricing within individual functional areas, this perspective is limited, as it doesn't take a holistic view of the enterprise.

Also, at the end of a contract term, companies will sometimes seek a benchmark without formally executing the clause to determine if the price is market competitive. This way they can use the results either as a renegotiation tool or to provide evidence to management that they don't need to go out for a full competitive bid. The potential downside of this approach is that it can be shortsighted and may not provide the full range of benefits; specifically, running scenarios for future state, working on stronger governance, understanding different sourcing strategies, and so forth.

What about the changing nature of today's outsourcing relationships calls for a more nuanced approach to benchmarking?

Rudy: Increasingly, outsourcing relationships involve driving transformational change within the client operation. So, rather than simply ensuring that the pricing a supplier provides is in line with the market, clients are looking to their outsourcing partners to build new operating models. Rather than saying, "How do I compare with the top ten percent of companies?" clients want to know, "What's the optimal future state I can achieve?"

[Clients want] benchmarks that not only validate that they are getting value for services today, but that the service bundles are in line with emerging trends. They're asking if other companies are buying outsourced and retained service bundles in different packages, and if they're missing a competitive advantage by not understanding where the market is headed.

They are beginning to ask for benchmarks around supplier agility, innovation, simplification, flexibility, and quality.

How can benchmarking be used to better leverage the benefits of today's outsourcing deals?

Rudy: Benchmarking is an essential part of the process of getting to the future state of standardization and utility computing. First off, a benchmark that assesses the current state of the overall operation provides a baseline analysis that an organization needs to chart a path forward. In order to understand where you want to get to, you need to know where you [are]. This type of analysis also quantifies the improvement opportunity -- the "size of the prize" that can be achieved with standardization and utility computing.

A detailed analysis of the current state is [also] essential to defining and assessing alternative approaches to change. Specifically, the benchmark can quantify in real dollars the cost of placing operational constraints on suppliers.

These constraints often come from the business, which doesn't have insight into the cost of asking for specific requirements. For example, there are costs associated with not replacing legacy systems, not consolidating data centers, or resisting the offshoring of resources. By quantifying the cost of [such] constraints, the CIO can justify making transformational changes to business as usual. [It gives] CIO the data needed to ask for real change, rather than just tightening the screws.

This approach puts the onus on the customer to manage costs by managing demand. Why is that ideal?

Rudy: In a traditional IT environment, the business has no visibility into how they use IT resources. When there's no connection between consumption and cost and no consequences for consuming inefficiently, the business will use more and more.

Our data shows that IT costs have gone up over the years because of huge increases in demand for and consumption of IT resources [that] have outpaced steadily declining IT unit costs. For example, the cost of disk storage hardware has gone down significantly on a per megabyte basis over the years, but the consumption of storage resources has gone through the roof, so total spend on storage has grown significantly.

A consumption-based pricing model provides an incentive for the business to consume resources more efficiently. It's like an open bar versus a cash bar. This visibility also allows [an] infrastructure [group] to point upstream [to] application development to where much of the consumption starts and where there may be less rigor around consumption.

This would seem to benefit the provider as well? Are there downsides?

Rudy: Standardization and consumption-based pricing allow the service provider to do what they're good at. Rather than customizing service delivery for each client, the service provider delivers services using standardized tools, frameworks and methodologies to leverage economies of scale. The focus is on the "what" rather than the "how"; this allows providers to optimize efficiency and increase their profit margins.

The potential downside is that unless the business has bought into it and users are sufficiently communicated to and expectations set, it could create a perception that service quality has decreased.

>What would need to change is the scope of what's being benchmarked and the outcomes in terms of how the findings are used. Clients and providers will need third parties with relevant data around standard services environments, as well as meaningful consumption-based metrics.

Rudy: The standard benchmarking clauses themselves, for the most part, are fine. Many benchmark clauses focus on measuring against the top quartile and use that as a hammer instead of a means for transformation, but the clause itself doesn't have to stop you from combining other objectives.

What would need to change is the scope of what's being benchmarked and the outcomes in terms of how the findings are used. Clients and providers will need third parties with relevant data around standard services environments, as well as meaningful consumption-based metrics.

Does this approach require a different approach to negotiations -- and renegotiations -- of outsourcing deals?

Rudy: In many cases the sourcing negotiation has to fundamentally change. The traditional approach is to use a highly prescriptive, rigid and detailed Request for Proposal that providers respond to by ticking off boxes on a list. But when clients are seeking to transform their operations, there's typically more than one right answer to the question of how to proceed.

We're seeing clients take a more open-ended approach. Rather than defining what the solutions should look like, clients are asking, "What do you see as the best solution?"

Is this approach right for every deal?

Rudy: Traditional benchmarks aimed at optimizing existing operations still have an important role to play. But in situations where organizations seek to transform their operating model, this broader approach to benchmarking is gaining increasing attention. More and more clients are looking for the next evolution in service provision, and business as usual is not an option.