How to conduct an ITSM assessment that actually means something

ITIL (Information Technology Infrastructure Library), a standard framework for managing the lifecycle of IT Services, is sweeping the U.S. Based on a 2011 analysis of 23 ITIL studies, Rob England concluded that the compound annual growth in ITIL adoption was 20%± and that ITIL training attendance increased at a compound annual rate of 30% for the past ten years. Despite this apparent surge of adoption, enterprises continue to struggle with ITIL’s daunting framework.

Recognizing the confusion inherent in ITIL alignment, numerous vendors have created “ITSM assessments” with varying degrees of complexity and debatable value. These assessments draw upon frameworks such as ITIL, CMMI-SVC, Cobit and, occasionally, BiSL or more specific constructs such as KCS and IAITAM. Where does one begin? What is most important? Where will improvement deliver the best payback? How can one ensure that all phases of implementation share a common and scalable foundation?

Figure 1: Fundamental Assessment Approach

All assessments follow a pretty basic formula:

Determine and document the current state of ITSM in the organization.

Determine and document the desired state of ITSM in the organization.

Establish a practical path from current to desired state (roadmap).

Simply stated, the objective is to successfully execute the ITSM roadmap, thereby achieving a heightened level of service that meets the needs of the business. But don’t let those vendors through the door just yet because this is where ITSM initiatives go sideways.

Current state, desired state and roadmap mean nothing without first establishing scope and methodology. How comprehensive should the assessment be? Does it need to be repeatable? Which processes and functions should be targeted? Should it be survey-based? Who should participate?

Rather than seeking input from the ever so eager and friendly salespeople, one can follow a simple three-step exercise to determine scope and methodology. These steps, described in the following sections, may save you millions of dollars. I have seen dozens of large enterprises fail to take these steps with an estimated average loss of $1.25M. For smaller enterprises ($500M – $1B in revenue), the waste is closer to about $450,000. The bulk of this amount is the cost of failed projects. In some instances those losses exceeded $10M (usually involving CMDB implementations).

Three Steps to a Meaningful ITSM Assessment

Though these steps are simple, they are by no means easy. For best results, one should solicit the participation of both IT and business stakeholders. If the answer comes easily, keep asking the question because easy answers are almost always wrong. Consider using a professional facilitator, preferably someone with deep, practical knowledge of ITIL and a solid foundation in COBIT and CMMI-SVC.

So, the three steps are really three questions:

Why do you need an ITSM Assessment?

What do you need to know?

How do you gain that knowledge?

Step 1: WHY Do You Need an ITSM Assessment?

IT Service Management aligns the delivery of IT services with the needs of the enterprise. Thus, any examination of ITSM is in the context of the business. If one needs an ITSM assessment, the business must be experiencing pain related to service delivery.

There should now be a list of processes with associated pain points. How well can the business bear the pain over the next few years? With this preliminary analysis, one should be able to create a prioritized list of processes that require attention.

For now, there is no need to worry about process dependencies. For instance, someone may suggest that a CMDB is required for further improvements to Event Management. Leave those types of issues for the assessment itself.

Step 2: WHAT Do You Need to Know?

Figure 2: Four Assessment Needs

Now that the organization understands why an assessment is required (of if an assessment is required), it can identify, at least in broad terms, the information required for such an assessment.

Referring the chart in Figure 2, IT management need only ask four questions to determine the needs of an assessment.

Is ISO/IEC 20000 Certification Required?

If the organization requires ISO/IEC 20000 certification, a Registered Certification Body (four listed in the U.S.) must provide a standardized audit, process improvement recommendations, and certification. For most enterprises, this is a major investment spanning considerable time.

Does Repeated Benchmarking Provide Value?

Does the organization really need a score for each ITIL process? Will the assessment be repeated on a frequent and regular basis? Will these scores affect performance awards? Will the results be prescriptive or actionable and will those prescribed actions significantly benefit the business?

The sales pitch for an ITSM assessment usually includes an ITIL axiom like, “You can’t manage what you don’t measure” (a meme often incorrectly attributed to Deming or Drucker). One must ask if scores are the best measure of a process? To what extent do process maturity scores drive improvements? Not much. Each process has its own set of Critical Success Factors, Key Performance Indicators and metrics. These are far more detailed and effective data points than an assessment score. Ah, but what about the big picture? Again, ITIL and COBIT provide far more effective metrics for governance and improvement on a macro level.

That said, there are some pretty impressive assessments available, some with administrative functions and audience differentiation baked into the interface. However, one should build a business case and measure, through CSFs and KPIs, the value of such assessments to the business.

Do you need an ITSM Strategy and Framework?

Does the organization already have an intelligent strategy for its ITSM framework? Is there a frequently refreshed roadmap for ITSM improvement? For most enterprises, the honest answer to this is no. Numerous Fortune 500 enterprises have implemented and “optimized” processes without strategy, roadmap, or framework. The good news is that they keep consultants like me busy.

To build an ITSM strategy, an organization needs enough information on each process to prioritize those processes as pieces of the overall service workflow.

To gauge the priority of each process, we focus on three factors:

Business value of the process – the extent to which the process enables the business to generate revenue.

Maturity gap between current and desired state – small, medium or large gap (scores not really required).

Order of precedence – is the process a prerequisite for improvement of another process?

To complete the strategic roadmap, one will also need high-level information on ITSM-related tools, integration architecture, service catalog, project schedule, service desk, asset management, discovery, organizational model, business objectives, and perceived pain points.

Are You Targeting Specific Processes?

To some extent, everything up to this point is preparation and planning. When we improve a process, we do that in the context of the lifecycle. This task requires deep and detailed data on process flows, forms, stakeholders, taxonomy, inputs, outputs, KPIs, governance, tools, and pain points.

As this assessment will be the most prescriptive, it will require the most input from stakeholders.

Step 3: HOW Do You Gain that Knowledge?

Finally, the organization identifies the assessment parameters based on the data required. Similar to the previous step, we divide assessments into four types.

ISO/IEC 20000 Certification

The only standardized ITSM assessment is the audit associated with the ISO/IEC 20000 certification (created by itSMF and currently owned and operated by APM Group Ltd.). The journey to ISO 20k is non-trivial. As of this writing, 586 organizations have acquired this certification. The process is basically measure, improve, measure, improve, ………. , measure, certify. Because the purpose of improvement is certification, this is not the best approach to prescriptive process optimization.

Vendor-Supplied ITSM Assessment

The administration, content, and output of ITSM assessments vary wildly between vendors. In most cases, the ITSM assessment generates revenue not from the cost of the assessment but from the services required to deliver the recommended improvements.

Rule #1: “If you don’t know where you’re going, you’ll probably end up somewhere else” (Lawrence J. Peter). Without a strategy and roadmap, assessments will lead you to a place you would rather not be.

Rule #2: The assessment matters far less than the assessor. When seeking guidance on ITSM optimization, one needs wisdom more than data. A skilled assessor understands this workflow in the context of a broader lifecycle and can expand the analysis to identify bottlenecks that are not obvious from an assessment score. An example is Release Management. The Service Desk may complain that release packages are poorly documented and buggy. Is that the fault of the Release Manager or is it a flaw with the upstream processes that generate the Service Design Package?

Rule #3: Scores are only useful as benchmarks and benchmarks are only useful when contextually accurate (e.g. relative performance within a market segment). Despite the appeal of a spider diagram, avoid scored assessments unless compelled for business reasons. Resources are better spent analyzing and implementing.

Rule #4: An assessment without implementation is a knick-knack. Validate the partner’s implementation experience and capability before signing up for any assessments and be prepared to act.

Rule #5: A free assessment is a sales pitch.

Rule #6: A survey-based assessment using a continuous sliding scale of respondent perception is a measure of process, attitude, and mood. So is a two year old child.

Rule #7: In ITSM assessments, simpler is better. Once a vendor decides that the assessment needs to produce a repeatable score, the usefulness of that tool will decline rapidly. If you doubt this, just look under the covers of any assessment tool for the scoring methodology or examine the questions and response choices for adherence to survey best practices.

Strategy and Roadmap Workshops

Enterprise Service Management strategies save money because not having them wastes money. Without guiding principles, clear ownership, executive sponsorship, and a modular, prioritized roadmap, the ITSM journey falters almost immediately. Service Catalogs and CMDBs make a strategy mandatory. For those who lack an actionable Service Strategy and Roadmap, this is the first assessment to consider.

An enterprise needs an experienced ITSM facilitator for strategy workshops. Typically, the assessment team will perform a high-level process assessment, relevant tool analysis, framework architecture integration study, and a handful of half-day workshops where the gathered information is molded into a plan for staged implementation.

Targeted Process Assessments

Organizations know where the pain points are and have a pretty good sense of the underlying factors. The assessor finds this knowledge scattered across SMEs, Service Desk personnel, business line managers, development teams, project office, and many other areas. The assessor’s value is in putting these puzzle pieces together to form a picture of the broader flows and critical bottlenecks. Through the inherited authority of the project sponsor, the assessor dissolves the organizational boundaries that stymy process optimization and, with an understanding of the broader flow, assists in correctly identifying areas where investment would yield the highest return.

For these assessments, look for a consultant who has insightful experience with the targeted process. An assessment of IT Asset Management, a process poorly covered in ITIL (a footnote in the SACM process), requires a different skill set than an assessment of Release and Deployment Management or Event Management.

The output from a Targeted Process Assessment should be specific, actionable, and detailed. Expect more than a list of recommendations. Each recommendation should tie to a gap and have an associated value to the business. Essentially, IT management should be able to construct an initial business case for each recommended improvement without a lot of extra effort.

Summary

Liam McGlynn

Organizations are investing tens of millions in ITSM assessments. I have seen stacks of them sitting on the shelves of executives or tucked away in some dark and dusty corner of a cubicle. Whether these assessments were incompetent or comprehensive, as dust collectors, they have zero value.

How prevalent is the lunacy of useless ITSM assessments? From my own experience and from conversations with others in the field, vendors are selling a lot of dust collectors. Nobody wants to be the person who sponsored or managed a high-profile boondoggle.

So the advice is this.

Don’t waste time on scores because there are better ways to sell ITSM to the board than a spider diagram.

Develop and maintain an ITSM Strategy and Roadmap. As Yogi Berra once said, “If you don’t know where you’re going, you’ll wind up somewhere else”.

Assessing and implementing need to be in close proximity to each other.

Get an assessor with wisdom who can facilitate a room full of people.

Finally, follow the three steps before you let the vendors into your office.

The journey may have many waypoints but let’s just make it once.

Liam McGlynn is a Managing Consultant at Fruition Partners, a leading cloud systems integrator for service management and a Preferred Partner of ServiceNow.

10 Responses to "
How to conduct an ITSM assessment that actually means something "

Hey Liam, great seeing you out here writing. No surprise it’s such a good article after having worked with you before and knowing what you bring. We need to reconnect as it’s been far too long since last time.

Two small points:
Rather than “pain” I prefer to say “need, pain, or risk” – three types of reason for an improvement. they all loosely mean “pain” I know, but it helps people think more broadly.
And if after all Liam’s excellent arguments, you still decide you need to measure current state, please don’t measure “maturity”. Measure risk and unrealised value.

Wow – excellent article. I like your writing style as well – crisp and clearly based on experience in practice and not just theory.

I have found that getting people to really articulate step 2 – Determine Desired State can be a bit difficult. I don’t normally find that people in management really know where they want to be – just “better” than what they are now. Perhaps this helps explain the power of various ‘scores’ and spider charts. Lacking all else – numbers provide some measure of (false) security. The idea that each finding and recommendation should (be able to) stand as a business case is excellent. The numbers we should be talking about are not maturity scores but revenue/profit/expense related.

I also appreciate how you have quite specifically stated that you need different skill sets across the ITSM landscape – what makes someone knowledgeable and skilled in Release does not necessarily translate as knowledgeable and skilled in Capacity…that being said I’m not sure the Industry really has enough depth in all areas where you can support such specialization for all engagements/firms. There seems to be a concentration around Incident/Service Desk/Change and….doing assessments.

One issue is that you have a quote appear twice and attribute it to a different people. First it was Lawrence J. Peters (misspelled – http://en.wikipedia.org/wiki/Laurence_J._Peter) and then later it was Yogi Berra. Either way, I like the quote.

Excellent article. It helps us to clarify and evaluate our current position, and where we are going. Definitely very necessary before choosing a tool. In many companies start with the tool, badly advised by vendors, without having begun by the evaluation. Thanks for the article.