Saturday, April 30, 2005

The LMS selection process in a nutshell

In working on learning strategies, I am sometimes asked to help a company decide which Learning Management System they should use. Here's the general approach that I recommend.

This is to be read in conjunction with my frequent admonitions to not allow the LMS to define your learning processes, nor to make its selection the starting point of your strategy development, nor to assume that any LMS is adequate to manage the totality of the learning that might take place. With that said, here’s an approach to selecting an LMS.

Form an LMS selection team that includes representatives of all those who will be involved in implementing and using it. Get someone from your IT department involved early, but ensure that they do not take ownership of the selection process. If you can afford it, get an objective outsider involved as well.

Agree on the strategic and operational processes by which you want to manage learning. These will follow from having already defined your strategic and operational learning objectives, which in turn will have been derived from the business goals of your various learning customers. Do not think in terms of LMS functionality, but in terms of process: what do you want to do, who is going to do it, how is it going to work.

Then look at this broad strategy, and list the requirements that a system supporting it must satisfy. Create a list of critical success factors for your e-learning systems. For example, if your strategy calls for you to implement competency-based learning, you may need to integrate with your Human Resource Management System (HRMS) data or your Enterprise Resource Planning (ERP) system. If you have operations around the world, you may have a need for multi-lingual or multi-byte character capability. Your list of requirements will probably have 12-24 items on it.

Some other factors that you should consider at this stage are:

Will you need local support in multiple countries?

What is your budget?

Do you want to host internally or have the vendor host for you?

How diverse is your target learner platform and your IT infrastructure?

Do you want to manage classroom activity (scheduling, evaluations)?

Will you be integrating third-party content?

Will you need a system that integrates with CD-ROM content or allows offline work to take place?

Do you need e-commerce or departmental charge-back capabilities?

How granular are your tracking and reporting needs?

Do you have to track compliance or certification training?

Do you have special requirements for data security?

Do you plan to integrate collaborative activities such as e-mail, chat, or communities in your learning?

Will you be managing synchronous virtual classroom activities?

Survey the market and come up with a short-list of systems that meet your critical needs. The survey can be done by issuing a request for information to all of the known vendors in the market (there are only 50 or 60 of any substance), by buying an off-the-shelf study, or by contracting someone to do the data collection work for you.

It is at this point where your LMS quest might lead you away from the straightforward purchase path: if your requirements don’t fit well what is available, you might explore the option of building rather than buying.

Reduce the list to less than half a dozen, using criteria that are important but did not make it to the critical list.

Next, develop a framework for evaluation of alternative systems. This is an important step, because it allows you to take control of the “demo process” and provides you with the right questions to ask. Without such a framework, you become a passive audience to the vendor’s presentation. The most helpful approach to defining an evaluation framework is to map out “a day in the life of a user” and then have the vendors talk about how their solution fits to that picture. You can break down a day in the life of an LMS administrator, a learner, an instructor, an instructional designer, an HRMS bureaucrat, and a learner tech-support person into a set of specific action steps, processes, or mini case-studies. Those steps, and the complexity of them, then become challenges to the vendor: Show me how your system supports these necessary processes.

Craft your RFP around your needs as already defined, don't use a template that you found on the web. Using someone else's idea of what is important is not only lazy, it can mislead the vendors and cause you to make decisions based on irrelevant criteria.Send out your RFP to your final short-list of vendors.

Once you have received responses to your request for proposal, cut the list to three. At this stage, you have enough information to differentiate the top candidates from the rest. All of your other criteria, the less important issues, the subjective feelings, the qualitative factors and so on can come into play.

Ask your final vendor list to set up demo sites that you can explore. Get learners, instructors, administrators and so on to go in and play, then provide you with feedback. Ask for references in companies that have installed the systems, and be sure to talk to as many of them as you can.

Once you have had the chance to try out the systems, have had the vendor presentations, and have seen how they address the day-in-the-life requirements, you should be comfortable about making a selection. Each party in the LMS selection team may have a different view, and should have a different perspective, but you will have enough structured data to come to a reasonably comfortable conclusion. Pick an LMS.

Oooh, that's a familiar comment, I wonder if it's from the same organisation I'm in!

However, there is one thing missing from that list I would think, and that's looking at what you might be using in the future.

A lot of those points are about what you are doing now, and you should always make sure that there are lot's of future proofing in such a system.

Looking at support, upgrades and customisations are quite important issues too. What if you need to make customisations, are these going to be supported, and will they be provided for during an upgrade or will you have to pay for those customisations each time you upgrade.

There are quite a few technical issues that need to be looked at as well and should be weighed up along with the learning points made here.

Good points. And while looking to the future, look at whether or not your vendors are likely to be around to provide you with the service and the upgrades a couple of years from now. I know so many companies who have spent hundreds of thousands, even millions, then found their vendor acquired and their chosen product line terminated. It seems that vendor longevity varies in inverse proportion to vendor size. So going with a big company is not always a safe bet.

I just went throught the LMS selection process and I'm happy to see that our group did most of what you outlined.As we head into the implementation process, I'm looking for resources/ideas/information on what to look out for during the implementation and rollout process. Maybe a good future article for you?

The one thing I see missing from a lot of "evaluate LMS's" strategy conversation is:

a. Actual time that is available to implement and manage an LMS. A lot of training departments need to be able to offer training, but realistically, they have to teach classes as their main activity - and there isn't, and won't, be someone to "manage" a complex LMS installation.

So, the simpler, the better. The cheaper, the better. The more support at a low price the better.

If you are not very familiar with the options available and the expected outcomes, prototyping on paper would probably really help. Prototype the webpages, walk through a simulation of how you expect courses and interaction to run.

without a real world representation of the digital environment people often fool themselves into beleiving something they are not getting.

Wouldl you only look at vendors? What about some of the open source solutions out there, such as moodle, dokeos, segue, Atutor, etc. Looking at these, even if you decided to go with a vendor package in the end, might help give you some insights into what you're getting into with an LMS.

I would of course look at open source. But i have found that every time I recommend, say, Moodle to a client it is rapidly rejected. Sometimes for good reasons, sometimes for sad reasons. Big companies still like to deal with big brands, and can't get their heads around making a major committment to a product that has no obvious vendor behind it.

Good article, we are currently in the process of Vendor selection for our LMS need. I would like to add to the article above that it is important for you to also check the skill level and tech savy ness of the end users. You also need to ensure that you encourage the users to make use of the LMS rather than asking the vendors to provide API for intergration with office products. The more the users hit your LMS the more the success rate. That's my view, would like to read comments from others.