Popular White Paper On This Topic

Please check out EPICOR and our Vantage ERP solution. It is a
fully integrated, end-to-end solution that is 100%.NET and built
on a 100% SOA architecture. Some information can be gathered at
www.epicor.com. Also, if you would like "hard lit."/CD or have
questions, feel free to contact me directly.

Can you confirm your statement below that Vantage 8 is 100% .NET and 100%
SOA - since it was my understanding is that it's native development
environment is the niche Progress toolset and using the Progress database,
with SQL Server database being available as an extra cost option, and with
some inherent performance issues with SQL Server being non-native.

Being at version 8 suggests it has been around considerably longer than
anything .NET.

Not really to the point -- indeed, this package may be on our list, but if I can't convince management that this vendor -- measured on similar criteria to other vendors -- is likely to be reliable, etc., then I can't sell them anything.

Resources (including white papers from vendors) that would support creating decent criteria and a path to collect the data are welcome. Please don't suggest another package for me, though.

"Resources (including white papers from vendors) that would
support creating decent criteria and a path to collect the data
are welcome"

-Good point. As you head down the eval. path, we would be more
than willing to shed light on why EPICOR has over 20,000
customers and is very strong in the government and defense
contracting field and provide you with the appropriate
"resources" to verify that. I can tell you just as a start, you
will find some interesting webcasts on our homepage under the
"events" dropdown menu that pertain specifically to your
vertical and how Vantage addresses those. As I stated before as
well, I will be happy to load you up with ammo and info to
address your requirements. I've also attached a link to an
article that might address some of your reliability concerns.
http://news.moneycentral.msn.com/ticker/article.as p?ID=5770109&F
eed=PR&Date
060605

There are over 1,000 manufacturing systems in North America and yet The Wall Street Journal has stated that “73 % of all manufacturers are dissatisfied with their current ERP systems.”

Why?

There are a number of theories why implementations have problems:
- Poor planning or no planning at all
- Top Management not involved or did not commit to the project
- Unreliable data
- Lack of training or implementation assistance
- Poor selection process
- Lost project momentum
- Business processes are not corrected
But the main reason that implementations fail is the legacy selection process commonly used is flawed.

The Budget Approval Dance

The first step in any selection process is expenditure approval. Middle managers spend days defining their selection process plan. The more detailed the budget approval request is, the more detailed the selection plan is, the more due diligence is assumed and therefore perceived risk is lessened. The legacy selection plan usually contains:
- A multiple page, detailed system requirements definition (sent to a short list of vendors for confirmation), and submitted as a Request for Proposal (RFP)
- Multiple, onerous “sales demos” with various systems
- Telephone reference calls on the selected vendor

The middle manager can get so obsessed with budget approval; and, the upper manager can get so consumed in confirming the selection process is sound, that the most important objective in selecting a new system can be overlooked. The most important goal should be to ensure your company is successful with the new system! Any other consideration should be secondary.

Systems Today Are Function-Rich

The more you evaluate systems, the more you recognize that all have an abundance of functionality. The reason they fail is not a lack of information, it is the exact opposite. They are too cumbersome and too difficult to learn.

Then why do we devote our entire search to evaluating which system has the best and/or most functionality? Is it because new systems are purchased only every ten years, therefore, no one individual has the experience to learn from their mistakes?

Let’s review the legacy selection approach described above.

1 - Issue a detailed multi-page novel called the “system requirements list” to all software vendors to fill out (honestly) thereby confirming which match. However software companies want to remain in consideration, and are motivated to answer each question with a carefully worded, “yes we do that!”

2 - The selection team then shoulders the arduous task of reviewing “sales demos” in an attempt to decipher the differences. “Sales demos” are designed to look good. If the “sales demo” did not look good, the software company would go out of business. Software companies hire professional presenters who know precisely which keystrokes will present their software in the best light.

Remember the first day you looked at the system you use now? How difficult did it seem then versus today? How many months did it take before the haze lifted and the system became second nature? Is it possible to recognize the pros and cons of a system you are reviewing for the first time in an 8 hour, “sales demo”?

3 - You now call references to confirm that companies are happy with their systems. Where did you get the references from? Did the software vendor carefully select “bullet-proof” customers that swear the software turns water into wine?

4 - And voila … you have selected the very best system for your company. Or have you?

A Fresh Approach To Selecting ERP Systems

If time is money, then speed is profit. How can we speed up the ERP selection process, yet not fall into the same traps as the majority do, which fail?

The most important factor in selecting a new system is: to make sure your company is successful with the new system. If 73% of manufacturers are not satisfied with their current ERP system and used the same selection process as you, why will yours be different? Maybe a different process should be investigated.

Not to mention that the step-by-step, due diligence process historically used, costs $10,000s of internal resource time.

This 3-Step Plan is only common sense. But best of all, it will take far less time to complete and your results will be guaranteed!

Step 1 – Define the critical requirements that are unique to your company and match to the vendors on your short list. Please note: Critical requirements only. You have to assume that all systems will have an “Aged Trial Balance”. This list should not be longer than two pages. Only continue to step 2 with those that sufficiently match.

Step 2 – Visit a company using the system, preferably one you can drive to, in a similar industry and size as your own. You may not get all criteria matched, but ask. With these demands, it is more difficult for the vendor to select who you visit.
I once heard of a manufacturer who was “paid” to be a positive reference for a software firm, yet they did not even use the software.
A visit is so much better than a phone call that in fact, it can take the place of the “sales demo” entirely.

Step 3 – After the first 2 steps, you should have a favorite system. If you are still not comfortable enough to take the plunge, this final step eliminates any further risk. Ask this vendor to bring in the trainer you will work with after the sale is complete, not the professional presenter. Trainers have to live with their promises after the sale, and will be quite forward about what the system can, or cannot do.

Have the trainer set up the software around your requirements and enter a sub-set of your data. The intention will be to present the system as if it were live at your facility. Offer to pay for this service. Trainers are not offered to prospective clients because existing customers are paying them to implement their projects. This “proof of concept pilot” may cost a few thousand dollars, but will be far less expensive than the time-consuming step-by-step method.

The most important factor in selecting a system is to make sure your company is successful with the new system. This 3-step process will guarantee results. Why would you use any other method?

Remember, if you fail to implement, why do you care what the software does?

As far as the 100% .Net claim goes can you please enlighten the
forum in a new thread on the embedded use of Open Edge, Open
Edge Proxy, Open Edge Application Server, Progress Network
Client, Progress database and the Epicor .NET API in the Vantage
product?

This is a good example of buyer beware and highlights the value
of the forum for getting objective information and not relying
on vendor claims.

I like many in this forum am always interested in the claimed stats of
vendors (and I work for one), and what it really reflects.

Even when factual, it leads me to some questions.

20,000 Epicor Customers are claimed below, and no doubt verifiable given
their colourfull acquisition history over recent years, however with a
fraction over 2,000 Epicor employees, I wonder how much attention can really
go into customer care and into new systems deployment, and with at least 6
current separate product lines to service, including Vantage 8, it begs the
question where do the R & D dollars get channelled, both now and into the
future.

It's a strange Customer/Employee ratio for a mainstream ERP author by any
barometer, other than those in tier 3 and 4 of this marketplace.

Also, the Epicor current annual report discusses a corporate strategy of
"fusion" of their principle current product lines (6) into a single
world-beater - so what shape will that really have for those who buy any one
of the 6 product very different lines today ?

Looks to me familiar to the current dilemma's facing Microsoft with 4 to
merge, and Oracle with 3 big ones to blend.

In the history of ERP over the last 25 years or so (yes starting with MRP) I
am not aware of a single success of such a bold strategy, and I am not one
of those who believe it can be solved if your pockets are deep enough, so I
await Valhalla and Utopia with baited breath, and really hope, for the sake
of the Customers in this market, that I am proved wrong, before the grim
reaper arrives to take what remains of my soul (probably dressed as Gates or
Ellison).

> In the history of ERP over the last 25 years or so (yes starting with MRP) I
> am not aware of a single success of such a bold strategy, and I am not one
> of those who believe it can be solved if your pockets are deep enough, so I
> await Valhalla and Utopia with baited breath, and really hope, for the sake
> of the Customers in this market, that I am proved wrong, before the grim
> reaper arrives to take what remains of my soul (probably dressed as Gates or
> Ellison).

Hear, hear, John!

As I asked in my ERP Graveyard blog, talking about Made2Manage's private equity group buying Intuitive from its private equity group:

"This all stopped making sense for customers a long time ago. Now it's clearly down to the economics of milking revenue streams and eliminating overhead - which is a decidedly short-term proposition, with an eye toward a financial exit. So I repeat, it begs the question: As customers get more and more worried, and defect to (ahem) more compelling options from the world of open source, who's going to buy these things after the attrition?"

>I like many in this forum am always interested in the claimed stats of
>vendors (and I work for one), and what it really reflects.
>
>Even when factual, it leads me to some questions.
>
>20,000 Epicor Customers are claimed below, and no doubt verifiable given
>their colourfull acquisition history over recent years, however with a
>fraction over 2,000 Epicor employees, I wonder how much attention can really
>go into customer care and into new systems deployment, and with at least 6
>current separate product lines to service, including Vantage 8, it begs the
>question where do the R & D dollars get channelled, both now and into the
>future.
>
>It's a strange Customer/Employee ratio for a mainstream ERP author by any
>barometer, other than those in tier 3 and 4 of this marketplace.

It may be only if you're thinking about 1980 barometer's.
Look at how many major players (including Oracle and Microsoft
among others) have outsourced some or all of their R&D,
development, support, and/or product maintenance. That alone
could explain the numbers.

Then combine that with what technology they use - if they
had nothing but old COBOL or programming (not to mention
debuggin) 1.2 billion lines of C++ you could understand
that. But this is 2006 where there are toolsets that
don't require 100/200 developers per product.

I'm not saying there is or isn't a problem, in fact
I'd even suggest that an employee ratio is totally the
wrong thing to look at in that area anyway. Look at
the product life cycle - either it's continuously
evolving or its stagnating. That's the best way
to judge effectiveness of the R&D work.

>Also, the Epicor current annual report discusses a corporate strategy of
>"fusion" of their principle current product lines (6) into a single
>world-beater - so what shape will that really have for those who buy any one
>of the 6 product very different lines today ?
>
>Looks to me familiar to the current dilemma's facing Microsoft with 4 to
>merge, and Oracle with 3 big ones to blend.

Not to mention infor's probably got the biggest array of
disparate products of any of them. No, the industry history
in that area is not good. It's probably even more dismal than
you're painting. However, that doesn't mean I can agree with
the sentiment that all of the customers deserve a huge amount
of pity. The picture for them is generally not so bleak.

Every situation is different, so yes there are cases where
all of the "fusion" is or already has caused customers to
be in very, very bad situations where they are forced to
move off the product they have almost instantaneously.
But for the most part, those are smaller systems where the
revnue stream's dissipation isn't all that worth preserving.
(because you can bet if there's enough $ in it, someone is
going to support the customer base even if it's not the
original software company) The large systems generally
are going to stay intact for any length of time until
a transition path is established, keeping at least some
level of support and maintenance until that happens.
When you look at both MS and Oracle's efforts they
certainly aren't overnight, even considering the huge
amount of resources that can and are brought to bear on
the projects. Think about the average lifespan of any
current technology - ERP systems only live 3-5 years anyway.
By the time these "fusion" efforts come around, the customer
base would have either accepted those or move to something else
in due course without creating a new product. This is not
really any different than what happens everyday amongst
all of the choices in software today.

I'm cynical of *any* product that tries to says it solves
every problem of every kind. That has nothing to do with
the decisions to merge or keep separate any one set of
products that any vendor has done. It's the claim, not
the product. In the mean time, while any particular
vendor works on their next version of a product, the
customer base goes about daily business. Software
products go through a normal periodic product re-writes
and upgrades that could have gone on even if some products
weren't being merged. Thinking they need pity, or are
somehow all of a sudden out in the cold is FUD.

I am confident that we can identify packages that meet general
and critical functional requirements. Indeed, I have several in
hand right now.

This is the question: how do I distinguish between the ability of
two different organizations to deliver on that functionality
through the implementation and beyond?

Yes, we will talk to other customers -- but that can be a
minefield of its own. Opinions aren't always based on evidence.
What should be asked; certainly people have traveled this
ground before.

The merger issue is a relevant one, but by itself it doesn't
exclude any package -- when we bought Lilly it wasn't owned by
Infor... And, more significantly, I would prefer to move toward
something than away from it.

In a specialized field like ours (small A&D eng & mfg), even the
big vendors (like Infor) will likely have a small presence in
our market -- I'm interested in how we could collect and present
the issues to our management.

My management considers support to be equally important to
functionality. (And I agree.) This discussion shows that
resources to examine support (including implementation support)
are much less developed than those to examine functionality.

Although the APICS story (thanks, pratzy)is moving in the right
direction (focus on critical requirements and support for rapid
and effective implementation), it still puts me in a position of
negotiating with one vendor rather than two. Not acceptable.

Sounds like I need to get my academic friends interested in this
problem so we can get some thinking based on fundamentals and
some results based on data.

You may want to visit the website for SoftSelect
(http://www.softselect.com). They have a free offering that shows how well
software vendors have sold into specific market niches. (Full disclosure,
yes, I use that company to help companies find ERP packages.)

As far as data to collect about ERP Vendors, I would recommend not looking
at checklist services as they can be confusing as well as misleading.
Questions like "does your software do shop floor simulation scheduling" is
not specific enough to do anything more than possibly eliminate software
vendors from competition. Analyze your top mission critical business
activities and learn how the software fits those activities would be more
meaningful. That effort would allow you to ask the question, "Can you
schedule the shop floor in a simulation mode for a specific work order using
two work centers to allow us to determine the best path through the shop
floor since we have several methods to route an order through the shop".
However, this comes with a cost. That cost though is more than recovered by
knowing that you have done your homework before buying software. Please
remember:
* software HELPS manage, software does not manage a company, people
do

DigiMIS are an Application Service Provider (A.S.P.) delivering hosted ERP solutions to sites throughout Europe and the US. We focus primarily on delivering the Syspro ERP solution to our customers - (www.syspro.com)

We currently deliver Syspro ERP solutions to several defence contractors in the US and can setup some initial conference calls or visits if that would help.

I would also be happy to arrange a live web demo of our solutions to allow you to see the system in action.

I would suggest that if you are looking at ERP- in your
interviews with potential implementors, get a feel for where
their strengths and experiences lie. YOu may even consider
bringing one set of consultants in to do the financials and the
other to do manufacturing- It is done at many of the larger
firms. Project manager gets toe valuate both firms
simultaneously.
rsn

Unfortunately this list can do that - some of the lack
of monitor invervention is as puzzling as the lack of
restraint. But it is what it is.

If no one else says so, my apologies for the slight
off topic sideline that I engaged in as a consequence of
the original message. Perhaps we forget to change the
subject line more often than we should.

>This is the question: how do I distinguish between the ability of
> two different organizations to deliver on that functionality
> through the implementation and beyond?
>
>Yes, we will talk to other customers -- but that can be a
> minefield of its own. Opinions aren't always based on evidence.
> What should be asked; certainly people have traveled this
> ground before.

I'm not sure that we can tell you much that you don't
already know. It's mostly common sense -

You can't predict the future. You can only take a look
at past performance and see what that says about the
company. (just like playing the stock market) Not
only talk to the customers the vendor gives you and
wants you to talk to, find other customers on your own
who the vendor may not be to reticent for you to talk to.
Ask them not just what they like, ask them what they
don't like. What the package does well *and* what it
doesn't do well. Always ask for the specifics, not generalities.
You'll only get the real story by knowing details.
Every package has flaws, every company their issues.
Be afraid if you ask what the flaws are and you get
no response.

Standard business resources like D&B and others
are nice, but get to know the people as much as
the company and the product. After you're comfortable
with the people, make sure they're not planning on
going anywhere.

---------------Original Message---------------
>I have to admit that I'm a bit stumped by the responses.
>
>I am confident that we can identify packages that meet general
> and critical functional requirements. Indeed, I have several
in
> hand right now.
>
>This is the question: how do I distinguish between the ability
of
> two different organizations to deliver on that functionality
> through the implementation and beyond?
>
>Yes, we will talk to other customers -- but that can be a
> minefield of its own. Opinions aren't always based on
evidence.
> What should be asked; certainly people have traveled this
> ground before.

I will explain from my experience in the ERP market and without
considering an specific area or funtionality so it will apply to
all software buying process. Honestly there is not a 100% secure
questionary you can develop and you may need to have experience
at a software company or working with software vendors to know
all the tricks they may do but still. But still you can follow
this path to make a desicion.

When you are choosing between two companies you need to consider
three major areas to ask questions : a) the product and b )the
company c)the costs!!!.
I find it to be as important to know if the product can do what
we want as if the company is the right one. Cost? some costs not
considered may take you to a very expensive software that seems
to be cheaper, sometimes the implementation can be as expesive
or more than the product or even the support can be very
expensive. It can be possible that two products can do preatty
much what you need but one company does not give support or has
never implemented, etc.

All this questions may have an impact in the decision and the
cost of the decision...

a) THE PRODUCT
-A.1) It may sound strange I say it but its true... the first
thing is do you know really what you are looking for? Many
companies I know they are lookig for something that they
listened is good bla bla bla... but they dont really know what
they are looking for and thats an advantage for the vendor who
has not ethic.
- Product functionality... ask for detail experience, if they
dont have exactly what you need, search for close topics. Ask
for examples on how they did it and if they dont have it aks for
examples on what they can manage.
-Demo. Ask for a demo of the product.. warning dont spect the
product to do 100% what you need but in that case be sure you
can customize the product and ask for a detail explanation on
how you can customize including tools and technology, even ask
for a demo of a simple customization. Ask for a demo on a
vainilla product ....why? because sometimes software vendors
may customize for the presentation without saying is a
customization so you have the feeling they do it but its not
true and you may pay for that as an extra.. other thing they may
customize for the presentation with ' magic wires' or not based
of a real business rule.
-Expect YES and NO answers for if the product can do it but
always ask for how you may do it. Only YES mmmmm may mean they
are just trying to make you feel ok.
-Do they integrate with other softwares? Do they already have any
client with the ntegration with x software? Do you need to pay
for the integration or is it included in the project?
-Technology foundations used to develop the product...
-If you have funtional rules based on any organization (APICS,
ect.) ask how they cover them or if they do cover the topics.

b) THE COMPANY - You need to ask questions based on the following
topics: The company itself and the experience they have and what
they can do to performe the implementation and support the
product ones it is implemented. Even of the company just sell
the product you need to be sure someone has the experience in
the market to help you implement and who can support it. Also
you need to think in 3 mayor time frames of all sotware buying
process (pre-purchasing time period, implementation time period
and post implementation time period)

B.1)Company foundations (before implementing) - who is this
company?
- It is important to know the time a company has been in the
market so you know you are buying a new product or something
that some else has used...how solid the company is and if there
is any risk the company will vanish and you will end having a
product no-one supports.
- Where are they located? In any country, International companies
that are new in the market may have no cultural experience or
may vanish in a period of time do to the fact they couldnt
perform in the country. Also location may imply a cost to get
the human resources to the location the product is needed. Other
thing is communications may not fit with your schedules due to
the fact you are in different location (ex: you need to report a
bug but they support 9 to 6 australian time).. you can imagine
more things related to this topic...
- Always YES is not a good sign!!! Vendors use to say yes to all
so they sell. Honest Companies may not have the solution right
away but will give one and at the end thats the import thing to
do. YES companies normally have no solutions and give you
workarounds so you are happy so normally YES companies give me
the feeling they dont really know what i want, they just want to
sell. Im more confortable with companies how say yes, let me
verify or this could be the options even through this answers
you can get to know if they have experience for what you need.

- Do they give training? Do they have training facilities? Do
they give training at the companies site?
- Do they implement the software?
- Do they support the software? Do they have a 24 hrs support
center or similar?

B.2)Company expertise (during implementation)- Do they know what
they are doing, are they prepared?
How many clients they have? How many implementations they have?
how many full implementations they have? Time for
implementations? Size of the implemenation teams? Requierements
for implementation? How often you need to upgrade?

B.3)Company support (post implementation) - You have the product
you are using it....now what!??? you need to be sure someone can
support the product when you are using it because you never know
whats gonna happen.

c) THE COST ...

C.1) Product cost? is cost based on licenses by user?
c.2) how often you need to upgrade? this may lead to an
additional cost every 2 or 4o years
c.3) Cost of implementation? time and resources?
c.4) Technical requierements? do i need to end up buying extra
things? (memory, DB, comunications, etc)
c.5) Training. Cost by student or group? training for
implementation teams, end users, trainers, etc... Do i need to
develop any training documenation?
c.6) Support. annual support cost? does it include upgrades? does
it include insite support?
c.7) Do i need to hire a consulting company?
c.8) Do i need to train my team in new technologies to support or
use the software.....

As you see there are many or a lot of questions to do hope this
helps to start.. if you need more detail let me know and I can
help you work on this even creating a questionary or any similar
document.

If there exist a gap betwwen the Sales and the Technical teams, then these vendors should be called cheaters. This I am sure is in complete understanding of the two departments.
I think we should use this forum to hold up such vendors. This will also reduce the problem of vendor selection. It is for this reason the client list and duration of a client is very important during vendor selection. I fact if they claim they have done similar things, the client must be cross verified.

It is critical to visit companies using the software
ERP Salespeople are trained to make the system look as easy as possible. All “sales demos” look great. If they didn’t, the software company would go out of business. It is impossible to know how easy the software is to use until you have been using it for 12 mos.
The only way to evaluate a system’s suitability is to visit companies using it.

Here are 3 references for you to call …
Software Vendors commonly offer three, carefully selected customer references for you to speak to. What do you learn from speaking to three satisfied customers? Absolutely nothing!

Software vendors will often give you customers that are located near their offices. If you are located near their training center, you may get excellent customer service as well. Unless you are, the local support this customer has received may not be representative of the support you will receive.

Ask for a list of customers in your area, using the same version as the one they are presenting to you for 12+ months, with similar requirements and a similar employee size, (everyone thinks their system is excellent the first six months, it takes at least one year to know the pros and cons). Then choose which to visit, preferably one that you know.

If they do not have customers in your area that are in your industry, ask to visit someone locally that is a similar size, and at least speak to customers in a similar industry (confirming the system works in your industry). But make sure they have been live on the version you are considering for an extended period, or else what is the value of their opinion?

Two reasons you will want to visit local companies are:
- to confirm the quality of local support. If they give you a company three States away to visit, that can only confirm how good the local support is three States away. In fact the trainers being provided in your town probably have entirely different skill levels than the ones they provide for Pittsburgh.
- Forces the software vendor to not only give you their “best” customer. The odds of your experience (an A+ implementation) being equivalent to their very best customer is very slim. Every company has only one very best customer. Besides what do you learn from a company that says the software vendor “walks on water”. Asking for local customers to visit will let you visit some B and B+ implementations. Likely closer to what you will experience. You want to learn how the software vendor responded when the implementation did not go perfectly (none ever do).
Visiting local customers will provide insight into the on-going support and training you should expect to receive.

If they don’t claim to have local customers using the system 12 mos. or more, they either do not (what does that tell you?) or none of their local customers are satisfied.

How much did it cost (over or under their original budget)?
Some software vendors quote a lower amount of training and services than may be necessary. If you don’t budget enough for training upfront, guess who you will buy additional training from later? As the FRAM oil filter commercial states: “You can pay me now, or you can pay me later.”
When speaking to customers, ask if they above or below their original budget for services?

How long did it take to implement (longer or shorter than their original plan)?
They say you can have a Cheap, Fast or a Quality implementation, but only pick two. The software vendor may have completed a 3 month implementation once, but what is their average time to implement? The industry average time to implement is 6 to 12 months. If they were able to go live fast, was it because they didn’t have systems in place previously?

What system did they use before?
A person’s opinion regarding their present system is directly relational to the satisfaction of their previous system experience (at this company or others they’ve worked for). If it was a “poor” system previously, any average system will seem wonderful now.

What version of the software do they have now?
What value is there if the software vendor has been showing the “new version” and the customers you speak to are on the “old version”. The old could be excellent and the new, full of bugs.

More Miscellaneous questions
- How many years since going live? (more than 12 mos?)
- How is the software support and response time?
- Where does “local” support come from? (same as you?)
- Are reports easy to modify to your requirements?
- Have you ever requested an enhancement and did they listen?
- Has your companies’ revenue grown since going live? If so, what %? How many employees have you added since going live? (More or less efficient since going live?)
- If you could do it again with what you know now, would make the same decision?

I apologize if I am repeating advice from previous posts on this thread,
and
depending on the vendor it might make sense to visit the vendor and
conduct
a vendor "review." Obviously you only do this extensive a review on your
short listed vendors. You may want to put together a small team who
will
look for information in the following major and sub categories. Pick
and
choose from the list only the most important ones to you.

I have to comment that this seems like a classic case of
overanalysis.

Spend some time to get to know the vendors/partners you are talking
to. Get to know them as people, and focus on their integrity and
commitment, not pick at nits. The right people committed to your
project and given the latitude to add value will deliver results.

No one approach guarantees success. However, too much emphasis on
checklists and features definitely takes time away from "critical
requirements" and important issues such as vendor fit, ability to work
with the consulting/implementation team. The best solution is the best
implementation. In a vacuum there is no "best" solution.

Some of these claims are very difficult to sort out. That being said,
Vantage is certainly very close. I always get concerned when clients
have these very rigid technical criteria that in most cases don't
correlate with long term success! In other words, let's keep our eyes
on the ball.

If at all possible, a visit to the vendor headquarters can be the most
beneficial approach.

When we have two finalists, which is common, we develop a set of
additional demonstration criteria that is unique to each of them, i.e.
to clarify the differences in the two solutions relative to the business
requirements, and then we do this at the vendor's location. We can
find out a lot about them, their people, and their commitment, which is
critical.

There's a lot of good advice in this response. Have to say that this
stream points out the exact reason that customers should work with
(notice I did not say "hire") a consultant to guide them through the
selection process. Yes, clients can figure out how to do this, but
like anything else, why would you want to risk the most critical
decision for a company without having every possible advantage. A good
consultant will educate you and take you by the hand and lead you to the
best solution for YOU, and minimize your risk. Keep the good advice
coming.

References should be useful assuming you ask the right questions, and
don't over-emphasize the answers.

We should probably suffice to say that there are no guarantees. Delivery
(on functionality or anything else), support for the product, even future
capabilities are dependent upon people. People who will change jobs, change
companies, retire, take leave of absence for other reasons etc.

You are in the unenviable position of trying to guarantee a future based
upon today's reality.

Software manufacturer/developer companies have personalities that are driven
by all the layers of management from the top right on down to the delivery
person. If any one of those drivers leaves the company or changes positions
the personality of the support and possibly the dedication to the support of
existing customers will change.

I would care about:
1. do they train their delivery people (consultants, help desk, tech
support)? How much and how often?
2. what is the longest and shortest time with the company for support and
delivery staff?
3. what is the longest and shortest time in any kind of target industry for
the support and delivery staff?
4. Are practice managers measured, compensated, hired and fired by
utilization of delivery staff? is there a measure of "success" for
clients?
5. how long are releases supported by the company before their help
desk/support line tells you "that problem is fixed in a later release. You
will have to upgrade to get the fix"?

Plenty more questions to ask. Pick your functionality, make sure you can
manage the technology, tools and functionality yourself if you're put in a
position where you must, make sure your people will accept it and work to
make it successful, then find the BEST delivery people you can find, either
from the vendor or from the independent providers pool. Don't skimp and
don't rush.