This is the blog of David M. Raab, marketing technology consultant and analyst. Mr. Raab is Principal at Raab Associates Inc. The blog is named for the Customer Experience Matrix, a tool to visualize marketing and operational interactions between a company and its customers.

But the real thrust of this paper is that you can save a lot of time and money by skipping a structured selection process and building a prototype.

It just about goes without saying that prototyping is a particular strength of Innovative Systems. According to the white paper itself, its products achieve the “fastest implementation in the industry” through “unmatched knowledgebases” (no pun intended, I suspect) that minimize custom tuning.

I’m not so sure I accept this particular claim, since several other vendors also offer specialized knowledgebases. But my real quarrel is with the notion of prototyping as a “faster, more cost-effective alternative to a formal vendor review.”

Simply put, a prototype is not a thorough test of product’s abilities. In most cases, it is limited to a sample of data from a subset of sources with a cursory review of the results. So prototypers learn quite a bit about the system set-up, user interface and reporting, but much less about true quality and scalability. Although I strongly encourage prototyping as part of structured selection process, it does not replace careful requirements definition and evaluation of alternative products against those requirements. Excluding products that cannot perform a quick prototype may make sense for Innovative Systems, but not for companies that want to ensure they find the best system for their needs. Similarly, minimizing the cost of a solution without taking into account the quality of its results is short-sighted. After all, the cheapest solution is to buy nothing at all—but that obviously does not give an adequate result.