How I’d Fix the RFP Process for Buying Software

August 24, 2010

The oth­er day Ron Schmeltzer tweet­ed how bro­ken the RFP process is for buy­ing soft­ware and his opin­ion of the process (he dis­likes it). [Cor­rec­tion: Ron was talk­ing about con­sult­ing RFP’s, so the post below doesn’t real­ly mat­ter. But, it did get me off my bum to write the post below, some­thing I’ve had in my head for quite some time.] I quick­ly respond­ed, ask­ing him how he’d do it dif­fer­ent­ly if he were a buy­er. I have some thoughts on this, but was curi­ous what he thought. Brett Miller respond­ed with a blog post he wrote back in July as to the pros and cons of the RFP process.

In gen­er­al, my opin­ion about Brett’s points are that the cons out­weigh the pros in most every case, and that there are alter­na­tive ways to achieve the pros with­out invok­ing the cons.

I feel quite strong­ly that the RFP process for soft­ware pur­chas­ing is total­ly bro­ken, and have an idea to replace it based on the work that some ear­ly founders of WebLay­ers did when I was sell­ing Action­al to them at Cred­it Suisse.

1. RFP’s are biased. Typ­i­cal­ly, RFP’s are issued by com­pa­nies after they’ve done some due dili­gence. That due dili­gence is “biased” based on who they spoke to, that bias finds its way into the form and func­tion of the RFP. If all ven­dors have been involved pri­or to the RFP issue, that’s fine. But, if not… then the RFP is weight­ed towards those who have par­tic­i­pat­ed. And, that’s not always good for the con­sumer.

2. RPF’s only pro­vide a par­tial view into what’s impor­tant.RFP’s often have hun­dreds of ques­tions, some requir­ing com­plex answers. They’re meant to (1) get a com­plete com­par­i­son of rel­e­vant infor­ma­tion, and (2) stan­dard­ize the answers. Well, by the time a pur­chase is made and an imple­men­ta­tion hap­pens, the state of the var­i­ous fea­tures will change, so know­ing the cur­rent state isn’t nec­es­sar­i­ly help­ful. I real­ize it pro­vides a base­line, but that assumes that none of the ven­dors stretch­es the truth. Also, a sim­ple ques­tion like “Do you sup­port WS-Secu­ri­ty” doesn’t have a sim­ple answer, like “yes”. Usu­al­ly, the answer is some­thing between “no” and “sort of”… there are inter­op­er­abil­i­ty issues, min­i­mum plat­form issues, which pieces of the stan­dard are sup­port­ed, and how the sup­port is imple­ment­ed. Sec­ond, stan­dard­ized answers are not use­ful for a large por­tion of the ques­tions… and in my opin­ion those are the impor­tant ques­tions. What RPF writ­ers real­ly should want to under­stand are what makes each ven­dor unique, and how their phi­los­o­phy around the solu­tion aligns with the needs of the orga­ni­za­tion.

3. RPF respons­es are dif­fi­cult to write, and even more dif­fi­cult to eval­u­ate. Final­ly, com­pa­nies usu­al­ly have a very short time to respond to an RFP mak­ing the respons­es less than the qual­i­ty doc­u­ments any­one would real­ly like. I know it’s sur­pris­ing to buy­ers, but the prod­uct infor­ma­tion you would expect to be cut-and-paste is not often avail­able. Even a pri­or RFP response that’s 3 months ear­li­er is prob­a­bly out of date. And, even the best “cut-and-paster” out there (I think I’m up there) is hard pressed to weave mul­ti­ple cut-and-paste sources back togeth­er into a pro­fes­sion­al look­ing and con­sis­tent doc­u­ment. What about the review process? It’s time con­sum­ing and sim­i­lar­ly biased. A grad­ing sys­tem would cer­tain­ly be unable to eval­u­ate things like strate­gic align­ment and uniques… and any­thing less is sub­ject to the pref­er­ences of the read­er… and what they hap­pen to pick up when read­ing the respons­es. Try this. After all the respons­es have been read, put a sim­ple 10 ques­tion list of features/capabilities in front of the read­ers… and ask them to match them to the ven­dors that wrote the answers. Do you think they’d remem­ber which ven­dor wrote which answer?

One final point. The time/effort it takes for the teams to write the respons­es and the team to eval­u­ate them all… isn’t there a bet­ter way to spend our col­lec­tive time to get bet­ter tech­nol­o­gy out there faster and solve prob­lems soon­er?

So, what do I rec­om­mend?

Keep in mind that I’ve been almost exclu­sive­ly at vendors/integrators in my career so admit­ted­ly I’m prob­a­bly leav­ing some administrative/purchasing require­ments out. How­ev­er, I think the fol­low­ing makes for a great place to start.

1. Use ana­lysts only to get a view of the land­scape and make sure you know all the rel­e­vant ven­dors out there. Ana­lysts don’t have the time to do much hands-on eval­u­a­tion to val­i­date what the ven­dors tell them. And, ana­lysts have their own bias­es which may or may not align with your own. Save the ana­lysts for when you have spe­cif­ic ques­tions about rel­a­tive ven­dor com­par­isons and mar­ket trends.

2. Along with a non-sub­jec­tive check­list of stan­dards and IT require­ments (such as inter­op­er­abil­i­ty with exist­ing systems/platforms, sup­port in par­tic­u­lar coun­tries, num­ber of SI’s trained in a prod­uct suite, etc.) deliv­er a set of use cas­es for how the prod­uct would be used. The use cas­es should include some long-term (and there­fore less spe­cif­ic items) and some short term cas­es. The short term ones should real­ly address the dri­ving need for the eval­u­a­tion. At least one use case should test per­for­mance and scal­a­bil­i­ty in order to prove out the scal­ing mod­el and help dri­ve to a final con­fig­u­ra­tion (and there­fore a final project cost). Oth­er use cas­es should include inter­op­er­abil­i­ty test­ing for inte­gra­tion to exist­ing sys­tems, and how the prod­uct gets migrat­ed between devel­op­ment and pro­duc­tion.

Slight aside. By check­list I mean there should be nowhere on this list to explain any­thing. Answers should be unam­bigu­ous lists, yes/no, dates, num­bers, etc. This keeps it black-and-white. If it needs an expla­na­tion, it best to have the ven­dor answer in the con­text of the use case.

3. Ask ven­dor par­tic­i­pants to fill in the check­list, give “essay” answers to the use cas­es, and then pro­vide 10 items that they believe you should eval­u­ate as part of the eval­u­a­tion. These 10 items will be all you need to under­stand how each believes they com­pete with the oth­er ven­dors, the ven­dors’ philo­soph­i­cal align­ment to the prob­lem space, and their unique val­ue propo­si­tions. By the way, these 10 items should include use cas­es on how to test them, and an expla­na­tion of why they are impor­tant to the pro­posed solu­tion. Of course, these 10 items might be non-tech­ni­cal… like they might be about stan­dards sup­port, or SI rela­tion­ships, or what­ev­er the ven­dor thinks is impor­tant.

I believe if we moved in this direc­tion, we’d have a process that got cus­tomers what they need, faster, with high­er qual­i­ty results. And, the efforts used in the deci­sion process (by imple­ment­ing the use cas­es in a POC) would be direct­ly rel­e­vant to deploy­ing the solu­tion, so once the process is com­plete, you’ve done more than select­ed a ven­dor, you’ve begun your imple­men­ta­tion.

David

It's a quick read, and, if you can believe it considering that it's a book on investing, fun.

If you're looking for a simple and successful investing strategy, one that's purposely designed to keep you motivated, The Elephant's Paycheck is for you. And if you're already an accomplished investor, this book is likely for your spouse or your children so that they can become interested in what you're doing with the family's wealth.

Reader Interactions

Comments

I’m actu­al­ly a huge fan of RFPs. Of course, I’m pre­dom­i­nate­ly look­ing at it from the client per­spec­tive, not the ven­dor side. Let me give you my rea­sons why I love RFPs.

1. They force the client stake­hold­ers to think about, write out, order and agree what the goals are for the project/software/etc. Com­pa­nies will often have very vague rea­sons for want­i­ng to do some­thing and even less under­stand­ing of how to eval­u­ate if its suc­cess­ful. Forc­ing exec­u­tives to agree to and state the pur­pose of the exer­cise is crit­i­cal. Believe it or not, RFPs are a great way of doing this, if for no oth­er rea­son than the fact that stake­hold­ers usu­al­ly have to go to the CFO or Board to get approval to spend that kind of mon­ey. (Note: The longer I’ve worked in soft­ware, the more I’ve come to love CFOs.)

2. It pro­vides a check­list that stake­hold­ers have to agree to and then use to jus­ti­fy why they want to go with a par­tic­u­lar solu­tion. Many times on the client side, one can only get lim­it­ed amounts of time from crit­i­cal stake­hold­ers. What­ev­er they pro­fess, key stake­hold­ers can rarely devote the type of time a major project needs.

First hav­ing the meet­ing to agree to the check­list and dis­cuss the impor­tance of dif­fer­ent items is key from an edu­ca­tion per­spec­tive. Then when it comes to ven­dor eval­u­a­tion, which is often months lat­er, the check­list pro­vides struc­ture from which to have dis­cus­sions and make evaluations/decisions.

Lack­ing the struc­ture of a check­list with which to eval­u­ate mul­ti­ple solu­tions, stake­hold­ers will often latch onto the first solu­tion they see that seems to address their needs. **please see note below.

3. Address­ing asym­me­tries in under­stand­ing and prod­uct exper­tise. Ven­dors know infi­nite­ly more about the mar­ket space than prospec­tive clients. Hear­ing dif­fer­ent ven­dors answer the same ques­tions is extreme­ly edu­ca­tion­al for dif­fer­ent stake­hold­ers. For exam­ple, often times a stake­hold­er won’t under­stand some answers being giv­en, but if three ven­dors answer a ques­tion in one way, while one ven­dor pro­vides a com­plete­ly dif­fer­ent answer; one gets an oppor­tu­ni­ty to have real dis­cus­sion about why.

4. Pric­ing. While price is rarely the decid­ing fac­tor, hav­ing mul­ti­ple bids is crit­i­cal for get­ting a good price. Espe­cial­ly if the selec­tion peri­od cov­ers the vendor’s year-end or quar­ter-end. Often times one ven­dor will drop their price to a ridicu­lous lev­el because that sales­per­son needs the sale for some oth­er rea­son.

There’s a great line from War­ren Buf­fett: “You can’t be any smarter than the dumb­est com­peti­tor you’re com­pet­ing against.” When there are equiv­a­lent prod­ucts and one ven­dor is will­ing to drop their price by 80%, even if you’d nev­er select that ven­dor, every oth­er ven­dor has to drop their price to an equiv­a­lent lev­el.

Price has noth­ing to do with select­ing who you’re going to go with, but hav­ing mul­ti­ple bids has every­thing to do with how much you pay for the prod­uct you’ve select­ed.

I hope this helps you under­stand why I love RFPs so much.

**Note: I don’t think peo­ple work­ing for ven­dors ful­ly under­stand the impor­tance of pre­sent­ing first. Most often in client firms, there’s real­ly only one way dis­cussed to do some­thing. So the first per­son who can sug­gest a viable way to do it gets to do it that way. So managers/executives are “trained” to go with what­ev­er way they see that works first and move for­ward rapid­ly with it. There is a huge advan­tage to being the first ven­dor who presents.

Thanks for the detailed response. I cer­tain­ly appre­ci­ate your per­spec­tive from a cus­tomer point of view.

How­ev­er… I’m not advo­cat­ing not putting some­thing out to bid. There­fore, your com­ment about pric­ing doesn’t apply. I still think you’d have mul­ti­ple bids, so you’d have the same effect of “keep­ing the ven­dors hon­est, and hav­ing a strong nego­ti­at­ing posi­tion”.

With respect to your oth­er points… I like your point about #3, where if one ven­dor answers dif­fer­ent­ly it’s a good place to probe. That’s kin­da like how the Israelis do air­line secu­ri­ty and it works quite well (com­pared to our way). How­ev­er, I’m not sure why all three of your points can’t be addressed by the use cas­es I men­tioned. In fact, a use-case dri­ven RFP would align much bet­ter with agile devel­op­ment meth­ods (which I think can be used to deliv­er any IT project in the enter­prise “bet­ter”). You’d have a prac­ti­cal way to mea­sure ROI, you’d have your check­list for stake­hold­er agree­ment, and you’d still have a way to com­pare ven­dors… though it would be about how they’d solve the use cas­es (or what they’d add/delete from them), rather than if they sup­port pieces of tech­nol­o­gy the RFP writer read about some­where and thinks is rel­e­vant.

In fact, since cus­tomers under­stand less about tech­nol­o­gy and more about their own use cas­es, by focus­ing on doc­u­ment­ing their use cas­es, they can prop­er­ly learn from the ven­dors to make an edu­cat­ed deci­sion.

What I find is that cus­tomers ask “tech­nol­o­gy” ques­tions on RFP’s that “don’t make sense” or “don’t make sense with­out more con­text” and so the answers become some­what non-sen­si­cal. I thinks ven­dors, by def­i­n­i­tion, under­stand tech­nol­o­gy bet­ter than cus­tomers, and so let the ven­dors talk tech­nol­o­gy, let cus­tomers cor­ral the ven­dors into focus­ing on the use cas­es they’ve defined.

My turn for an aside… I in now way think cus­tomers don’t know any­thing about tech­nol­o­gy. Many are extreme­ly knowl­edge­able, and in many ways more knowl­edge­able than ven­dors because they have real-world expe­ri­ence. Most pre­sales folks don’t get so see how the soft­ware is actu­al­ly used. How­ev­er, in a broad sense, ven­dors will have a bet­ter under­stand­ing of the rel­e­vant tech­nol­o­gy issues and pros and cons. Each ven­dor will have their own, and the job of the eval­u­a­tor is to aggre­gate all the dif­fer­ent philoso­phies, and pick one that aligns with their own.

I hope I’m mak­ing sense.

In sum­ma­ry. I think the use cas­es I sug­gest meet your require­ments bet­ter than a sim­ple list of tech­nol­o­gy ques­tions do. They hold ven­dors account­able to results, rather than just tech­nol­o­gy because they are com­mit­ted to deliv­er­ing use cas­es, not just prod­uct fea­tures. And, the work done in eval­u­a­tion, when done towards use cas­es can be imple­ment­ed right away (with­in pro­to­col of migra­tion to pro­duc­tion) rather than the POC/evaluation being an abstract exer­cise after which the real work gets start­ed.

I also think that ask­ing the ven­dors what they think is impor­tant is a good way to learn about the space, the rel­a­tive strengths and weak­ness­es of the prod­ucts, and so on.

By the way, I like the idea of CFO’s too… it’s just that there’s one or two I know that have been real ass­holes.

And, while there are some good rea­sons to present first, pre­sent­ing last also has some advan­tages. First ven­dor to do the POC is usu­al­ly the one who helps the cus­tomer set­up their POC envi­ron­ment, so it looks like they’ve had a hard­er time. Also, hav­ing more time to pre­pare by not going first can make you look pol­ished. And, pre­sent­ing last gives you the final word, and an oppor­tu­ni­ty to answer all the remain­ing objec­tions because your com­peti­tors have fired all their ammo… and you’re left able to leave uncer­tain­ty about those ven­dors in your wake with­out those ven­dors always hav­ing an oppor­tu­ni­ty to respond (or to respond in the inti­mate way you do when doing a week long POC).

very inter­est­ing post — I myself cre­at­ed a new process work­ing with Siemens years ago when work­ing on some stuff for the BBC — hav­ing suf­fered the huge­ly drawn out RFP process that left the cus­tomer often more con­fused — we decid­ed to take an approach we called a Request for Engage­ment (RFE).

What should be in an RFE?:

Lim­it­ed page num­ber response
Keep it tight and rel­e­vant — rec­om­men­da­tion is for less than 40 pages
But if a ven­dor calls and asks for more, it’s usu­al­ly for good rea­son and you should be flex­i­ble (with­in rea­son)

Force the ven­dor them to cut the mar­ket­ing mate­r­i­al — you can even state that inclu­sion of mar­ket­ing mate­r­i­al will be marked down

Give spe­cif­ic use cas­es and ask ven­dors to respond to these cas­es

Include spe­cif­ic fail­ure use cas­es

Make your doc­u­ment mod­u­lar and ser­vice based
Help the ven­dor to answer and allows you to share the doc­u­ment review with oth­ers
Place your scenarios/questions into a mod­u­lar struc­ture

Make sure that your struc­ture is con­ducive to being split into sec­tions for answer by the ven­dor

As this ques­tion:
What are your Unique sell­ing points and why are they valu­able to me?
Ven­dors often know more about the com­pet­i­tive land­scape than you.

Move things for­ward with a Pilot Project, not a Proof of con­cept.

What every we all come up with, the RFP and asso­ci­at­ed process is old, out of date with mod­ern approach­es and very expen­sive for every­one. The BBC project in ques­tion? They had 44 ven­dors respond… They had to hire an SI just to read and col­late the respons­es! The small­est response was 6 vol­umes in size…

That exam­ple of the BBC is hilar­i­ous, and not sur­pris­ing. I think that the cost most com­pa­nies put into RFP’s should be mea­sured against their ROI… then they’d think about a dif­fer­ent way to do it. And, you make a key point of doing a pilot, not a POC. That cer­tain­ly would improve ROI and time to mar­ket.

Trackbacks

[…] against each oth­er in a proof-of-con­cept. A lot of time and mon­ey is wast­ed shame­less­ly. I wrote more about this else­where. [↩] My favorite anal­o­gy is that of the musi­cian. You can teach some­one to play the piano. […]