Yesterday we featured the first part of our interview with Peter Marshall of software firm Commerce Decisions. Today we will look at his views on evaluation tender methodologies.

So Peter, what is the problem here?

“Basically we are using techniques we don't understand and we can then get undesirable results. For instance, we have seen tenders where the evaluation methodology in fact means that the contracting authority is willing to pay twice as much - or more - for an “excellent” solution compared to an “acceptable” solution.”

“Now that may be right, but if you ask most budget holders they might say I will pay 20% more - but not twice as much. Yet that is the implication of the system they are using. This confusion comes because often the users don't understand the implications and effects of the evaluation schemes. And the related but separate problem is that often these methods don’t even rate the bids consistently and reliably – we can end up with results that are almost arbitrary”.

Can you give an example of that? I’ve certainly seen different scoring processes that can lead to different supplier choices.

“Yes, this is one I talked about at the eWorld session. Let's say we run a competition between two suppliers – call then Green and Red. We evaluate the tenders on cost, weighted at 40%, and technical factors, weighted at 60%, as the two top level criteria. Here are the results.

“The cost scores are calculated by taking the ratio of cost compared to the cheapest bid – a very common methodology (cheapest bid / each other bid). So in this case, Red just wins as a lower quality but significantly cheaper bid”.

Technical

Cost

Cost score

Total score

Green

54/60

£8 M

25

79

Red

41/60

£5 M

40

81

OK, that all looks fine...

“But now, we discover that another tender which we though had arrived late was in fact submitted on time. We need to score that too - the Blue bid. So Blue scored 32 on Technical and came in at £4 million. Applying the same formulae, this is how the scores now look – the other bidders scores for cost change because they are calculated against the lower Blue bid”.

Technical

Cost

Cost score

Total score

Green

54/60

£8 M

20

74

Red

41/60

£5 M

32

73

Blue

32/60

£4 M

40

72

So Blue does not win – but merely introducing that 3rd bid changes the decision from Red to Green!

“Exactly! So how can we say this is achieving best value for money, or is a fair process, if the ranking is as arbitrary as this? The marking of one bid depends on the price of another – even if that bid is not a strong one overall.”

I have also constructed examples a bit like this. But why is it that these dodgy methodologies are still used so often?

“The good news is that suppliers understand this even less well than contracting authorities. So we don’t see many challenges to buyer decisions. What tends to happen, I believe, is that suppliers pitch their bids around what they believe the budgets to be. But because they can’t know how important cost is going to be in the marking scheme, that can’t truly “optimise” their bids in terms of real VFM. What we get if we are lucky are the best bids that are affordable in the supplier’s’ eyes. These evaluation processes are so obtuse, we don’t get optimal bids because suppliers don’t understand what we mean by VFM – so they can’t offer it”.

So what’s the answer?

“We need to be more transparent on scoring, and be clear about how we value cost against the other evaluation factors. That will help suppliers offer real value, and take away the illogicality of their score being dependent on other bidders”.

“At Commerce Decisions we work with our clients to implement different ways of combining the technical scores and prices to reliably rank the tenders based on their perceived value for money. We call our method “Relative Value For Money” or RVFM”.

Many thanks to Peter Marshall – and if you want to get into these areas in more detail, you can contact him via info@cd.qinetiq.com.

Voices (2)

bitter and twisted:
19.03.2015 at 8:40 am

What empirical evidence is there that tendering actually works?

Why not just call in the stakeholders favorite and beat them up with the cheapest serious alternative ?

Remember – I never joke.

The stakeholders get their way. The beancounters are placated.

‘Cos lets face it. Procurement is actually all about The Specification, and Big Buys are fundamentally about Avoiding Crap.

I hope to see the back of the erroneous (cheapest bid / other bid) mechanism. It creates non-linear results and anomalies like the example given. Completely agree that no one using the mechanism seems to understand what issues they are getting themselves into.

We’ve written up flaws on this mechanism and some of the other popular types that exist:

Associate Sponsors

Procurement Research

Spend Matters research is based on a year-long editorial calendar, exploring bigger picture areas and issues.

New! Everything You Always Wanted To Know About Your Spend Data (But Were Afraid To Ask)In this briefing paper we take a look at how spend analytics, and smart and creative use of the source data, can help organisations answer some of those tricky, complex, or even embarrassing questions relating to procurement expenditure. We look at questions such as: How do I get to the bottom of tail spend? and How much corruption really goes on my organisation?

New! Geo-political Risk - An Informed Global View Is Essential The fifth of our short papers outlining key supply chain risk areas looks at geo-political risks, which include war and revolution as well as labour disputes and the like. Clearly, this sort of risk event can be amongst the most serious in terms of implications for buyers.

New! "Man-Made" Risk - Different Risks Require Thoughtful StrategiesThis is the fourth of our short papers outlining key supply chain risk areas. This examines "man-made" risks, a broad category that ranges from fires and explosions to strikes and labour disputes. All can have a serious impact for customers of the firms affected, so effective risk management is vital.

New! Natural Disasters – How to Mitigate Unavoidable RisksThis is the third in a series of short papers outlining the supply chain risks that can have the most devastating effects on your business. This paper examines the risk to suppliers (and ultimately the buyer) of a natural disaster - a risk type that is mostly unavoidable but one against which you can mitigate.

New! Supply Chain Risk – Getting To Grips With n-Tier VisibilityIn this briefing paper, we look at supply chain risk, and in particular the risks that emerge from beyond the first-tier (the direct suppliers to our organisation). We include ideas on how to gain greater visibility of the whole supply chain or network, and what to do once we have it.

New! The Merger – A Procurement StoryA short novel in four chapters. It follows and scripts the daily lives of two CPOs whose businesses have been recently merged: the one, a fully tech-savvy, source-to-pay function, the other, relying on personal expertise and hands-on manual processes. We learn of their opposing challenges and how they resolve them -- together.

Improve Your Procurement Negotiation SkillsThe work of behavioural psychologist and Nobel Prize winner Dr Daniel Kahneman suggests some winning techniques. How to use priming, attitudes to risk and anchoring to your advantage in negotiations!

Full Value Buying: Moving Beyond Price NegotiationWe look at range of procurement mechanisms that can drive better value, and then consider two aspects in more detail - demand management and specification - which can drive far greater benefits than a pure price focus!