Often times I am faced with the situation where a new client comes to me with an application that has literally 100s of unnecessary features and it is quite clear that things need to be drastically simplified for the project to have any chance of succeeding. How do you convince the client to take a more Minimum Viable Product (MVP) approach and simplify?

edit:

So the current top answer is to provide the client with a time/cost estimate for the huge application. I'm not too fond of this answer because it doesn't address the real problem with this situation. And that is - it's a bad practice to spec out a massive application and then try and build it from the get go. I feel much more comfortable initially building a small, simple MVP foundation. And then adding small features to that foundation one by one. So how do I convince the client to approach building software in this way?

It sounds like you're describing the difference between waterfall and agile/iterative development. If you google the pros/cons of those two approaches, you'll see all of the benefits of agile and you can use those as your argument. I have a list, but it's not handy at the moment.
–
Bob HornSep 21 '12 at 4:08

and I'd present an alternative, which greatly increases the chances you will get the project, with more realistic goals.
–
Paul HiemstraSep 20 '12 at 21:27

1

Where possible, break the estimates into "core", "nice to haves", "you have to be dreaming" (don't word it that way in front of the client though). Be careful in doing the entire Business Analysis work for free though.
–
mattnzSep 20 '12 at 21:47

@PaulHiemstra - excellent point. I'm used to working with internal clients, but the advice holds there as well.
–
TelastynSep 20 '12 at 22:40

You don't actually need to put estimates on all those features. be agile, pick the top twenty, and say you'll be happy to put those in version 1.0 for $x. State that you're unwilling to invest time up front estimating the price of versions 1.0 up to 1.8.
–
MSaltersSep 21 '12 at 10:08

I've learned that the fastest way to help a client Get A Clue® is to have them talk me through a user story for each and every feature or page they want. Several things happen, including:

They have to think about what the hell the page/feature is actually supposed to do.

As you ask for more and more detail ("and then? ... and then? ...") they begin to understand that the whole thing isn't going to come into being by having Magic Space Monkeys™ fly in and do it over the weekend.

They become true partners in the creation process. Which means that they will understand why changing their mind about something that is already 80% complete will cause a) schedule delay, and b) cost increase.

Seriously. User Stories by owners is one of the best tools I know for bringing at least some sanity to the process.

While discussing the cost and time-to-production aspect, ask for a ranking of the requirements ("must have", "should have", and "nice to have") so that the minimum viable product set of features that are essential are "must have" in version 1, then put the remainder of the requirements into sets of backlogs that could be accomplished as sprints subsequent to the first version. Hopefully the non-essential "nice to have" ones will go to the back of the pack and by the time you get to those sprints new ones that are more essential (from actual experience with the product) will have floated to the top.

The client should appreciate that you are considering the cost of their business and the importance of getting product quickly and you're not directly dismissing their "nice to have" by putting them in the backlog.

Edit to respond to OP edit:
To convince the client why it's a good idea to release early a minimum viable product explain that until the product is used by real users it's difficult to know whether it will be successful (especially in terms of usability). If the customer is hesitant to put out an early product to their whole user base, recommend doing a "limited beta" where it is available only to a targeted subset of their customers. Tell them the flip-side of this approach is a long, costly development cycle with a late determination that the product is unusable. 37 Signals has produced a couple of good references about this: Getting Real and Rework. The Getting Real is available on the web for free.

The core cause of your frustration with the situation is probably one of perception and misleading/wrong terms used by the customer. The customer does not usually come to you with a list of requirements, but a wish-list of every single thing they could think of that might be useful to them. Those are not all requirements because the customer hasn't spent the time yet to really think if each feature is truly required.

This is not necessarily always a problem

If your customer has the money for all those features and is willing to part with it, and you don't really care about solving the actual, real issues that the customer has, then this could be a very lucrative project. It happens, just very, very rarely, and for most developers it's soul-killing work because you can feel in advance that the project will not be successful for the customer in the end (even if it's financially successful for you as a developer). It's also high-risk because you're likely to end up with a fixed-cost project with a lot of uncertainty, and it's really issue to misjudge risk on large projects.

What if it is a problem?

Lets assume you're not in that rare situation. In this case you will want to address the two main shortcomings of the wish-list as given:

It's unlikely that the customer has a correct idea of the costs of developing such a large list of requirements, so you're unlikely to get the contract for the amount of money you actually need to do it.

It's unlikely that this wish-list accurately and succinctly describes the real problem that the customer has and wants to solve.

In my experience you need to address 2 to fix 1. Drilling down to the actual problem means that you, the developer, now have the necessary input to make creative leaps in solving the problem in ways that the customer themselves never even thought of. These solutions are likely to be much cheaper and quicker than the implementation of the full wish-list.

How do you fix that?
Like Matthew Flynn says in his answer - start by making the customer prioritize the requirements. This is not always easy, but force them to do it. If necessary use the phrase "If someone held a gun to your head, which single requirement would you keep?".
You will often find during this process that the customer really doesn't have a very clear idea of what the individual requirements mean. In that case do what Peter Rowell suggests and get the customer to work through User Stories. You and the customer will start to understand the problem and requirements much better, and then you can back to prioritization. Repeat those steps as often as you need to until you feel comfortable that you know enough to solve the customer's problem.

How does that answer the question of developing a solution?
Once you have a prioritized list of requirements, you have the input you need to suggest an incremental development process to your customer.
You don't have to call it Agile, but you can suggest to break down the contract into smaller pieces for each requirement (or indivisible set of requirements) and deliver them one by one with validation by the customer.
Or you can go all-out and use the many resources available online and offline to convince the customer that it's in their best interest to cooperate in one of the Agile styles of development.
In any case, you can provide your contract/project proposal in a form that clearly suggests these building blocks of requirements in priority order, each with their own cost and conclusion. Hold that carrot in front of the customer, and they may even think it was their own idea to decide whether to continue after increment :).

Thanks. But if you know the client wants to pay on a per project basis and all of this analysis work is to be done up front(which could potentially take dozens of hours) before the project price is decided, then how do you structure compensation for the initial analysis work?
–
RyanSep 21 '12 at 19:22

@Ryan - I would downright refuse to do any detailed analysis work up-front for a large project because a) it would give the wrong idea (see the "Cone of Uncertainty" - en.wikipedia.org/wiki/Cone_of_Uncertainty) and b) it's actually valuable work that the customer could take elsewhere to get the project done. Having actually ended up in that exact situation at least once that I know, I now make sure that we also charge for the analysis work (refundable if the client accepts the project).
–
Joris TimmermansSep 24 '12 at 7:10

I'd first try and explain to then that their requirements are not realistic, and present them with a set of counter requirements. Explain that this will solve their problem more simply, and faster, thus with less costs and risks. Do not try and make this into a technical discussion, the client does not care about that. The client does care about getting a good product in time, solving their business case. If the client does not budge, either accept the contract and do the work, or reject and explain the client why you cannot take respnsibilty for the project in this form.

Similar to what the other folks have suggested, but slightly different, I'd suggest that everything the customer may be valid, but they have to PRIORITIZE. Which item needs to be done first. Which item needs to be done second. Most importantly, if you run out of time, which items will it hurt the least not to have. Prioritize each item from 1 to 732 (or whatever) and complete them in that order.

A historical example of an application that ended up over budget and behind schedule due to excessive requirements is the FBI's Virtual Case File. It was intended to replace several dozen existing applications, and they all had to work completely, at once, on switch over. The project was ultimately cancelled. What would have been successful was to architect a framework, and incrementally replace individual applications into the new system. Instead, politics and infighting lead to every application stakeholder declaiming that their app was the most important one, and everyone goldbricked their specs with "must haves" from all the features they wanted added to the existing app. By the end, the volume of change requests written each day exceeded the amount of code actually written each day.

I've seen the "I gotta have it all" work in 2 cases. In one, the large financial client (using waterfall of all things), had 40 different systems (our company made one of the 40) get replaced and made operational in one fell swoop. Integration testing and communication were huge issues. My estimate is that they burned about $100k/day in conference calls (when you count the billing rate of everyone on the calls). In both of the cases, it took strong negotiators to hammer out a list of what was going to be delivered and then nailed everyone's feet to the ground. There was no moving of goalposts, no renegotiation. Both jobs ended up on-time, and on-spec. One was way over budget, the other was on-budget. For this you need very good project managers who know what their teams can deliver (the sort who can say that feature Q takes 3.5 weeks to make and test - and are at least 90% accurate in delivery promises).

Lacking the excellent PMs, negotiators and metrics, I'd advise pushing the client towards incremental deliveries. If they still want the entire gold brick at once, they might be better served by helping them find some other consultancy. Sometimes the best answer is "we can't help you."

As other answers state, prioritization is very important. A handy way of doing this is through the MoSCoW-method. But you might want to start with dividing the whole list up, otherwise the feature list itself will give you (or your customer) comprehension problems :)

Next to this, you have the problem of looking at the problem as a whole. You can probably solve this by sitting with your customer and go through the following steps:

Go globally through the features and distill categories from them

Prioritize (using MoSCoW) the categories, and maybe define a hierarchy (one basic category with default features, categories for major subjects, categories for specific variations of the major subjects). This might alter the categories you defined in the previous step (thanks to new insights).

Go through each feature one by one, and assign them to a category

Prioritize (using MoSCoW) the items in the top-x categories.

This will give a nice and condense top-down view of your requested features, and will give you handlebars to define different iterations to start with a base and extend it with other features.