Over Engineering Software

by Mike Willbanks on February 17th, 2008

Many times as developers, we tend to take our projects and over engineer them since we foresee most of the features that we may want in the future even although there is no purpose for it quite yet. This is quite a hindrance to actually ever getting the software we develop out the door as we can continue refactoring, extending and making our software more complex for what the end-users of a system might need in the future or even potentially edge cases that will never hit 90% of the population.

I have come to the distinct conclusion that when we start to over engineer a project, the chances of it ever getting out the door get slimmer by the second. I am not saying that it is bad to look at what is going to come in the future, however, we need to first develop for the need at hand and extend later by not coding ourselves into a box. This would mean ensuring that your data model, code structure and business logistics fall in line in a way where if things change you aren’t consistently having to modify mass amounts of code. Remember, OOP (Object Oriented Programming) is your friend.

Lets take for instance authentication, authorization and privileges. Presently, your software needs a single login, no groups or specific privileges are needed. You know that you will need this in the future but why develop it now? Simply create it in a way that is expandable in the future and move on. Keep it on a future to do list and cross the bridge when it is actually essential. This might not be the best example as many large applications that we develop may need this out of box. If we look specifically at the development time when you are creating all of this it would be simple to create the architecture behind it and implement it later.

A practice that I will sometimes use in my code, is if I know I am going to be building it out in the future, is to build the class definitions and having them return true. Say we are going to implement groups in the future but at the current moment they are not needed. To go through each section of code in the future implementing the checks would take more time than just implementing it in the beginning. Now with the class definition and the method returning true you could simply call a method to check if the group or user has access to the page (which at this point will return true) and when you implement the business logistics later there is no need to create a large scale change. Yet again, as I stated prior authorization and authentication may not be a specifically great example here.

You will have likely taken slightly more time but in the end saved a mass amount of time by doing this. You certainly will likely miss some areas with that base check but everything should be golden at this stage and you saved yourself hours of writing the business logic. This certainly doesn’t apply to every feature you think of because that would be madness and that would completely negate the purpose of this post.

A simple way to gauge if you should develop now or later; do all of these apply?

Is this critical to my users success?

Is the application crippled (unusable) with out this feature?

Is the cost to develop it now 50-75% less than it would be in the future?

Is there a business need to support this feature?

I believe these are all critical questions to ask yourself when you are developing, otherwise, you may end up with 20 unfinished projects that will never see the light of the day because the enthusiasm of starting the project has diminished and since there is nothing out the door since it is unfinished there is no community to help build your enthusiasm about the project. At this state you are burnt out, bored and that project may never see the light of day again regardless of how great of an idea it was in the first place or what solutions it may have solved.

I refactor a lot. My experience is that in my mind, things will become much more clear when I have some basic structure in place, so the first thing I’ll do is write myself a small prototype, and extend that over and over until I have the features I want.

Because I work with a framework (symfony in this case), I have the added advantage of being able to install plugins and configure a lot. Things such as authentication are all caught with the sfGuard plugin, and then adding the correct authorization is a matter of editing a configuration file telling which modules can be accessed with which credentials.

so aside from writing OO code, doing it with a framework will speed up development and re-usability even more.

My approach used to be yours, and try to build a semi-framework for every feature.. Later on I realized there was more work in maintaining the abstraction, while in a lot of cases it was unnecessary to begin with..

Writing maintainable code is very important, and design for change is too.. My moniker is now ‘build for refactoring’ instead of ‘build for abstraction’.

I believe it all depends on what you are needing. I do not always follow in the steps of building out certain areas unless I deem them as absolute necessity going forward. I typically do refactoring in much of my code.

For instance, lets take the last personal project I worked on. Phase 1 included only core behavior, while I plan for an administration area in the future, I didn’t build any of it, rather built the components that I will be utilizing for certain sections. Phase 2 is building an administration area and refactoring the code to run off a more centralized area (right now data is simply cached from a third party in files rather than being extensible from a database perspective). Phase 3 is to build the features on top of what was previously there.

Rather, what I speak of is simply to stop over engineering certain aspects. I could have taken phase 2 and said, well it is necessary for the data to be stored in the database right away, when I have no clue of the marketability of the said product. It solves the task but doesn’t completely give all of the functionality that it could have. Meanwhile, the object oriented structure in place to allow modifying the data source with little to no extra effort. Which left an interface for building in the future.

It is both maintainable and extensible to build what I may need on top of the components already built. As developers we will be faced with the decision of what approach is going to be better (build for refactoring or build for abstraction).

About

I am a Software Engineer Manager / Software Engineer working out of Eagan, MN at a non-profit called CaringBridge. I presently organize the MNPHP User Group and MN MySQL User Group here in Minnesota area.

I have been programming with PHP and MySQL for 10+ years. I have the ZCE in PHP and Zend Framework. Most of my development now days is focused on high availability and high performance applications. My resume can be found on LinkedIn.