Featured Post

Games generate more data then an average application because of the game state machine . Terabytes of data can be accumulated in a short pe...

Wednesday, September 10, 2008

When The Agile Development Process Does Not Address The Business Model

I recently built a development team and development process around a very aggressive business model. The model could be described as a “marketing” centric model as opposed to a “product” model. In the midst of the organization and infrastructure building exercise I explored various development processes including XP, RUP and Agile. During the later stages of the organization’s growth I focused specifically on Agile to address some of the quality, speed to market and communication issues the organization was experiencing. After a lot of study and some trial and error it became obvious that Agile development was not the right fit. Much of this misalignment resulted from the forces being placed on the development organization by the business model and operational expectations associated with this model.

Short Cycle Development – The organization had an implicit expectation that changes to the production system were required on a daily basis. This translated into several pushes to production in a 24 hour period. Often product changes were pushed on Fridays and over the weekend.

Detailed Specification Requirements – The business owners wanted to know specifically what each feature would look like and were not interested in releasing “iterative” versions to incrementally add functionality to the application. This is directly correlated with the short cycle expectations and releasing product on a daily basis. With short cycles the product was difficult to break down into even smaller iterative bytes.

Quality Control - Quality was enforced by a QA department that worked from a detailed specification. Quality checking and an understanding of the change came after the initial development cycle using the specification as a guideline. Thus, the "waterfall" process at the backend of the process.

Aversion To Infrastructure Investments - The notion of investing in building a unit testing infrastructure was not a high priority. It was important to get product out the door so any investment in development resources not focused on direct product releases was discouraged. Also, the unit test infrastructure would have been best built at the initial outset of establishing the applications architecture. Coming back retroactively and adding this to the platform would have been very difficult.

Organizational Structure – This organization was a startup and leveraged individuals over a number of different projects for the purpose of preserving capital making it difficult to form true product teams where QA, PM and Development collaborated in defining, developing and testing the application.

The Scrum – It was hard to form a real scrum for a single release or product. The scrums turned out to be product update meetings on all outstanding projects. These were good meetings but not true “scrum” sessions. They were also hard to pull together because of the waterfall culture of the company. People had a tendency to meet and strategize within their organizations rather then cross organizationally.

In the end the development process evolved organically from the culture. It turned out to be a hybrid of RUP using Use Cases to cover usage models. Waterfall process was used to move the products through the cycle. The PM’s would define and build requirements; Development would build and submit to QA for testing and eventual release.

Was this the optimum process for this organization? Not really, releases were not always released on time. Some of this was due to the extreme need for speed leading to mistakes in specifying the features. Quality remained an issue because of the pressure to release and to the general lack of understanding of the impact of a change on the larger system. On the other hand rapid a short cycle release culture was institutionalized allowing for a rate of change to the production system that was relatively very impressive. Relative meaning, in relation to other development processes that dictate a more scheduled and less frequent release cycle.

In retrospect, one of the biggest mistakes made was to not build a unit test infrastructure early on in the architecture and design of the system. This would have allowed for a higher quality release to production while still maintaining a short development cycle culture.

2 comments:

I think your situation, from the short description, is a great example of how the product development process needs to involve changes from the business side, in tandem with development. Agile (or RUP or planned) processes require a team approach, rather than a siloed approach.

I don't think you can adopt a process and change the culture. I think you need to work on the culture first.

Is there a small product area in which you could start with a small group (product owner, 2 developers, test engineer) and through a series of small releases, demonstrate the value of an agile process, good software construction practices (unit tests, continuous integration, etc), and an integrated team?

John,You are spot on with this comment. The parts of the organization outside of the engineering group had no real interest in conforming to a development process and methodology. They viewed themselves as individuals requesting products and services and not participants in the process. It is amazing how a company's culture can cause a process to falter and only be partially implemented.