A system engineer has to do what a system engineer has to do...

Submitted by dag on Sun, 2010/04/04 - 14:38

After reading You may want to avoid hacking your open-source CMS today I was saddened, not because Drupal fell short (it didn't), not because Open Source usage is flawed (it isn't), but because it reminds me of situations I have endured in the past from the receiving end unfortunately.

Summarizing the story, The Onion forked (as in adapted to their own wishes) Drupal 4.7 apparently not realizing the costs of maintaining/syncing their own product. Wrong expectations make up a big part of a disappointment. Good expectations are part of doing your homework.

As a system engineer, I have a hard time understanding how a reputable organization can make a decision to modify existing software without thinking of the implications. If you set up infrastructure for yourself or others, the maintenance cost is an important criteria in making a (any) decision.

I often see a difference in attitudes between typical developers and system engineers, the former focused on the end-product and requirements (and likely a deadline), the latter focused on the end-solution and maintenance. A programmer's job has usually stopped when the system engineer's work has to start.

Sometimes it is frustrating that the long-term effects are overlooked and a piece of software is forced into production without any regard of maintenance (security, support or development) costs.

So a valuable lesson is in this story, if you introduce something new, make sure you have a good grip on what the costs are, also after it is successfully launched. In economics that's the difference between extraordinary costs and recurring costs. Don't leave that up to the marketeer, salesman or developer, maybe involve your system engineer next time ;-)

It is up to The Onion to see if they're better off with Django than Drupal, I guess it depends on how much development is part of their core-business. At least I hope they have thought about the development and maintenance costs this time around. And according to this Reddit thread they know what they are doing and where they are going :-)

I have not looked at Drupal for a while now. Even worse, I stopped using it just after 4.x.

Looking at the code from that time I then already decided it's not the best beast to maintain. Looking at it as both a system administrator and as a developer I made some changes to it resulting in a usage of only about 1/3 of the resources it used.

I did ask some of the developers on IRC why the translation tool was so bloated. I took a complete string and used that to take the translated text, even when it was in the same language. I do hope they altered that now? Reducing this by a few adjustments gave it much more speed. When mentioned on IRC they stated they would not change it just because it worked for them.

As a long time (but somewhat burnt out) Drupal fan, I have a lot of sympathy for the Onion team. I develop and maintain one main Drupal site, but it has been continuously upgraded and developed (by me) using every version since 4.4. I'm in the middle of a Drupal 6 upgrade plus site reworking to include Views, CCK, OG and new themes etc.

As for The Onion, 95% of their coding was properly done as modules, and they only hacked core where necessary for scalability. Lots of big sites had to do that with Drupal back in the 4.5/4.7 days, they were only doing what they had to. We never had to though luckily.

One telling fact that emerged was that their 4.5 to 4.7 upgrade took them the same amount of time as rewriting from scratch with a different language and platform. I think they has decided on the rewrite quite some time ago.

And the API churn since 4.7 has been increasing exponentially. Not only that, but the whole approach to building a Drupal site and what modules to use is also being continually thrown on its head too. It seems like every Drupal upgrade I've ever done has involved migrating off at least one orphaned module.

It was this aspect that the Onion guys alluded to. Moving to Python and Django gives them a much more maintainable and stable platform to code on into the future. They are doing this for well thought out System Engineering reasons.

These days with Views, CCK, Panels, Features, etc etc contrib is now far more complex than core. Contrib is becoming extremely difficult to keep up with.

My current upgrade is being done with an eye to making the Drupal 7 upgrade easy, but after that I'm seriously considering a move to a Python MVC-style framework (maybe Turbogears instead of Django though).

Two important points I'd like to make (based on the relevant Redit(?) thread):

1) The Onion developers weren't seriously considering any PHP CMS for the new site version, because of their general dislike of the language.

2) In 4.x it was much hard to get a site like the Onion working 100% without hacking core. If it was D6 or D7 (and to a lesser extent D5) then it would be inexcusable, but I suspect they didn't have much choice in D4.

As a long time (but somewhat burnt out) Drupal fan, I have a lot of sympathy for the Onion team. I develop and maintain one main Drupal site, but it has been continuously upgraded and developed (by me) using every version since 4.4. I'm in the middle of a Drupal 6 upgrade plus site reworking to include Views, CCK, OG and new themes etc.

As for The Onion, 95% of their coding was properly done as modules, and they only hacked core where necessary for scalability. Lots of big sites had to do that with Drupal back in the 4.5/4.7 days, they were only doing what they had to. We never had to though luckily.

One telling fact that emerged was that their 4.5 to 4.7 upgrade took them the same amount of time as rewriting from scratch with a different language and platform. I think they has decided on the rewrite quite some time ago.

And the API churn since 4.7 has been increasing exponentially. Not only that, but the whole approach to building a Drupal site and what modules to use is also being continually thrown on its head too. It seems like every Drupal upgrade I've ever done has involved migrating off at least one orphaned module.

It was this aspect that the Onion guys alluded to. Moving to Python and Django gives them a much more maintainable and stable platform to code on into the future. They are doing this for well thought out System Engineering reasons.

These days with Views, CCK, Panels, Features, etc etc contrib is now far more complex than core. Contrib is becoming extremely difficult to keep up with.

My current upgrade is being done with an eye to making the Drupal 7 upgrade easy, but after that I'm seriously considering a move to a Python MVC-style framework (maybe Turbogears instead of Django though).