Framework comparison summary

To be able to size up an objective summing image I started to build an e-commerce website in all frameworks on the same model. These are my experiences after two weeks.

Full of surprises

I thought PERL would win gloriously on this test. The results was really impressive but Django did even better. I remember when I first saw the results I couldn’t believe them and run the test again. The other surprise was the really bad performance of PHP. Somehow I expect PHP would be the worst out of the candidates but I didn’t think that difference will be this big. That’s one of the reasons I didn’t test symfony 1.4 eventually. I didn’t see the point.

Installation & coding

Catalyst

In my humble opinion installing Catalyst was really troublesome. A lot of problems occured during the process. If you want to do it from ground zero you should be aware that it can take a lot of time and you should have the nerves to do it through. But once it is installed everything is fine. Developing is easy. Tremendous number of CPAN libraries are out to help you in your daily work. PERL has the benefit from being robust and flexible. You can do whatever you want and however you want almost on every abstraction level. Catalyst is a true alternative for every web developer who wants performance, flexibility, reliability and the power of PERL.

Django

Django installation was like a dream after Catalyst. It took 5 minutes, at most. Create the working site with the imported modules took another 5 minutes. Admin interface worked out of the box. Although I had some problems with imported models I can have them fixed quickly. If I mentioned the flexibility of Catalyst I have to write about the restrictions of Django. In contrast to PERL’s “There’s more than one way to do it” principle there is only one true way to do something in Django properly. Although it’s not really true after two weeks of developing I found myself to recreate algorithms, model definitions and processes to be more “Djangoish”. Whether it’s a good or bad thing everybody decides themselves. At the beginning I cursed a lot but I get used to think the Django way and I like to have standards. I have to admit I can develop functions (it’s important to emphasize not algoritmhs but functions) faster than I ever could in PHP.

Symfony

I’m working with symfony ever since 1.0 beta came out. I loved it especially after the home-made frameworks and giant if built index.phps. I made a whole bunch of sites (e-commerce, cms, business data processing etc.) based on symony 1.0, 1.2 and 1.4. I knew that this was not the most optimal system but learning curve is relatively short comparing to the aboves. And developing was quite fast. The advantage of PHP is beyond doubt when you have a company and have to hire people. Usually PHP developers are cheaper and easier to find than PERL or Python developers. But if I would make this benchmark earlier I would have changed most of my sites to Python or PERL and not made an effort to be able run Symfony sites (with couple of thousand visitors per day) on a single Amazon microinstance.

Documentation

There’s nothing really to complain about. All of them are very well documented. There are a lot of blogs and forums where we can find help if we stuck on a problem.

Plugins

I’m not really go deep in to this subject but as far as I can tell only symfony is the only conventional pluginnable system. The plugin philosophy shaped in accordance with the philosophy of the language. Out and away PERL has more a solution set for common problems not so much plugins. Python has some plugins but these are rather toolkits or full apps than pluggable moduls. I don’t want the judge any of them. I can live with all of them.

Numbers

And after all here are the magic numbers.

Catalyst

Django

Symfony 1.2

Installation

2 hours

5 minutes

10 minutes

Import from DB

10 minutes

1 minute

1 minute

Requests/sec

Higher is better

Index

Show

Index

Show

Index

Show

AB 5 conc

93,7

122,76

123,42

158,78

24,29

23,77

AB 20 conc

94,39

122,18

118,64

167,52

23,81

23,6

AB 50 conc

94,51

120,24

116,92

160,34

23,75

23,67

Resource usage

Lower is better

Memory

11,40%

11,40%

7,80%

6,10%

2,20% (per thread)

2,20% (per thread)

CPU

27,20%

25,10%

3,10%

2,20%

100,00%

100,00%

You might like these too

Solr benchmark – first blood This is a quick impression about the freshly installed Solr 3.5 server.
Enviroment
The base system is a Amazon Microinstance equivalent virtual mach...

About charlesnagy

I'm out of many things mostly automation expert, database specialist, system engineer and software architect with passion towards data, searching it, analyze it, learn from it. I learn by experimenting and this blog is a result of these experiments and some other random thought I have time to time.

Categories

About Charles Nagy

Database specialist

Automation expert

System engineer

Software architect

“An expert is a man who has made all the mistakes which can be made, in a narrow field.” - Niels Bohr

Don't be afraid to fail, to test, to experiment. This is what teaches you things you cannot learn from books. Everything written, told, heard are things somebody already know. If you want to be better in something you have to do things that nobody did before and push those boundaries as much as you can.