PHP: We are getting slow and sluggish, and we’re lazy and arrogant about it.

In my most recent blog, I wrote about how I feel that too much of the world’s logic is coming onto the shoulders of PHP, these days. Today, I’m will be showing you why and how PHP’s powers could be harnessed better and more. We, as PHP web developers, should be absolutely fully aware that we’re allowing insane amounts of processing power to do too much work when it’s absolutely unnecessary to do so dynamically.

Say what now?

Here’s part of the problem: PHP developers usually don’t really know the difference between what’s static and what’s not. Simply put: there is only one variable. The request. Anything else should be as static as possible. If I would introduce a configuration file with loads of variables, a typical C programmer would argue: “Those are build parameters. They don’t change unless your environment changes, so they are static and should be handled by a preprocessor and macros. You don’t need continuous evaluation of static data”. A typical Java programmer would probably put stuff like that in a static final constant property somewhere, knowing (or assuming) that the compiler will optimize and inline usage of these settings 1.

PHP doesn’t do either. We have no preprocessor, and we have no compiler optimizations on that level. Mainly, because PHP originally was a platform that was supposed to do some simple processing and spew out some dynamic content, as fast as possible. But the PHP community is growing up. We want real OO programming, we use design patterns, we think about dependencies, maintainability, scalability. Stuff that grown-ups do.

But then, what the hell is wrong with this picture?!

I was having performance trouble with one of my Symfony projects at work. So I decided to do some performance testing. Please note that I absolutely love Symfony. There is no framework out there that will actually help you design your application better, and it brought me and many of my colleagues more joy in life. Really. Please understand that this isn’t really about Symfony, because I am pretty sure most of it goes for any other current-generation framework.

I set up some benchmarking comparisons. This isn’t scientific evidence, to be frank. This is experimental science. I was having a hunch, and this indicates that my hunch was correct. So here goes.

The example controller in the Symfony standard edition

Whenever you start a Symfony project, you would typically bootstrap from the standard edition, remove the demo code and add your own bundle. The example controller does something pretty simple. It responds to a URL /hello/..., where the dots may contain anything but a forward slash. Here’s the code:

The idea is simple. /hello/{name} routes to this controller, the controller returns an array of named variables, and the @Template annotation makes sure the template corresponding to the controller is rendered. This is usually a twig template, and in this case, the template looks like this:

Hello {{ name }}!

Not just to be a pain in the ass…

How would you have done this 15 years ago? Probably something like this. Create a PHP file in hello/index.php containing:

Right? It doesn’t really do anything more than that. And if I’m absolutely honest about it, useless as the example may be, it shouldn’t be done any other way.

So, just for the sake of curiosity, lets see what a difference in performance this is. I set up two identically configured hosts on my local machine in Apache, but with different webroots, just to be as close as possible to a regular development environment I normally use. The first web root is the regular web/ directory from Symfony (and I renamed it to web-1). The second is another directory containing the setup as described above.

With ‘ab’ (Apache Benchmark), you can get a pretty good idea of performance increases if you’re optimizing your website, one way or another. You can basically pass in an URL and have multiple concurrent users hammer that URL for a specified number of times. So that’s what I did. I usually start with 1000 requests, with 20 concurrent users. This usually gives a good comparable impression of speed.

First off, the dev, debugging version of the symfony app. Of course, you wouldn’t normally test performance in a development or debugging environment, but it’s here for the sake of comparison. Here’s the most relevant2 results of ‘ab’.

Wait. What? Over 5500 requests handled each second? That means a performance increase of 30. Not 30%, a FACTOR of 30. Nearly 30 times faster. The last time I saw numbers like these were when I ran benchmarks on a Varnish cache….

This would mean that if I were to run the same test with 30 times as much requests, the 1998 version should manage equally:

Let’s look at this for a few seconds. Just to be sure we understand what this means. This means the stripped version of the functionality runs roughly a factor 30 faster than the one implemented using the framework, both with 20 and 250 concurrent users.

But if these numbers are possible, why then, why do we think all the framework’s benefits outweigh these mind-boggling disadvantages?

The counter-arguments

You have no request listeners and therefore no security, no logging, etc, etc…

I know. And you are right. I don’t. But none of those are needed in this example. And that’s exactly what’s wrong with our mindset. We load tons of utilities and tons of logic to requests of which in 80% of the cases, only 20% actually need, and we keep on saying that the ease of use and the well-thought design are good arguments to have such a horrendous performance impact.

Be honest. If you would buy a new server, would you rather have it installed bare bone without any software on it so you can carefully pick your stuff, or would you have the entire Ubuntu Software center installed, so you can use it whenever you can?

Sure, there is a middle road, but lets find a middle road that is the best, not the one that is the most convenient just for us developers.

But you can cache it!

There is only one right answer to such a suggestion. Caching sucks3. Yes, I know, caching is cool, because it can make stuff insanely fast (try Varnish one time. Unsavory goo will drip from your lips, I promise you). But that’s not the point, really. Caching complicates stuff. Complications are just like expensive toys. 1) You don’t really need them, 2) they make you worry about breaking things that shouldn’t really matter and 3) you already have enough of them.

So we should install php3 again and get rid of the frameworks?

Balls, no! This is something the people behind the frameworks should be aware of. But at the very least, the people using them should be aware of it. Don’t tell your project manager (or worse, your client) that the performance is okay, and that you just need a few days R&D to implement caching, just tell them the framework is relatively slow but that you think the extra cost in hardware and complexity of caching mechanisms outweigh the design and maintainability benefits. And be very careful to really mean it and know what you’re talking about.

Code generation is paramount to performance!

I’m not the type of guy to point out problems and not think of solutions beforehand. I have been thinking about this for quite some time, and thought of building something myself. But the fact of the matter is: I don’t have the time nor the persuasive character to actually get it done.

What would be needed is a build system that “folds out” into smaller pieces of somewhat repetitive PHP. We write the development code just like we did before, but when deploying to a non-development environment, all the bits and pieces should be as static as possible. Anything that is dynamic but could, in theory, be static, must be factored out. This means, in case of Symfony, that

It is a bad practice (or even unsupported) to have dynamics inside your Bundle classes. Use the service container for this. This way, the bundles needn’t be loaded nor initialized on each request;

The service container and accompanying files should be loaded lazily. This can be achieved using require_once calls whenever necessary. The compiled code should do the same.

Service container and compiled DI code (we have this already, but could even be more effective) compiles into separate files in stead of a big class that gets loaded everytime.

Configuration must be compiled statically

Templating must be optimized such that it generates the most efficient PHP code possible. Using magic like the detection of ‘getters’ and such should be removed. May be even consider it a bad practice to use objects in templates at all.

Routing must be done statically. Dynamic routing is runtime logic and can be done inside the controllers.

Request listener code is compiled into the controller files, so templating listeners, error handling and security are as efficient as possible.

Resolution of bundle aliases is done at compile time

Handling sub requests is done by simple includes

And on goes the list.

And now, for some action

I have built a little “proof of concept” that shows you the idea. I wrote the resulting code by hand, but with most of the logic already in place, a compiler with some optimization passes in the resulting AST could do anything that i did by hand automatically.

Here’s the performance impact difference:

The ‘hello {name}‘ example:

Standard Symfony

Optimized version

188 rps

1640 rps

An example using doctrine

In the second example, (which is configured at route db/{name}), the entire service container is loaded, by including the appProdProjectContainer from the cache. In the last example, routed through db-optim/{name}, the doctrine service and it’s dependencies are loaded by including files with their service definition. This adds another ~10% increase in performance, which is still worth considering, imho.

Note that the doctrine services only use per-request caching (ArrayCache), in this example.

What’s the catch?

The catch is, most of the overhead from Symfony’s kernel is from the request listeners. There are a whopping 15 (fifteen) services that listen to every request in Symfony. All of these listeners can be optimized into pieces of code which are included in the controller files, where the “listening logic” (i.e., checking if the listener should do anything) can be done by a static line of code. In the examples above, you can see this happening for the templating listener, which is only checking if the return value of the controller is an array. The fact that it’s in place right there, can be controlled by the @Template annotation. The same would fly for @Secure and @Route annotations. Some listeners contain global static logic (such as the locale listener) or are generic, but nonetheless static (such as exception listeners.

With the Kernel structure as it is now, there is no way to have these listeners come into play only when needed. They are initialized every request, and therefore all dependencies that these listeners may have, are initialized. There is only one way to get around this. Compile all initialization into the controller files, but guarded within the conditions these listeners should provide. Just like the is_array() example for the logic of the Templating listener, this can be done for any other listener.

The sharp reader must have noticed that I disabled all listener logic in my example. This is the reason.

To conclude

I have posted before about a paradigm shift towards declarative programming. I am also arguing that this shift should include a better sense of balance between performance and maintainability. PHP’s road to success was paved by performance. If we lose that, there is not much reason left not to choose other platforms.

A final disclaimer: I am not bashing. I hope that this will lead to more good in the PHP community. At the very least, I dropped my thoughts and can leave it simmering for now. If you read the post entirely, thanks for your patience.

I’m not sure if the Java compiler works this way, but I’m just assuming that it does some kind of optimization for constant values. ↩

I am using the following command line to do these tests: ab -n 1000 -c 20 http://the/url/ | egrep '^(Time taken|Concur|Requests per|Complete)'. The regular output of ab holds a lot more statistical information. I am performing these tests locally, so they vary a lot; I have of course some other programs running which interfere with the performance. I am aware of that. The numbers don’t really matter that much, but the relative differences do. ↩

Cache invalidation is one of the two hard things in computer science. The other one is naming things and off-by-one errors. ↩

9 Comments

Although I believe I was the one telling you about Varnish Cache and the fun things you can do with ESI, I totally agree that it mainly solves most developers’ preference not to benchmark and optimize. Your benchmarks don’t really surprise me, as most frameworks are bloated.

Though code generation and pre-processing can be seen as a form of caching, these are actually desired types of caching when the generation of the cache is done during code deployment and is otherwise untouched. This circumvents the cache invalidation issue.

As a matter of fact I once created a proof of concept using the C pre-processor (CPP) from the GNU compiler collection (GCC) to real-time pre-process PHP scripts. I’m pretty sure you remember. This is too crude and not suitable enough for enterprise level applications, but it’s a start. I firmly believe frameworks should be able to output less beautiful, less object oriented code and instead generate procedural code with kick-ass performance.

This is best of both worlds: nice code in your source control system (as you don’t add generated code) yet performance when you need it. Basically you could have all debug code, comments and undesired profiling routines stripped from the deployed code, like most compilers (optionally) do. One of the goals would be to reduce the number of IOPS, by reducing includes and eliminating templates, and other operations that beautify applications yet destroy performance. For C we have a lot of tools like automake, autoconf, libtool, flex, bison, etc. to make development easier, while none of these tools are required on a server to run binaries compiled using them.

TL;DR: frameworks should generate efficient, probably procedural code to do the actual work.

I did know Varnish before you told me about it, but you know I value your judgment and it possibly cleared any lingering doubts. I also do recall your ideas of preprocessing PHP, and they still haunt me ever since 😉 The idea of PHP burning cpu cycles and IO when it is in fact a check we know the outcome of beforehand gives me sleepless nights…

Very interesting post. A good software architect chooses the right tools, based on the challenge right ?

In some of the projects we use symfony for we have specific actions (API methods, Script/XHR requests,…) that skip the framework and directly go to PHP (without framework), similar to the example you gave (although sanitising the input and adding some more logging ;). I guess that gives us best of both worlds. Performance + RAD.

Great article and I do agree with the basic thrust of it. What concerns me is the way PHP is evolving. It is shaping up to become object oriented, which is not the original intent of the templating language it used to be. One of the crazy things about PHP is the number of templating sub-languages available. That’s a layer of fat that is not needed and this is why I have taken up writing a procedural framework with no MVC. Just commonsense and programming the way it used to be. I am just wondering at what version PHP will become OOO – Object Oriented Only. I dare say that 90% of web application do not warrant using OOP. If they do then it should not be with PHP. Time to create a new language?

I seriously love your site.. Excellent colors & theme.
Did you make this site yourself? Please reply back as I’m
trying to create my own personal site and would love to find out where you got this from or exactly what the
theme is named. Many thanks!

It’s a pity you don’t have a donate button! I’d definitely donate to this
excellent blog! I guess for now i’ll settle for book-marking
and adding your RSS feed to my Google account. I look forward to brand
new updates and will talk about this blog with my Facebook group.

Engineers have come up with a solution to resolve these issues with a help of
mobile software that will act as a mobile spy to monitor all the activities in a particular mobile phone.
You can use a free of charge telephone tracker app but they
are really effortless to detect and do not do close to as
very much as this app does. Right after all, it really is your corporation that feeds and outfits your relatives and puts a ceiling over their mind.