I have a multi-process PHP (CLI) application that runs continuously. I am trying to optimize the memory usage because the amount of memory used by each process limits the number of forks that I can run at any given time (since I have a finite amount of memory available). I have tried several approaches. For example, following the advice given by preinheimer, I re-compiled PHP, disabling all extensions and then re-enabling only those needed for my application (mysql, curl, pcntl, posix, and json). This, however, did not reduce the memory usage. It actually increased slightly.

I am nearly ready to abandon the multi-process approach, but I am making a last ditch effort to see if anyone else has any better ideas on how to reduce memory usage. I will post my alternative approach, which involves significant refactoring of my application, below.

Many thanks in advance to anyone who can help me tackle this challenge!

Although this is not really an answer to your question but a suggestion to have look: Have you had a look at php hiphop? php hiphop recompiles your php into c++. Eventually this could work for you but this depends on how you wrote the code and what you use for functions. I've no idea if it works with pcntl but it could be a huge increase of speed and memory usage as of my experience.
–
Julian HilleFeb 22 '13 at 8:52

Yes, I have actually. I came across its existence while researching this problem. The main reason I haven't tried HipHop is just because I have no experience with it (not a good reason, I know). It's a good suggestion, and I think it could form a good answer to this question. If you have experience, would you like to formulate some thoughts into an answer?
–
robguinnessFeb 22 '13 at 9:11

1 Answer
1

Mutli-process PHP applications (e.g. an application that forks itself using pcntl_fork()) are inherently inefficient in terms of memory because each child process loads an entire copy of the php executable into memory. This can easily equate to 10 MB of memory per process or more (depending on the application). Compiling extensions as shared libraries, in theory, should reduce the memory footprint, but I have had limited success with this (actually, my attempts at this made the memory usage worse for some unknown reason).

A better approach is to use multi-threading. In this approach, the application resides in a single process, but multiple actions can be performed concurrently* in separate threads (i.e. multi-tasking). Traditionally PHP has not been ideal for multi-threaded applications, but recently some new extensions have made multi-threading in PHP more feasible. See for example, this answer to a question about multithreading in PHP (whose accepted answer is rather outdated).

For the above problem, I plan to refactor my application into a multi-theaded one using pthreads. This requires a significant amount of modifications, but it will (hopefully) result in a much more efficient overall architecture for the application. I will update this answer as I proceed and offer some re-factoring examples for anyone else who would like to do something similar. Others feel free to provide feedback and also update this answer with code examples!

*Footnote about concurrence: Unless one has a multi-core machine, the actions will not actually be performed concurrently. But they will be scheduled to run on the CPU in different small time slices. From the user perspective, they will appear to run concurrently.