I have a VPS (with 256MB of memory) that has hung more than once today. It just froze entirely, and was not accessible via SSH. I had to run my provider's "force power off" procedure to bring it back. Then the server lasted for a few hours before doing it again.

The first time there was a message about running out of memory and attempting to kill the mysqld process. The next I managed to find this in the syslog file:

I've configured Apache and MySQL to be fairly mindful of the low RAM, at least to my knowledge, and I'd like to know why this is happening. Do I simply need to upgrade, or is there something wrong?

Here is an excerpt from my Apache config:

#
# Timeout: The number of seconds before receives and sends time out.
#
Timeout 30
#
# KeepAlive: Whether or not to allow persistent connections (more than
# one request per connection). Set to "Off" to deactivate.
#
KeepAlive On
#
# MaxKeepAliveRequests: The maximum number of requests to allow
# during a persistent connection. Set to 0 to allow an unlimited amount.
# We recommend you leave this number high, for maximum performance.
#
MaxKeepAliveRequests 200
#
# KeepAliveTimeout: Number of seconds to wait for the next request from the
# same client on the same connection.
#
KeepAliveTimeout 3
##
## Server-Pool Size Regulation (MPM specific)
##
# prefork MPM
# StartServers: number of server processes to start
# MinSpareServers: minimum number of server processes which are kept spare
# MaxSpareServers: maximum number of server processes which are kept spare
# MaxClients: maximum number of server processes allowed to start
# MaxRequestsPerChild: maximum number of requests a server process serves
<IfModule mpm_prefork_module>
StartServers 1
MinSpareServers 1
MaxSpareServers 3
MaxClients 50
MaxRequestsPerChild 1000
</IfModule>

4 Answers
4

You really need to trim up your services. Better yet, you might want to consider using inetd or xinetd (whichever applies to your installation). Rather than leave several services launched and waiting (and hogging precious RAM) you might be better with a CPU performance hit from launching a service when it's needed. And that's what inetd/xinetd were really meant for - they were intended to provide a low-memory way to allow services to launch on demand, rather than keep them hanging around all the time. Of course, your mileage may very - you can still end up with memory resource issues - but if it gets you over the rough spot for the time being...

Follow-up on where to start (from comment):

I would look at getting apache moved over first. Reduce the maximum child count to just 2 - unless you are servicing more users than yourself, it helps little to have 3 running processes.

For dovecot, specifically the imap-auth processes, you only need one. You could run it in inetd as well, although it appears to be fairly thin. If you don't use IMAP or POP on your server, turn it off. Worst case, you're back to reading mail in a command line - not pretty but functional.

You can also move sshd, although I would initially do it at an alternative port number and confirm it connects before you flip the switch on that one.

Consider moving to Exim if possible (which is the default install for Debian/Ubuntu setups). I love Postfix and I don't use Exim, but Debian chose it as the default for a reason - it requires little resources and is fairly secure.

Look at your /proc/sys/vm/swappiness setting (echo /proc/sys/vm/swappiness) and determine if it's too high or too low. A higher number will help to free up memory (by swapping to VM ahead of time) but when it's too high you'll swap like mad. Most installs say "60" but for low memory, it should probably be more like 85. I don't recommend 100 in your setup.

Lastly, think "green screen". You're talking about using techniques that have been around for a long time on much older systems (because they were resources constrained also). It's a bit old-timey to do things this way but if you can get your work done in a client program in a shell, rather than launching services, you'll stretch your computing resources that much further.

Any recommendations on what to prune out? Quite a few of them were included on install, and I haven't researched too much into what can be safely turned-off yet.
–
redwall_hpSep 23 '09 at 4:46

If it supports inetd, then get it to run under inetd. Simply put, unless it's a login shell, you can get just about everything running there. I think you can even get sshd to launch via inetd (although I would move it to port 2222 or 222 or something for safety, until you confirm it works correctly - that and it cuts down on the ssh-worm noise you'll get in your logs).
–
Avery PayneSep 23 '09 at 5:06

I'll look into moving some of the processes to inetd. (I'm running on Ubuntu/Debian, which I believe has inetd.) I'm fairly new, experience-wise, to Linux server admin (though I'm comfortable working with the OS, and have been experimenting with it for a few years). And, yes, I do run IMAP. I had installed Postfix and Dovecot myself, partly due to their easy configuration. Thanks for the help.
–
redwall_hpSep 23 '09 at 20:38

Keep an eye in the number of httpd processes that are running and how much memory they are using. I think your web server is swapping, maybe because the MaxClients directive is exceeding the optimal value for your server. You need to calculate average RAM you need for each apache process (according the results of ps is around 30M) and set MaxClients with a value that not exceeds the total available RAM with enough room for other processes like MySQL.

If you type the command 'free' when the server begins to slow down, you will see the memory usage and I am willing to bet, as HD said, that your server is hitting up the swap space. Going by your PS output, you are almost at 256M there.

You may want to move MaxSpareServers down to 2 or even 1. Also, there are alternatives to Apache, for example lighttpd which is supposed to be lightweight and use less memory.

I have a personal opinion, and I'm sure some will disagree, that you shouldn't have less than 512M or 1G ram in a server.

1. I run free now and then when simply to keep an eye on things. But with the advent of the hanging, I haven't been around to notice it slowing and run a check. 2. I've heard that Lighttpd has issues with memory leaks in some cases. Is that still true? 3. I've been hoping that I could get by for awhile at least without upgrading to 512MB of RAM, but I am definitely aware that I simply won't be able to avoid it at one point or another.
–
redwall_hpSep 23 '09 at 4:10

Running Apache, MySQL and probably PHP in a 256 MB VPS is kind of a tight game. 256 MB RAM is not quite much for running those services, on the contrary, it is very, very little. First thing to do would considering if not getting a better VPS is an option.

Then, if getting a better VPS is not an option, one serious word of advice: ditch Apache completely!

Apache has become quite a memory hog those days and since you don't have the luxury of free memory in masses, Apache really, really should go.

Take Nginx instead, which is also quite powerful, but takes much less ressources to be able to deliver!

And of course in case of using PHP, you need to install PHP-FPM in conjunction with Nginx then and you need to configure your pool properly.

You might also want to fine tune the various caches in MySQL, meaning probably reducing their sizes!