500 errors (503, etc.) on WordPress

[I spoke too soon! – this was NOT the “SOLUTION: I’ve moved to managed SQL on dreamhost and all issues with 400 adn 500 errors have disappeared. Cost $15 per month PLUS cost of the general server. But it is worthwhile, since I’m at least able to DO something.” I’ve had to think again]

I apologize for the delay and for the inconvenience. I’m sorry that you
are encountering these issues. It would appear that the sites that are
running under your user “xxx” are occasionally hitting our shared
server’s memory limits. This is why your site is seemingly slow, and the
500 errors are directly related as well. Here is an excerpt from the log
kept by our Process Watcher, the daemon that is killing your troublesome
php5.cgi processes:

Procwatch is a daemon that runs constantly on shared servers to monitor
the usage of RAM/CPU and execution time so that no single user can use an
inappropriately high percentage of the shared resources and impact the
overall health of the server or the server’s ability to serve all users’
pages.

Now, it’s important to get these errors fixed to keep your sites running
with optimum speed, as well as keep our servers happy. Unfortunately it
can often be a bit of a trial-and-error procedure, but I will provide as
much information as I can to aide you in this process.

Firstly, it should be noted that any sites running under your user can
contribute to the problems you are encountering. For example, if site A
is running processes that are taking up 85% of your individual limit, and
site B tries to run a concurrent process that takes up 20% of the memory
cap, site B will have it’s process killed since it is the one that took
you above your limits, even though site A’s script is the obvious memory
hog… For this reason, it’s important to keep all of your user’s sites
optimized and running smoothly.

There are quite a few things that can contribute to high memory
consumption on any given site, but I will mention the most common
causes…

Plugins – Especially third-party plugins can often be poorly written and
run up the memory consumption. I often recommend disabling all
non-critical plugins and see if the problem gets better, then enabling
them one-by-one until you are able to identify one that causes problems.

Database related overhead, or an unnecessary amount of database queries –
If you are trying to query your database for thousands of results at a
time, this can also cause issues… Try to keep all database sizes and
queries to a minimum by deleting irrelevant or old information from your
database. You might also try optimizing tables in your databases by
visiting your database through PHPMyAdmin. Simply use the “Check tables
having overhead” link located directly underneath your tables and then
“With Selected:”, choose “Optimize Table”. This might reduce overhead in
your database, which is basically a lot of empty and redundant space that
can slow your queries down.

It is also possible that a spike in traffic, or a heavy run by a site
indexer (such as the GoogleBot), or even abusive hits by a given IP
address, could cause this on a short term basis, so you might want to
inspect your access logs for such activity (you can often manage this
problem by implementing a robots.txt or by blocking abusive IP addresses
with .htaccess).

You might as well want to try looking over the suggestions offered at our
wiki pages on this matter:

Implementing the above suggestions more often than not will fix the
ProcWatch errors, but if you are still finding that your sites encounter
the same problems, you might want to consider moving to a VPS
(http://wiki.dreamhost.com/DreamHost_PS) or dedicated server where you
can reserve sufficient RAM for your own processes to run!

I am sorry if this news is discouraging, but I felt it was important that
you have the full benefit of knowing what is occurring here so you can
make the appropriate decision to meet your needs for this site.

If you have any other questions, or if I can help you further, please let
me know! I hope you have a great day!