There seem to be more of these when the load is higher (2-3) during backups, and I even managed to replicate this during a load of 3 when tar / mysqldump was running during a backup (the user sees a 500 error message after 30 seconds). Could the server be overloaded?
This question seems to be related PHP + Fcgid hangs if download interrupted but not the same.

This is a top notch server, and I am surprised this would be too much. Here are some specs:
6-7 Drupal sites with Webmin

Set the timeouts higher. This has to be done for every vhost (don't forget SSL!), as this setting gets changed every time another vhost loads and will remain until the spawned process dies.
Easiest way would be to edit /etc/apache2/mods-available/fcgid.conf. This is what we are using:

Thanks, I checked and MaxRequestLen is set to 1073741824 Also, if I would need to increase IPCCommTimeout to 600 for my app that would be a hell of a lot of time. This should be max 5-10 secs no? :)
– giorgio79Apr 23 '12 at 17:27

Of course, you should only increase slightly and have a look whether it fixes your problem. Normal values are between 20 and 40 (s). We use the 600 s for a Magento installation, which has to do extremely load-expensive operations. However, you should have an extra eye on it, if these timeouts occur while you are not mysqldumping.
– SebiFApr 24 '12 at 12:02

mysqldump by default will write lock the database while it runs so the data isn't altered during the backup, which can cause corruption. Drupal writes to the database on every request, so requests will hang while mysqldump is running, and eventually timeout.

If you're using InnoDB (or can convert to it), then you can use Percona XtraBackup to do hot backups. Short of that, don't pipe mysqldump to tar or gzip; dump the .sql file and then run tar/gzip on it after mysqldump has completed (and released the lock).